# Interface of policy¶

Note

## About policy¶

Policies give you fine-grained control over the training process. This example demonstrates how policies work and how you can create your own policies.

### Parameter Spaces and Phases¶

Parameter spaces like linspace and cosinespace are the basic building blocks.

from poutyne import linspace, cosinespace


You can define the space and iterate over them:

space = linspace(1, 0, 3)
for i in space:
print(i) space = cosinespace(1, 0, 5)
for i in space:
print(i) You can use the space and create a phase with them:

from poutyne import Phase

phase = Phase(lr=linspace(0, 1, 3))

# and iterate
for d in phase:
print(d) You can also visualize your phase:

import matplotlib.pyplot as plt
phase.plot("lr"); Phases can have multiple parameters:

phase = Phase(
lr=linspace(0, 1, 10),
momentum=cosinespace(.99, .9, 10),
)

phase.plot("lr");
phase.plot("momentum") ### Visualize Different Phases¶

steps = 100

fig, ax = plt.subplots()
# Constant value
Phase(lr=linspace(.7, .7, steps)).plot(ax=ax)
# Linear
Phase(lr=linspace(0, 1, steps)).plot(ax=ax)
# Cosine
Phase(lr=cosinespace(1, 0, steps)).plot(ax=ax); ### Visualize Multiple Parameters in One Phase¶

steps = 100
phase = Phase(lr=linspace(1, 0.5, steps), momentum=cosinespace(.8, 1, steps))

fig, axes = plt.subplots(1, 2, figsize=(12, 3))
phase.plot("lr", ax=axes)
phase.plot("momentum", ax=axes); ## Build Complex Policies From Basic Phases¶

You can build complex optimizer policies by chaining phases together:

from poutyne import OptimizerPolicy

policy = OptimizerPolicy([
Phase(lr=linspace(0, 1, 100)),
Phase(lr=cosinespace(1, 0, 200)),
Phase(lr=linspace(0, .5, 100)),
Phase(lr=linspace(.5, .1, 300)),
])

policy.plot(); ### Use Already Defined Complex Policies¶

It’s easy to build your own policies, but Poutyne contains some pre-defined phases.

from poutyne import sgdr_phases

# build them manually
policy = OptimizerPolicy([
Phase(lr=cosinespace(1, 0, 200)),
Phase(lr=cosinespace(1, 0, 400)),
Phase(lr=cosinespace(1, 0, 800)),
])
policy.plot()

# or use the pre-defined one
policy = OptimizerPolicy(sgdr_phases(base_cycle_length=200, cycles=3, cycle_mult=2))
policy.plot(); Pre-defined ones are just a list phases:

sgdr_phases(base_cycle_length=200, cycles=3, cycle_mult=2) Here is the one-cycle policy:

from poutyne import one_cycle_phases

tp = OptimizerPolicy(one_cycle_phases(steps=500))
tp.plot("lr")
tp.plot("momentum"); 