CustomSchedule¶
A custom schedule that is defined by a graph function that takes in a step count and returns a parameter value.
A simple example is a Placeholder node with (slot)name "step" followed by a "Scale by Constant Factor" (Scaling) node that scales the step count by some factor, e.g., -0.01, and an "Add Constant Value" (Shift) node, which adds an offset (e.g., 1.0). Typically this would then be followed by a clipping node that ensures that the parameter value remains in a specified limit range, e.g., [0, 1]. This could be accomplished by ending the graph with a Clamp node. Finally you then wire that node's "This" output port into the "graph" port of the CustomSchedule node to complete the setup. The effect of the step_multiplier is to effectively stretch out or compress the schedule; the graph will effectively receive an adjusted step count that is divided by the step_multiplier. Version 0.5.0
Ports/Properties¶
step¶
Current step (iteration) count.
value¶
Schedule value at current step count.
graph¶
Schedule function.
graph__signature¶
Signature for the "graph" input. This represents the signature for the subgraph that is wired into the "graph" port. This is formatted as in (a,b,c) where a,b,c are names of placeholders that are expected in the subgraph that goes into the "graph" port. Alternatively, it can also be provided in data structure form as a list of lists, as in: [['a','b','c']].
set_breakpoint¶
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.
metadata¶
User-definable meta-data associated with the node. Usually reserved for technical purposes.
step_multiplier¶
Multiplier for the step count. This value is multiplied with each of the step counts to uniformly speed up or slow down the schedule through a single parameter. When used to define an optimizer used by the DeepModel node, this can also be set to 0.0, in which case the multiplier is chosen such that the schedule reaches its final value at the end of the training process, but note that this is not always possible, namely for schedules that are never reach a final value. Otherwise, to make a schedule dependent on the number of steps done by a node, you may normalize your schedule to eg 1000 steps and then wire a formula that calculates the steps done by some process divided by 1000 into this node.