PartitionedStep¶
Apply a set of steps to different labeled subsets of the parameters, using a separately provided labeling dictionary, yielding a composite update step.
Using this node involves grouping your parameters into different groups, each identified by a specific group label. First, you list your group labels in the label0 to label9 entries. Then, you wire different steps (e.g., Adam, SGD, nothing, etc) into the corresponding step0 to step9 ports to configure which step shall run on which group of parameters. And finally you need to describe what parameters of your model, which are assumed to live in a parameters dictionary, belong to which group. This is done by specifying a labeling dictionary that has the same form as your model parameters dictionary, but instead of the actual tensors it contains the group labels (see help for that setting for more details). The update step will then run the specific labeled steps on parameters with the corresponding labels. Note that, if you want to run the same step on a heterogeneous data structure (e.g., nested dictionary) of parameters, you do not need this node since all step nodes already work on nested data structures. Version 0.2.0
Ports/Properties¶
gradients¶
Gradients to be transformed.
weights¶
Optional current weights.
state¶
Explicit state of the node.
labeling¶
A nested data structure (dictionary) of labels for individual parameters. This parameter can be used to assign different groups of parameters in your model different labels, by mirroring the data structure that holds your model parameter, but containing strings rather than the actual weight tensors. These labels are then used to select which of the wired-in steps to apply to which parameters. For example, if your model weights are stored in a dictionary with keys {'layer0', 'layer1', 'layer2', 'layer3'}, and you want to apply step0 to only the weights under 'layer1' and 'layer3', and step1 to the weights under 'layer0' and 'layer0', you might set the labeling port to {'layer0': 'a', 'layer1': 'b', 'layer2': 'a', 'layer3': 'b'}, and then set the label0 and label1 ports to 'b' and 'a' respectively. If your parameters dictionary contains a subtree and you want to assign the same label to all parameters in it, you can simply omit the subtree and instead of it include a string label in your labeling dict (i.e., the labeling can be a prefix tree, not necessarily a full tree). You can also wire the output of CreateDict into this port to generate the dictionary programmatically.
label1¶
Apply step1 to parameters with this label.
step1¶
Step 1.
label2¶
Apply step2 to parameters with this label.
step2¶
Step 2.
label3¶
Apply step3 to parameters with this label.
step3¶
Step 3.
label4¶
Apply step4 to parameters with this label.
step4¶
Step 4.
label5¶
Apply step5 to parameters with this label.
step5¶
Step 5.
label6¶
Apply step6 to parameters with this label.
step6¶
Step 6.
label7¶
Apply step7 to parameters with this label.
step7¶
Step 7.
label8¶
Apply step8 to parameters with this label.
step8¶
Step 8.
label9¶
Apply step9 to parameters with this label.
labelN¶
Additional labels corresponding to additional steps.
step9¶
Step 9.
stepN¶
Additional Steps.
label0¶
Apply step0 to parameters with this label.
step0¶
Step 0.
set_breakpoint¶
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.
metadata¶
User-definable meta-data associated with the node. Usually reserved for technical purposes.