RecurrentLoop¶
Loop a graph representing a recurrent neural network across an input array (e.g
., a time series). This treats one of the axes in the given input data as the "traversal" axis (e.g., time) along which to step, and applies the the loop body (i.e., the network) to the successive elements of the data along that axis, while carrying over the network activations of any recurrent layers contained in the loop body to the next iteration. This node can be thought of as a "layer" that sweeps across some traversal axis in the input, while carrying over recurrent state across elements. Alternatively, the node can be viewed as a convenience variant of the collecting Fold Loop node (FoldCollect, see documentation), where the loop body is the network, and the state is the carry state of all the recurrent nodes contained in it. In practice, this node is indeed equivalent to such a Fold loop, where the initial state is a dictionary of the initial states of all the recurrent nodes in the body (keyed by the layername of each of the nodes, which must therefore be unique), and the graph has an implicit extra "state" input placeholder taking in that dictionary, splitting it using BreakStructure, and wiring the states to the respective recurrent nodes' carry input. Likewise, the nodes' carry outputs are wired into a CreateDict node, which forms the output state, and the body is made to return a 2-element list of that state dictionary and the regular result of the loop body. Therefore, this node could be replaced by such a setup without loss of functionality, and that can in some cases be useful for additional control, for example if you want to loop other state across the traversal axis, or your graph contains nested recurrent loops, or uses network submodules in it (which are currently not supported by this node). More Info... Version 1.0.0
Ports/Properties¶
net¶
Recurrent network.
net__signature¶
Arguments accepted by the recurrent network. If you wish to loop over multiple pieces of data (e.g., arrays, packets) simultaneously (which all must have the same length), you can list additional items here, using any name of your choosing. A recurrent network is specified in NeuroPype quite similarly to a loop body in, e.g., a For Each loop. You start with a Placeholder node whose slotname matches the name listed here, and then follow that with one or more successive NN nodes, which may include any of the recurrent nodes (e.g., GatedRecurrentUnitLayer, LongShortTermMemoryLayer), but you can also use non-recurrent layers, other mathematical operations, or stateless normalizations such as LayerNorm. Conceptually, this loop body is then given the content of a single slice of your data being looped over (along those chosen axis, e.g., time) and generates some outputs for that slice. When the loop body is called on the next slice, the RecurrentLoop node will implicitly pass in the carry state of any recurrent nodes contained in the body from the previous loop iteration. This is a convenience feature that is exactly equivalent to using a FoldCollect node and manually threading through the state of each recurrent layer. The initial state is obtained by running the loop body once on the first slice of the data, in which case each recurrent layer will emit its initial state over the respective carry output (therefore, the manual equivalent would be to manually perform this pre-pass and then constructing a dictionary of initial states from the carry outputs of the loop body's recurrent nodes).
data1¶
Dataset 1 to iterate over.
data2¶
Dataset 2 to iterate over.
data3¶
Dataset 3 to iterate over.
data4¶
Dataset 4 to iterate over.
data5¶
Dataset 5 to iterate over.
dataN¶
Additional datasets.. .
initial¶
Optional initial carry state.
result¶
Processed data.
final¶
Final carry state after last operation.
traversal_axis¶
The axis over which to loop while carrying over the activations in the recurrent nodes. For packet data, this can be one of the named axes, time being the default (while instance is usually reserved as the batch axis). If there are multiple axes of the same type, this will resolve to the first such axis, but you can also write e.g., time.mylabel to specify an axis based on not just its type buy additional its custom label (if any). You may also specify a numeric axis index, although this is more commonly used if you only process numeric arrays. If you process numeric arrays and time is listed, the first axis will be used. In any case, the selected the axis will be omitted from the data that the loop body sees on any given call (i.e., the loop body only sees successive slices of the packet along that axis).
set_breakpoint¶
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.
metadata¶
User-definable meta-data associated with the node. Usually reserved for technical purposes.
reverse¶
If True, process the iterables in reverse order. The output will then be the state after having processed the leftmost element of the iterables.
log_errors¶
If True, log exceptions occurring in the loop body.
compile¶
Whether and how to compile the loop body for more efficient execution. Auto will compile the loop only if it occurs in a context where compilation is necessary, for example inside a model passed to nodes such as DeepLearning, ConvexModel, or one of the Inference nodes. The 'jax' option will generally attempt to compile the loop to run on the jax backend; this comes with a series of limitations -- all nodes in the body have to work with jax data types (numeric values only) and operations, and break/continue nodes cannot be used. 'Off' is the appropriate setting for anything except for the most performance-critical computations consisting of mainly math operations, possibly with some data reformatting. Off can also be useful in a situation where the loop occurs in a compiled context, but the iterable is a static constant (e.g., fixed list) and relatively short; in this case, the loop will be completely unrolled, which can be more efficient than actually looping.
unroll¶
Optionally the unrolling factor for this loop, if compiling. Typically this is a small power of two such as 4 or 8. This is mainly an efficiency improvement for extremely tight loops that perform very cheap math operations.