Module: control_flow

Control flow nodes.

These nodes can be used to implement non-linear control flow such as looping, conditional processing, and functions. These features are mostly for lightweight glue code, while complex logic is best written in Python. All control flow nodes receive a subgraph that they control (that is, evaluate zero or more times in case of loops or conditionally in case of If/Else) as an input. As such, the controlling node typically follows the nodes being controlled.

ArrayForEach

Apply the loop body to each slice of an array or packet along some axis and stack the results along some axis.

This is similar to a foreach node for a list, but acts along an axis of an array or packet and produces an array or packet as a result. Importantly, the node supports auto-vectorization, where the loop body is transformed into array operations that act on all elements of the looped axes simultaneously. The result can also be compiled (if a carefully chosen subset of supported nodes is used in the loop body) to be run very efficiently on the GPU or multicore CPU. Whether the loop body will see the looped-over axis (as a singleton, i.e., one-element, axis) or not can be controlled via the pass_singleton_axis property. The loop body can also loop over multiple input arrays simultaneously, but note that the loop only makes a single pass along the respective arrays in lock-step, which therefore need to have the same length along their respective looped-over axis. If a packet has multiple chunks each with different axes, one may also specify a comma-separated list of axes to loop over to indicate the axis to loop over for each such chunk. Otherwise, any chunks that do not have the sole specified axis for the respective packet are implicitly dropped. By default, the result is stacked along the same axis as the input; if a singleton axis is passed through to the loop body, the body can control the output axis values, and otherwise the input axis values are passed through unchanged. Alternatively one may also stack along a different axis of a given type; as a convenience, one may pass in a dictionary (structure) of axis fields (each typically an array) used to initialize or override the fields of the concatenated axis (this also works when stacking along the same axis and/or when passing through a singleton axis). Additionally one may also pass in input arguments without looping over an axis by setting the respective axis to "none". The node can also be used with plain arrays, blocks, chunks, or packets in any combination. The loop body (the stretch of nodes that are repeatedly run) begins with a Placeholder node that receives the current slice of the input argument, or multiple such placeholders if one wishes to loop over multiple arguments simultaneously, and the final node of the loop body is then wired into the "body" input of the Array For Each node. The node will invoke the loop body for each slice being iterated over and stack the resulting processed slices.

More Info...

Version 1.0.0

Ports/Properties

  • metadata
    User-definable meta-data associated with the node. Usually reserved for technical purposes.

    • verbose name: Metadata
    • default value: {}
    • port type: DictPort
    • value type: dict (can be None)
  • body
    Graph to apply (loop body).

    • verbose name: Body
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • body__signature
    Arguments passed to the loop body. The loop body will generally receive a slice of the array being iterated over (along the axis specified by the axis1..N parameters). The loop can iterate over multiple arrays (or packets) simultaneously, in which case the loop body will receive one slice from each wired-in array as separate arguments, which may be named slice1, slice2, and so forth (or any other names can be used here). Your loop body therefore needs to contain this many Placeholder nodes, with their respective slotnames matching the names listed here (e.g., slice1, slice2, etc.). Then, anything downstream of (i.e., depending on the value of) those Placeholders nodes will collectively constitute your loop body. The final output node of your loop body is then wired into the "body" input port of the loop node. In graphical UIs, the edge will be drawn in dotted style to indicate that the preceding graph itself is given to the loop node as the loop body (which will then execute it zero or more times), which is in contrast to a regular (solid) edge that indices normal data flow wherein first the preceding node runs, and then its output is passed to the subsequent node. The ArrayForEach node will then stack the results of the loop body into a single array or packet along the chosen stacking axis.

    • verbose name: Body [Data Slice]
    • default value: (slice1)
    • port type: Port
    • value type: object (can be None)
  • data1
    Array 1 to iterate over.

    • verbose name: Data1
    • default value: None
    • port type: DataPort
    • value type: AnyNumeric (can be None)
    • data direction: IN
  • data2
    Array 2 to iterate over.

    • verbose name: Data2
    • default value: None
    • port type: DataPort
    • value type: AnyNumeric (can be None)
    • data direction: IN
  • data3
    Array 3 to iterate over.

    • verbose name: Data3
    • default value: None
    • port type: DataPort
    • value type: AnyNumeric (can be None)
    • data direction: IN
  • data4
    Array 4 to iterate over.

    • verbose name: Data4
    • default value: None
    • port type: DataPort
    • value type: AnyNumeric (can be None)
    • data direction: IN
  • data5
    Array 5 to iterate over.

    • verbose name: Data5
    • default value: None
    • port type: DataPort
    • value type: AnyNumeric (can be None)
    • data direction: IN
  • result
    Result array.

    • verbose name: Result
    • default value: None
    • port type: DataPort
    • value type: AnyNumeric (can be None)
    • data direction: OUT
  • axis1
    Axis over which to iterate for data1. None means do not iterate, i.e., pass the data as-is. You can also enter the axis label here if you have multiple axes of same type but with different labels, as in feature.mylabel. If you intend to loop over one axis for the first chunk but another axis in the next chunk, you can also specify a comma-separated list of axes here.

    • verbose name: Loop Axis (Data1)
    • default value: axis
    • port type: ComboPort
    • value type: str (can be None)
  • axis2
    Axis over which to iterate for data2. See axis1 documentation for additional information.

    • verbose name: Loop Axis (Data2)
    • default value: none
    • port type: ComboPort
    • value type: str (can be None)
  • axis3
    Axis over which to iterate for data3. See axis1 documentation for additional information.

    • verbose name: Loop Axis (Data3)
    • default value: none
    • port type: ComboPort
    • value type: str (can be None)
  • axis4
    Axis over which to iterate for data4. See axis1 documentation for additional information.

    • verbose name: Loop Axis (Data4)
    • default value: none
    • port type: ComboPort
    • value type: str (can be None)
  • axis5
    Axis over which to iterate for data5. See axis1 documentation for additional information.

    • verbose name: Loop Axis (Data5)
    • default value: none
    • port type: ComboPort
    • value type: str (can be None)
  • stack_axis
    Axis along which the outputs will be stacked. For non-packet outputs, this can also be the numeric position where the axis shall appear in the output array. If this is set to (same), and only one kind of typed axis was used in axis1..N, then this will stack the outputs data along the same kind of axis, which will be prepended unless pass_singleton_axis was True, in which case the position of that axis will be preserved. This can also be a comma-separated list of axes, in which case the first chunk is stacked along the first axis, the second chunk along the second axis, and so forth.

    • verbose name: Stack Along Axis
    • default value: (same)
    • port type: ComboPort
    • value type: str (can be None)
  • pass_singleton_axis
    Whether the loop body shall receive the looped-over axis as a one-element axis. Note that this is only supported for axes with all-numeric fields (e.g., time, lag, or frequency). The alternative is to drop the axis from the data seen by the loop body.

    • verbose name: Pass Singleton Axis To Body
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • field_overrides
    Optionally a dictionary of axis field overrides to use in the concatenated data. This is mainly useful when not passing a singleton axis, and stacking along an axis other than '(same)', in which case the resulting axis would be initialized to default/dummy values unless those fields were overridden via this argument.

    • verbose name: Override Stack Axis Fields
    • default value: {}
    • port type: DictPort
    • value type: dict (can be None)
  • split_random_seed
    Whether the DrawRandomSeed node should yield different seeds in each loop iteration. WARNING: even if this is set to True, it is currently only respected when using the from_haiku_context option in DrawRandomSeed, and not otherwise. To still use split seeds in conjunction with an enclosing WithRandomSeed, you can use DrawRandomSeed before the loop and draw a full array of seeds, which you can then pass to the loop as an additional data argument.

    • verbose name: Split Random Seed
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • backend
    Optional compute backend to use. Keep means to retain the current backend of the given input data. Numpy is the basic CPU backend, and forcing it will cause the loop body to be applied to each slice of the given array(s) in sequence along a desired axis, and the results will be stacked along the chosen output axis. Jax instead will invoke the loop body only once on the 0th array slice, and work out the result of applying the body across all slices in a batched fashion using vectorized math operations (the principle is to send a special sentinel array called a Tracer through the loop body, which behaves like a single array slice as far as the loop body is concerned, but which actually applies all operations that it "experiences" to the original full array in a batched or "broadcasted" fashion). This is much faster than the numpy backend since the overhead associated with the body is only incurred once. Other backends currently are analogous to the numpy strategy, but this may change in future NeuroPype versions.

    • verbose name: Backend / Compilation
    • default value: jax
    • port type: EnumPort
    • value type: str (can be None)
  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

Break

Break out of the current loop.

This node will cause an ongoing loop to terminate and return the most recent result before the break was encountered. This is analogous to the "break" statement in conventional programming languages. A typical usage pattern is to use this node on one of the branches of an IfElse node to skip the rest of the enclosing loop if some condition is met. An important limitation is that this node may only be used inside loop constructs and cannot be used to exit out of function calls (i.e. out of a function invoked using the Call node).

More Info...

Version 1.0.0

Ports/Properties

  • metadata
    User-definable meta-data associated with the node. Usually reserved for technical purposes.

    • verbose name: Metadata
    • default value: {}
    • port type: DictPort
    • value type: dict (can be None)
  • result
    Result of the graph.

    • verbose name: Result
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: INOUT
  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

Continue

Skip the rest of the current loop cycle and continue with the next loop iteration, if any.

This is analogous to the "continue" statement in conventional programming languages. A typical usage pattern is to use this node on one of the branches of an IfElse node to skip the rest of the enclosing loop cycle if some condition is met. The node can also be used to skip (filter out) elements in a list/dictionary/etc comprehension. An important limitation is that this node may only be used inside loop constructs and cannot be used to exit out of function calls (i.e. out of a function invoked using the Call node).

More Info...

Version 1.0.0

Ports/Properties

  • metadata
    User-definable meta-data associated with the node. Usually reserved for technical purposes.

    • verbose name: Metadata
    • default value: {}
    • port type: DictPort
    • value type: dict (can be None)
  • result
    Result of the graph.

    • verbose name: Result
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: INOUT
  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

Except

Evaluate the try body while guarding for exceptions.

If an exception occurs, the provided lists of exception types are checked in order to see if the exception is of one of the types listed. If so, the corresponding except body is evaluated, and otherwise the next list of exception types (if any) is checked. The result of the Except node is then either the result of the try body (if it succeeded), or the result of the triggered except body if an exception occurred. If a finally body is provided, it is always evaluated after the try body and the except body (if any). If the finally body is configured to have a placeholder, then the finally body may transform what would otherwise be the result of the Except node; otherwise, the finally body is only invoked for side effects (e.g., deleting a file created by the try body) and the finally body's return value is ignored.

More Info...

Version 1.0.0

Ports/Properties

  • metadata
    User-definable meta-data associated with the node. Usually reserved for technical purposes.

    • verbose name: Metadata
    • default value: {}
    • port type: DictPort
    • value type: dict (can be None)
  • try_body
    Body to execute with exception guarding.

    • verbose name: Try Body
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • try_body__signature
    Optional slotname of a placeholder that marks the beginning of the try body. The try body is a subgraph which is executed normally, except that any exception that occurs is caught and handled by the exception handlers (if any). The last node of that subgraph is wired into the Except node's "try_body" port, which makes it run under the control of the Except node. In graphical UIs this edge will show as dotted to indicate that this is not normal forward data flow but a subgraph (i.e., the try body) is being passed to the Except node. If no exception occurs, the try body's return value is returned as the result of the Except node. To limit the scope of the try body, you can specify the name of a placeholder node here on which the try body depends (for example, (try) instead of (~)). Then, only nodes downstream of this placeholder node will be part of the try body. Otherwiese, the try body will be the upstream graph and any of its downstream "loose ends".

    • verbose name: Try Body [Signature]
    • default value: (~)
    • port type: Port
    • value type: object (can be None)
  • handler1
    Exception handler 1.

    • verbose name: Handler1
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • handler2
    Exception handler 2.

    • verbose name: Handler2
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • handler3
    Exception handler 3.

    • verbose name: Handler3
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • handler4
    Exception handler 4.

    • verbose name: Handler4
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • handler5
    Exception handler 5.

    • verbose name: Handler5
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • finally_body
    Finally handler.

    • verbose name: Finally Body
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • finally_body__signature
    Optional input argument name of the finally body. By default, the finally body (if one is provided) accepts no arguments and is therefore only evaluated for its side effects (e.g., printing a message or closing a file), which generally happens after the try body and any triggered exception handler nodes have run, whether an exception occurred or notm and whether it was caught or not. The finally body's return value is ignored. However, if you specify a name here (e.g., (finally) instead of (~)), then the finally body must also contain a Placeholder node whose slotname matches this name, and the placeholder will receive what would normally be the return value of the Except node, and the body is allowed to transform that value (e.g., by returning a different value). Note that the normal return value of the Except node is either the return value of the try body (if it succeeded) or that of the exception handler that was triggered. The last node of the finally body is then wired into the Except node's "finally_body" port. In graphical UIs this edge will show as dotted to indicate that this is not normal forward data flow but a subgraph (i.e., the finally body) is being passed to the Except node. The value returned by that final node is then also returned as the result of the Except node. If no exception handler triggered, then the Except node has no return value since it will throw an exception; in this case, the finally body receives a None value as its input argument.

    • verbose name: Finally Body [Signature]
    • default value: (~)
    • port type: Port
    • value type: object (can be None)
  • result
    Result of the operation.

    • verbose name: Result
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: OUT
  • exceptions1
    List of exceptions to catch in handler 1. To catch no exceptions, you can clear this value to an empty list, i.e., []. This can list any exception type, including custom ones. You can find a listing in the Throw node's list of predefined throwable exceptions. If none of the listed types match the exception thrown by the try body, the next handler is tried. However, keep in mind that this means that, if your first handler is a broad exception handler that catches all exceptions (such as the default type Exception), then the other handlers will never be tried.

    • verbose name: Exceptions1
    • default value: ['Exception']
    • port type: ListPort
    • value type: list (can be None)
  • exceptions2
    List of exceptions to catch in handler 2.

    • verbose name: Exceptions2
    • default value: []
    • port type: ListPort
    • value type: list (can be None)
  • exceptions3
    List of exceptions to catch in handler 3.

    • verbose name: Exceptions3
    • default value: []
    • port type: ListPort
    • value type: list (can be None)
  • exceptions4
    List of exceptions to catch in handler 4.

    • verbose name: Exceptions4
    • default value: []
    • port type: ListPort
    • value type: list (can be None)
  • exceptions5
    List of exceptions to catch in handler 5.

    • verbose name: Exceptions5
    • default value: []
    • port type: ListPort
    • value type: list (can be None)
  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

Fold

Fold elements of a collection or iterable using a function that successively combines (reduces) the elements with an initial value.

The basic formula implemented by this node is, for a 3-item list: result = op(op(op(initial, item1), item2), item3) where op is the function given as the loop body, which takes as its first argument the current state and as its second argument the next item in the iterable, and computes the new state. A simple use case of this node is taking the sum of a list of numbers. The initial value would then be set to zero, and the body adds its first and second arguments together. However, this node is really a general purpose loop that can be used to implement imperative-style loops in a functional style. To do this, simply make the initial state a dictionary with your variables before the loop is entered, and the loop body returns a new dictionary that has the new values assigned to the variables. An infinite or counting-based for-style loop can be implemented by using a RangeIterator as the iterable. You can also loop over multiple iterables simultaneously, in which case your loop body and signature must have correspondingly more placeholders for the additional iterables (e.g., state, item1, item2, ..). While packets are not generally iterables, when a packet is passed into this node as the iterable, the node will iterate over the first axis of any chunk in the packet, which all must be of the same length. You can control what axes to iterate over using the MoveAxisFirst node. Beware that the node will drop any chunks that are either empty or are marker chunks. As with arrays, the loop body will receive the k'th slice of the packet with the leading axis dropped from each chunk. If the body is not given, or if the iterable is empty or not given, then the initial value is returned as the result. The body may use the Continue node to skip (drop) the current item and proceed to the next item in the iterable, or the Break node to terminate the loop early, causing the result to be the current state. The body may also have only one argument, in which case only the current state is passed in, and the iterable is solely used to determine the termination condition.

More Info...

Version 1.0.0

Ports/Properties

  • metadata
    User-definable meta-data associated with the node. Usually reserved for technical purposes.

    • verbose name: Metadata
    • default value: {}
    • port type: DictPort
    • value type: dict (can be None)
  • body
    Reducer function (loop body).

    • verbose name: Body
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • body__signature
    List of arguments names accepted by the loop body. Since a fold loop, in contrast to a for each loop, carries over state from one iteration to the next, your loop body must accept at least two arguments: the previous iteration's state (or initial value), which must always be the first argument listed here, and the current item from the iterable being looped over. If you wish to loop over multiple iterables (e.g., lists/arrays/iterators) simultaneously (which all must have the same length), you can list additional items here, using any name of your choosing. Your loop body must then have one Placeholder node for each of the loop variables listed here, with its slotname matching the respective name (i.e., state, item1, item2, ...), although you can also choose any other names here. Then, anything downstream of (i.e., depending on the value of) those Placeholders will collectively constitute your loop body. The final node of your loop body is then expected to return the state that will be passed into the next iteration (if you wish to carry over multiple values, you may use e.g., a CreateList node or a CreateDict node for this). The final output node of your loop body is then wired into the "body" input port of the loop node. In graphical UIs, the edge will be drawn in dotted style to indicate that the preceding graph itself is given to the loop node as the loop body (which will then execute it zero or more times), which is in contrast to a regular (solid) edge that indices normal data flow wherein first the preceding node runs, and then its output is passed to the subsequent node. The output of the Fold Loop node itself is then the final state value that it received from your loop body on the last iteration.

    • verbose name: Body [State, Item]
    • default value: (state,item)
    • port type: Port
    • value type: object (can be None)
  • iterable1
    Iterable 1 to iterate over.

    • verbose name: Iterable1
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • iterable2
    Iterable 2 to iterate over.

    • verbose name: Iterable2
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • iterable3
    Iterable 3 to iterate over.

    • verbose name: Iterable3
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • iterable4
    Iterable 4 to iterate over.

    • verbose name: Iterable4
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • iterable5
    Iterable 5 to iterate over.

    • verbose name: Iterable5
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • iterableN
    Additional iterables.. .

    • verbose name: Iterablen
    • default value: None
    • port type: DataPort
    • value type: list (can be None)
    • data direction: IN
  • iterable
    Alias for the iterable1 port.

    • verbose name: Iterable
    • default value: None
    • port type: AliasPort
    • value type: object (can be None)
  • result
    Result of last operation.

    • verbose name: Result
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: OUT
  • initial
    Initial value for the state. It is recommended that this has the same structure (e.g., dictionary fields) as what is returned by the loop body. For compiled loops, this is required.

    • verbose name: Initial
    • default value: None
    • port type: Port
    • value type: object (can be None)
  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • reverse
    If True, process the iterables in reverse order. The output will then be the state after having processed the leftmost element of the iterables.

    • verbose name: Process In Reverse Order
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • log_errors
    If True, log exceptions occurring in the loop body.

    • verbose name: Log Errors
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • compile
    Whether and how to compile the loop body for more efficient execution. Auto will compile the loop only if it occurs in a context where compilation is necessary, for example inside a model passed to nodes such as DeepLearning, ConvexModel, or one of the Inference nodes. The 'jax' option will generally attempt to compile the loop to run on the jax backend; this comes with a series of limitations -- all nodes in the body have to work with jax data types (numeric values only) and operations, and break/continue nodes cannot be used. 'Off' is the appropriate setting for anything except for the most performance-critical computations consisting of mainly math operations, possibly with some data reformatting. Off can also be useful in a situation where the loop occurs in a compiled context, but the iterable is a static constant (e.g., fixed list) and relatively short; in this case, the loop will be completely unrolled, which can be more efficient than actually looping.

    • verbose name: Compile
    • default value: auto
    • port type: EnumPort
    • value type: str (can be None)
  • unroll
    Optionally the unrolling factor for this loop, if compiling. Typically this is a small power of two such as 4 or 8. This is mainly an efficiency improvement for extremely tight loops that perform very cheap math operations.

    • verbose name: Unroll
    • default value: None
    • port type: IntPort
    • value type: int (can be None)

FoldCollect

A variant of the Fold loop node that additionally collects a secondary output of the loop body in each iteration and stacks them into an array or Packet.

See the documentation for the Fold loop for a basic introduction. Unlike Fold, the loop body must return a two-element list: the first is the updated state (which is the only output in case of the plain Fold loop), and the second is the value to collect. The result of the Fold Collect loop is then all the values that were emitted in this way stacked along a new leading axis. The value can be a number, array or data of type block, chunk, or packet, where again the stacking is always along a new leading axis. In the latter cases, if there was a unique leading axis in the loop input, that axis is preserved and will become (again) the leading axis in the output, otherwise the leading axis will be a plain untyped axis (type Axis). The loop body is also allowed to emit a nesting of lists or dictionaries of stackable data as the emitted value, and the loop output will respect the same data structure but contain stacked data in the respective places.

More Info...

Version 1.0.0

Ports/Properties

  • metadata
    User-definable meta-data associated with the node. Usually reserved for technical purposes.

    • verbose name: Metadata
    • default value: {}
    • port type: DictPort
    • value type: dict (can be None)
  • body
    Reducer function (loop body).

    • verbose name: Body
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • body__signature
    Names of arguments accepted by the loop body. Since a fold loop, in contrast to a for each loop, carries over state from one iteration to the next, your loop body must accept at least two arguments: the previous iteration's state (or initial value), which must always be the first argument listed here, and the current item from the iterable being looped over. If you wish to loop over multiple iterables (e.g., lists/arrays/iterators) simultaneously (which all must have the same length), you can list additional items here, using any name of your choosing. Your loop body must then have one Placeholder node for each of the loop variables listed here, with its slotname set to the respective name (i.e., state, item1, item2, ...), although you can also choose any other names here. Then, anything downstream of (i.e., depending on the value of) those Placeholders will collectively constitute your loop body. In contrast to the simple Fold Loop, which only returns the final state, the Fold Collect loop is expected to return a two-element list: the first is the updated state (which can be any data type or structure of your choosing), and the second is the value to collect. Therefore, the final node of the loop body is usually a CreateList node with two inputs wired into it. The final output node of your loop body is then wired into the "body" input port of the loop node. In graphical UIs, the edge will be drawn in dotted style to indicate that the preceding graph itself is given to the loop node as the loop body (which will then execute it zero or more times), which is in contrast to a regular (solid) edge that indices normal data flow wherein first the preceding node runs, and then its output is passed to the subsequent node. The loop node itself has two main outputs: one is the final state received from the loop body on the last iteration (in the final output port), and the other is the collected values, which is either an array or packet stacked along a new leading axis, or a list (depending on the "collect" setting).

    • verbose name: Body [State, Item]
    • default value: (state,item)
    • port type: Port
    • value type: object (can be None)
  • iterable1
    Iterable 1 to iterate over.

    • verbose name: Iterable1
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • iterable2
    Iterable 2 to iterate over.

    • verbose name: Iterable2
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • iterable3
    Iterable 3 to iterate over.

    • verbose name: Iterable3
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • iterable4
    Iterable 4 to iterate over.

    • verbose name: Iterable4
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • iterable5
    Iterable 5 to iterate over.

    • verbose name: Iterable5
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • iterableN
    Additional iterables.. .

    • verbose name: Iterablen
    • default value: None
    • port type: DataPort
    • value type: list (can be None)
    • data direction: IN
  • iterable
    Alias for the iterable1 port.

    • verbose name: Iterable
    • default value: None
    • port type: AliasPort
    • value type: object (can be None)
  • collected
    Collected results in stacked form.

    • verbose name: Collected
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: OUT
  • final
    Final state after last operation.

    • verbose name: Final
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: OUT
  • initial
    Initial value for the state. It is recommended that this has the same structure (e.g., dictionary fields) as what is returned by the loop body. For compiled loops, this is required.

    • verbose name: Initial
    • default value: None
    • port type: Port
    • value type: object (can be None)
  • collect
    Form into which results are collected. If set to list, the outputs of the loop body are returned as a list. If set to stacked, the outputs are stacked along a new first axis. In this case, if the input data was packets, and the first axes being iterated over were the same in all chunks, then the first axis of the output will be the same as in the input (otherwise a generic axis is used).

    • verbose name: Collect
    • default value: stacked
    • port type: EnumPort
    • value type: str (can be None)
  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • reverse
    If True, process the iterables in reverse order. The output will then be the state after having processed the leftmost element of the iterables.

    • verbose name: Process In Reverse Order
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • log_errors
    If True, log exceptions occurring in the loop body.

    • verbose name: Log Errors
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • compile
    Whether and how to compile the loop body for more efficient execution. Auto will compile the loop only if it occurs in a context where compilation is necessary, for example inside a model passed to nodes such as DeepLearning, ConvexModel, or one of the Inference nodes. The 'jax' option will generally attempt to compile the loop to run on the jax backend; this comes with a series of limitations -- all nodes in the body have to work with jax data types (numeric values only) and operations, and break/continue nodes cannot be used. 'Off' is the appropriate setting for anything except for the most performance-critical computations consisting of mainly math operations, possibly with some data reformatting. Off can also be useful in a situation where the loop occurs in a compiled context, but the iterable is a static constant (e.g., fixed list) and relatively short; in this case, the loop will be completely unrolled, which can be more efficient than actually looping.

    • verbose name: Compile
    • default value: auto
    • port type: EnumPort
    • value type: str (can be None)
  • unroll
    Optionally the unrolling factor for this loop, if compiling. Typically this is a small power of two such as 4 or 8. This is mainly an efficiency improvement for extremely tight loops that perform very cheap math operations.

    • verbose name: Unroll
    • default value: None
    • port type: IntPort
    • value type: int (can be None)

ForEach

Apply a function or loop body to each item in a collection or iterator.

The loop body (the collection of nodes you wish to run repeatedly) must begin with a Placeholder node that receives the item, and end with the ForEach node. The ForEach node will invoke the loop body for each item in the iterable input port, and collect the results into a list, dictionary, or flatten them into a single container of the same type as the type of the item the ForEach node receives from the end of the loop body. (Use the collect property to change this behavior.) By default the ForEach node returns only the last result from the loop body. By default, nodes in the loop body do not carry over their internal across iterations of the ForEach node; this behavior can be changed by setting the stateless property to false. (For example, if you want to use PathIterator as an iterator inside a ForEach loop, you would want to configure ForEach to carry over the PathIterator's state by setting stateless to false.) You can use a Continue node in the loop body to skip the current item and proceed to the next item in the iterable, and the Break node to terminate the loop early. If items are being collected (based on the setting of the collect` property), this will truncate the collection at the point where the loop was terminated. If the iterable is an iterator node, you can wire the iterator node'sthisport into theiterableport of the ForEach node. The iterator node will be invoked for each repetition of the loop, pass the next item to the ForEach loop. (See the PathIterator and EnumerateIterator node documentation.) The variable passed by ForEach to Placeholder is by default nameditem, but can be renamed by changing thebody(item parameter) property. (If using a ForEach node in a python script, change the node'sbody__signature=constructor argument.) Note that if you rename item to some other name, you also need to change theslotname(name) in the Placeholder node to match. If the loop body does not use the iterable item that is passed to the Placeholder node, you can instead wire the Placeholder node'supdateport into theupdateport of the first node in the loop body in order to process the loop body without passing any value to it. The ForEach node can be used as a list or dictionary comprehension (and other types of comprehensions) by using thelist,dict, etc options in thecollectproperty. In this case the choice ofcollectdetermines the output data type, which need not be the same as the iterable that is being iterated over. It is also possible to use the ForEach node to flatten a nested collection, by using theflattenoption incollect``. This is is most useful when you want to implement a nested for loop (with multiple ForEach nodes) but want to collect the final results into a single flat container. In this case you choose flatten in the outer loop(s), and select the desired container type in the inner(most) loop. Note that flatten, while it can handle packets and arrays implements a simple default rule; for more complex cases you are advised to instead collect results into a list, and then concatenate the items yourself using whatever logic you need.

More Info...

Version 1.0.1

Ports/Properties

  • metadata
    User-definable meta-data associated with the node. Usually reserved for technical purposes.

    • verbose name: Metadata
    • default value: {}
    • port type: DictPort
    • value type: dict (can be None)
  • body
    Graph to apply (loop body).

    • verbose name: Body
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • body__signature
    The list of loop variables received by the loop body. In a "For Each" loop, the loop body depends on only a single loop variable, namely the current item from the wired-in iterable. Consequently, your loop body should contain a single Placeholder node whose slotname must match the name listed here (e.g., item, but you can also choose a different name). Then, anything downstream of (i.e., depending on the value of) that Placeholder will constitute your loop body. The final output node of your loop body is then wired into the "body" input port of the loop node. In graphical UIs, the edge will be drawn in dotted style to indicate that the preceding graph itself is given to the loop node as the loop body (which will then execute it zero or more times), which is in contrast to a regular (solid) edge that indices normal data flow wherein first the preceding node runs, and then its output is passed to the subsequent node. The For Each node will then return a collection (see collect setting) of the values received from the final node on each iteration.

    • verbose name: Body [Item Parameter]
    • default value: (item)
    • port type: Port
    • value type: object (can be None)
  • iterable
    Data to iterate over.

    • verbose name: Iterable
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • result
    ForEach result.

    • verbose name: Result
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: OUT
  • collect
    Collect results into a list, dictionary, graph, packet, array or flatten the results into a single container of the same type emitted by the function. For graphs, a mix of nodes and edges may be emitted, and the resulting graph will have only those edges for which both endpoint nodes were also emitted separately (i.e., emitting an edge will not implicitly add its endpoint nodes). If 'last', only the last result is returned (as in Repeat). When applied to numeric arrays, the 'flatten' option concatenates the arrays along their existing first axis while the 'array' option stacks them along a new first axis. Packet and Graph are analogous to dict and list options, in that they are comprehensions that allow the user to itemize the constituents of these data structures (for packets, each is a 2-element list of [chunkname, Chunk] and for Graph each is a nodes and/or an edges). The dict-of-lists option is convenient when the loop body returns a list of dictionaries, and you wish to collect the results as a dictionary of lists; this can be a useful pattern for loop bodies with multiple return values, where the return values are wired into a Create Dictionary node and that dictionary is returned as the loop body result. The loop itself will then return a dictionary that has a list of items for each key.

    • verbose name: Collect
    • default value: list
    • port type: EnumPort
    • value type: str (can be None)
  • stateless
    If True, then the loop body will not carry over its previous state across iterations. This can be used to ensure perfect isolation between iterations. In contrast, the option should be disabled if you wish your loop body nodes to retain prior state across loop iterations; an example is if you use an iterator node (separate from the loop's wired-in iterable) inside the loop body.

    • verbose name: Stateless
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • compile
    Whether and how to compile the loop body for more efficient execution. Off is the recommended default for anything except for the most performance-critical computations consisting of mainly math operations, possibly with some data reformatting. The 'jax' option will compile the loop to run on the jax backend; this comes with a series of limitations -- all nodes in the body have to work with jax data types (numeric values only) and operations, and break/continue nodes cannot be used. Auto will only compile the loop if it occurs in a context where compilation is necessary, for example inside a model passed to nodes such as DeepLearning, ConvexModel, or one of the Inference nodes.

    • verbose name: Compile
    • default value: off
    • port type: EnumPort
    • value type: str (can be None)
  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

FunctionCall

Call/apply a function with some positional arguments and/or named ("keyword") arguments.

This will invoke a function that is declared elsewhere using the Function node and return the result. The arguments apply in the order in which they are listed in the function's graph [signature] parameter. Besides positional arguments, Call can also bind named placeholders before evaluating the graph. Note though that placeholders that are bound by any lambda in the contained graph cannot be assigned by name, i.e., only free placeholders can be assigned this way.

More Info...

Version 1.0.0

Ports/Properties

  • metadata
    User-definable meta-data associated with the node. Usually reserved for technical purposes.

    • verbose name: Metadata
    • default value: {}
    • port type: DictPort
    • value type: dict (can be None)
  • function
    Function to call. This can be the name of a function that is declared elsewhere, or it can also be a callable object, for example as returned by the Lambda node. When used in scripts, this can also be any other callable Python object. Lastly, if the as_python option is set, this may also be the name of a Python function or an in-place defined lambda function (see also as_python option).

    • verbose name: Function
    • default value: None
    • port type: Port
    • value type: object (can be None)
  • stateful
    Persist graph's state across calls. If this is unset, the call behaves like a regular function call such that the graph's state is reset on every call. If set, the call behaves statefully, that is, any stateful nodes in the called graph (e.g., filters) will retain their state across calls. Thus, the call behaves as if the graph was materialized at the call size. Different call sites to the same graph still have independent state (i.e., different calls are different materializations).

    • verbose name: Stateful
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • as_python
    If set to unsafe, the funcname will be interpreted as a Python function name or in-place defined lambda expression. In this case, BE SURE to never pass untrusted inputs into the funcname argument.

    • verbose name: Python Code
    • default value: no
    • port type: EnumPort
    • value type: str (can be None)
  • pos1
    Positional Argument 1.

    • verbose name: Pos1
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • pos2
    Positional Argument 2.

    • verbose name: Pos2
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • pos3
    Positional Argument 3.

    • verbose name: Pos3
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • pos4
    Positional Argument 4.

    • verbose name: Pos4
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • pos5
    Positional Argument 5.

    • verbose name: Pos5
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • pos6
    Positional Argument 6.

    • verbose name: Pos6
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • pos7
    Positional Argument 7.

    • verbose name: Pos7
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • pos8
    Positional Argument 8.

    • verbose name: Pos8
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • pos9
    Positional Argument 9.

    • verbose name: Pos9
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • posN
    Additional positional arguments.. .

    • verbose name: Posn
    • default value: None
    • port type: DataPort
    • value type: list (can be None)
    • data direction: IN
  • name1
    Keyword argument name 1.

    • verbose name: Name1
    • default value:
    • port type: StringPort
    • value type: str (can be None)
  • val1
    Keyword argument value 1.

    • verbose name: Val1
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • name2
    Keyword argument name 2.

    • verbose name: Name2
    • default value:
    • port type: StringPort
    • value type: str (can be None)
  • val2
    Keyword argument value 2.

    • verbose name: Val2
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • name3
    Keyword argument name 3.

    • verbose name: Name3
    • default value:
    • port type: StringPort
    • value type: str (can be None)
  • val3
    Keyword argument value 3.

    • verbose name: Val3
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • name4
    Keyword argument name 4.

    • verbose name: Name4
    • default value:
    • port type: StringPort
    • value type: str (can be None)
  • val4
    Keyword argument value 4.

    • verbose name: Val4
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • name5
    Keyword argument name 5.

    • verbose name: Name5
    • default value:
    • port type: StringPort
    • value type: str (can be None)
  • val5
    Keyword argument value 5.

    • verbose name: Val5
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • name6
    Keyword argument name 6.

    • verbose name: Name6
    • default value:
    • port type: StringPort
    • value type: str (can be None)
  • val6
    Keyword argument value 6.

    • verbose name: Val6
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • name7
    Keyword argument name 7.

    • verbose name: Name7
    • default value:
    • port type: StringPort
    • value type: str (can be None)
  • val7
    Keyword argument value 7.

    • verbose name: Val7
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • name8
    Keyword argument name 8.

    • verbose name: Name8
    • default value:
    • port type: StringPort
    • value type: str (can be None)
  • val8
    Keyword argument value 8.

    • verbose name: Val8
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • name9
    Keyword argument name 9.

    • verbose name: Name9
    • default value:
    • port type: StringPort
    • value type: str (can be None)
  • val9
    Keyword argument value 9.

    • verbose name: Val9
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • nameN
    Additional keyword argument names.. .

    • verbose name: Additional Names
    • default value: None
    • port type: ListPort
    • value type: list (can be None)
  • valN
    Additional keyword argument values.. .

    • verbose name: Valn
    • default value: None
    • port type: DataPort
    • value type: list (can be None)
    • data direction: IN
  • argdict
    Additional keyword arguments.. .

    • verbose name: Additional Keyword Arguments
    • default value: None
    • port type: DictPort
    • value type: dict (can be None)
  • result
    Result of function call.

    • verbose name: Result
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: OUT
  • snapshot
    Snapshot of function graph after call.

    • verbose name: Snapshot
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • pos0
    Positional Argument 0 (deprecated).

    • verbose name: Pos0
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • name0
    Keyword argument name 0 (deprecated).

    • verbose name: Name0
    • default value:
    • port type: StringPort
    • value type: str (can be None)
  • val0
    Keyword argument value 0 (deprecated).

    • verbose name: Val0
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • funcname
    Function to call.

    • verbose name: Funcname
    • default value: None
    • port type: AliasPort
    • value type: object (can be None)
  • funchandle
    Function to call.

    • verbose name: Funchandle
    • default value: None
    • port type: AliasPort
    • value type: object (can be None)
  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

FunctionDef

Define a function.

A function is a graph that contains placeholders which represent named function arguments. The function also has a signature, which is a list of of argument names in some order, which enables invoking the function without having to name the arguments. This node is used by wiring a subgraph into its graph port. The subgraph must contain one or more Placeholder nodes for any function arguments, and the function signature must name all the placeholders in some order, which is in the "Graph [signature]" (or graph__signature in code) port.

Version 1.0.0

Ports/Properties

  • metadata
    User-definable meta-data associated with the node. Usually reserved for technical purposes.

    • verbose name: Metadata
    • default value: {}
    • port type: DictPort
    • value type: dict (can be None)
  • graph
    Graph that defines the function.

    • verbose name: Graph
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • graph__signature
    Arguments accepted by your function. This is the list of argument names that your function receives, in the order in which they are expected if the function is called with positional arguments. You may also list keyword-only arguments after a + symbol, i.e., as in (arg1,arg2,+,kw1,kw2). Your function graph must contain one or more Placeholder nodes for each of the arguments listed here, with their respective slotnames set to the argument names listed here. Then, anything downstream of these placeholders collectively constitutes your function body, and the final node of your function body is expected to return the function result. This final node is then connected to the function node itself by wiring it to the "graph" input port of the function node. In graphical UIs this edge will be drawn in dotted style to indicate not a normal forward data flow, but the fact that the function body (graph) itself is passed to the Function node. The function declaration node follows the function body in the graph, rather than preceding it.

    • verbose name: Graph [Signatu Re]
    • default value: (arg1)
    • port type: Port
    • value type: object (can be None)
  • function
    Resulting function object.

    • verbose name: Function
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: OUT
  • funcname
    Name of the function. The function can then be invoked under its name elsewhere in the graph via the Invoke node.

    • verbose name: Function Name
    • default value: myfunction
    • port type: StringPort
    • value type: str (can be None)
  • friendly_name
    Friendly name of the node. This is the name that would show up in visual editors when the function is invoked as a node. This is capitalized and with spaces, as in "My Function".

    • verbose name: Friendly Name
    • default value: None
    • port type: StringPort
    • value type: str (can be None)
  • desc
    Description of the function. The first sentence is taken as the executive summary and should not exceed 60 characters). The next paragraph is the essential description, and any following paragraphs are considered additional description text. This should not list the arguments, but can give a high-level overview of what the function can accept and what it does. It is possible to use limited amounts of HTML formatting, for instance for emphasis.

    • verbose name: Description
    • default value: None
    • port type: StringPort
    • value type: str (can be None)
  • version
    Version of the function.

    • verbose name: Version
    • default value: 0.1.0
    • port type: StringPort
    • value type: str (can be None)
  • status
    Development status of the function.

    • verbose name: Status
    • default value: alpha
    • port type: EnumPort
    • value type: str (can be None)
  • export
    Export the function for use by other pipelines.

    • verbose name: Export
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

GraphPlaceholder

A placeholder that can accept a graph or callable function.

This can be used when declaring functions that accept other functions as arguments. This is used, for example, for call-back functions or small ad-hoc functions (like a+b) that are passed into another function. Note that if the graph that you receive itself contains free placeholders which need to be bound downstream in your function in order for the received graph to be runnable, you will have to declare those free placeholders in the "graph [contained free names]" field (or in code, in the graph__signature field). However, this is not always the case, in fact most ad hoc functions do not have any free placeholders, and can instead be called with some number of positional arguments (depending on the intended use case).

More Info...

Version 1.0.0

Ports/Properties

  • metadata
    User-definable meta-data associated with the node. Usually reserved for technical purposes.

    • verbose name: Metadata
    • default value: {}
    • port type: DictPort
    • value type: dict (can be None)
  • graph
    Received graph.

    • verbose name: Graph
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • graph__signature
    Signature for the "graph" input. This represents the contained free names for the subgraph that is wired into the "graph" port. This is formatted as in (a,b,c) where a,b,c are names of placeholders that are expected in the subgraph that goes out of the "graph" port. Alternatively, it can also be provided in data structure form as a list of lists, as in: [['a','b','c']].

    • verbose name: Graph [Contained Free Names]
    • default value: (~)
    • port type: Port
    • value type: object (can be None)
  • slotname
    Name of the placeholder.

    • verbose name: Name
    • default value: graph
    • port type: StringPort
    • value type: str (can be None)
  • verbose_name
    Optional verbose name for the placeholder. Can be used in function declarations for a more descriptive name.

    • verbose name: Verbose Name
    • default value: None
    • port type: StringPort
    • value type: str (can be None)
  • desc
    Help text for the placeholder. Only needs to be specified in function declarations. This is typically a single sentence.

    • verbose name: Desc
    • default value:
    • port type: StringPort
    • value type: str (can be None)
  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

IfElse

If-else conditional control flow aka conditional computation.

This node evaluates what is wired into its "then" branch if the condition was true, and what is wired into the "else" branch if the condition was false. This is called the "taken" branch. Crucially, the non-taken branch is not evaluated (i.e., the nodes do not run). If the taken branch has nothing wired into it, then the output defaults to None. If nothing is wired into the condition port or if it receives a None, then the else branch is evaluated and its value returned.

Version 1.0.0

Ports/Properties

  • metadata
    User-definable meta-data associated with the node. Usually reserved for technical purposes.

    • verbose name: Metadata
    • default value: {}
    • port type: DictPort
    • value type: dict (can be None)
  • condition
    Input condition.

    • verbose name: Condition
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • then_branch
    Then branch. Executed if condition is true.

    • verbose name: Then Branch
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • else_branch
    Else branch. Executed if condition is false.

    • verbose name: Else Branch
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • result
    Result of the branch that was evaluated.

    • verbose name: Result
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: OUT
  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

Lambda

Create an anonymous (lambda) function.

This node expresses that any named placeholders in the graph that it receives (i.e., the function body of the node) have a specific order. As a result, a graph that ends in a Lambda node can be passed into a Function Call node as the "function" argument, alongside any positional arguments. As such, a graph ending in a Lambda function represents an anonyomous (nameless) function that can be invoked with positional arguments just the same way as a named function (which is declared using the Function node).

More Info...

Version 0.1.0

Ports/Properties

  • metadata
    User-definable meta-data associated with the node. Usually reserved for technical purposes.

    • verbose name: Metadata
    • default value: {}
    • port type: DictPort
    • value type: dict (can be None)
  • graph
    Graph that defines the function.

    • verbose name: Graph
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • graph__signature
    Positional arguments accepted by the lambda function. Lambda nodes are rarely needed in NeuroPype since a pairing of a Placeholder and a (name, ...) signature in a node defines a lambda function. However, the Lambda node can be used to return a function from a function, or to declare a order to the arguments of a graph when that graph is invoked with positional arguments. The graph preceding your lambda node must contain one or more Placeholder nodes for each of the arguments listed here, with their respective slotnames set to the argument names listed here. Then, anything downstream of these placeholders collectively constitutes your lambda function body, and the final node of your lambda function body is expected to return the function result. You can also have additional unlisted placeholders in your lambda function body, which are only assignable when the lambda function is invoked with additional keyword arguments. This final node is then connected to the lambda node itself by wiring the desired output port of your final lambda function body node to the "graph" input port of the lambda node. In graphical UIs, this edge will be displayed in dotted style to indicate that this is not normal forward data flow but that the function body (graph) itself is passed to the Lambda node. The Lambda node follows the lambda function body in the graph.

    • verbose name: Graph [Signature]
    • default value: (myargument1,myargument2)
    • port type: Port
    • value type: object (can be None)
  • function
    Resulting lambda function.

    • verbose name: Function
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: OUT
  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

ParallelForEach

Parallel version of the For Each node.

Apply a function or loop body to each item in a collection or iterator. Unlike the (sequential) For Each loop, the loop always behaves in a "stateless" fashion, meaning that loop nodes do not carry over state across iterations. See For Each node for the basic principle of this type of node. The Parallel For Each node allows use of the Continue node to skip individual items, but does not support the Break node, since processing is not sequential. If the node receives an iterator, the node may attempt to exhaust the iterator before any processing can start, therefore it is best used with iterators that are able to deliver all the data without blocking (e.g., waiting for data). The node can run computations for multiple iterations in parallel, using the same options of other parallel nodes, if you specify num_procs as either None (default all cores) or some number >1. You may also experiment with the number of threads that each process may use (num_threads_per_proc), since numpy can easily cause excessive thread churn (100% utilization). If you have multiple GPUs and your pipeline uses one or more GPU backends (e.g., jax or torch), then the node can spread the iterations out over the GPUs, depending on num_procs_per_gpu and compute_backends. Note that pipelines will only use GPUs if you explicitly enable it in specific nodes.

More Info...

Version 1.0.0

Ports/Properties

  • metadata
    User-definable meta-data associated with the node. Usually reserved for technical purposes.

    • verbose name: Metadata
    • default value: {}
    • port type: DictPort
    • value type: dict (can be None)
  • body
    Graph to apply (loop body).

    • verbose name: Body
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • body__signature
    The list of loop variables received by the loop body. In a "For Each" loop, the loop body depends on only a single loop variable, namely the current item from the wired-in iterable. Consequently, your loop body should contain a single Placeholder node whose slotname must match the name listed here (e.g., item, but you can also choose a different name). Then, anything downstream of (i.e., depending on the value of) that Placeholder will constitute your loop body. The final output node of your loop body is then wired into the "body" input port of the loop node. In graphical UIs, the edge will be drawn in dotted style to indicate that the preceding graph itself is given to the loop node as the loop body (which will then execute it zero or more times), which is in contrast to a regular (solid) edge that indices normal data flow wherein first the preceding node runs, and then its output is passed to the subsequent node. The For Each node will then return a collection (see collect setting) of the values received from the final node on each iteration.

    • verbose name: Body [Item Parameter]
    • default value: (item)
    • port type: Port
    • value type: object (can be None)
  • iterable
    Data to iterate over.

    • verbose name: Iterable
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • result
    ForEach result.

    • verbose name: Result
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: OUT
  • collect
    Collect results into a list, dictionary, graph, packet, array or flatten the results into a single container of the same type emitted by the function. For graphs, a mix of nodes and edges may be emitted, and the resulting graph will have only those edges for which both endpoint nodes were also emitted separately (i.e., emitting an edge will not implicitly add its endpoint nodes). If 'last', only the last result is returned (as in Repeat). When applied to numeric arrays, the 'flatten' option concatenates the arrays along their existing first axis while the 'array' option stacks them along a new first axis. Packet and Graph are analogous to dict and list options, in that they are comprehensions that allow the user to itemize the constituents of these data structures (for packets, each is a 2-element list of [chunkname, Chunk] and for Graph each is a nodes and/or an edges). The dict-of-lists option is convenient when the loop body returns a list of dictionaries, and you wish to collect the results as a dictionary of lists; this can be a useful pattern for loop bodies with multiple return values, where the return values are wired into a Create Dictionary node and that dictionary is returned as the loop body result. The loop itself will then return a dictionary that has a list of items for each key.

    • verbose name: Collect
    • default value: list
    • port type: EnumPort
    • value type: str (can be None)
  • num_procs
    Number of processes to use for parallel computation. If None, the global setting NUM_PROC, which defaults to the number of CPUs on the system, will be used.

    • verbose name: Max Parallel Processes
    • default value: None
    • port type: IntPort
    • value type: int (can be None)
  • num_threads_per_proc
    Number of threads to use for each process. This can be used to limit the number of threads by each process to mitigate potential churn.

    • verbose name: Threads Per Process
    • default value: 4
    • port type: IntPort
    • value type: int (can be None)
  • compute_backends
    GPU compute backends that may be used by the pipeline. If you include GPU compute backends here, workloads using those backends will be farmed out across multiple GPUs (if available) when running cross-validation folds in parallel. The 'auto' mode will attempt to auto-detect any backend settings in the given pipeline's nodes, but note that this will only catch nodes where this is explicit in the node's properties, and GPU workloads missed in this fashion will run by default on GPU 0.

    • verbose name: Compute Backends
    • default value: ['auto']
    • port type: SubsetPort
    • value type: list (can be None)
  • num_procs_per_gpu
    Number of processes to use per GPU. This is only relevant if you have GPU compute backends enabled. If your GPU(s) are under-utilized during cross-validation, you can increase this to run this many CV folds on each GPU.

    • verbose name: Processes Per Gpu
    • default value: 1
    • port type: IntPort
    • value type: int (can be None)
  • multiprocess_backend
    Backend to use for farming out computation across multiple (CPU) processes. Multiprocessing is the simple Python default, which is not a bad start. Nestable is a version of multiprocessing that allows your pipeline to itself use parallel computation. If you are getting an error that "daemonic processes cannot have children", the most likely cause is that you have two nested parallel loops, and at least the outer loop needs to be set to nestable. Loky is a fast and fairly stable backend, but it does not support nested parallelism and has different limitations than multiprocessing. It can be helpful to try either if you are running into an issue trying to run something in parallel. Serial means to not run things in parallel but instead in series (even if num_procs is >1), which can help with debugging. Threading uses Python threads in the same process, but this is not recommended for most use cases due to what is known as GIL contention.

    • verbose name: Multiprocess Backend
    • default value: loky
    • port type: EnumPort
    • value type: str (can be None)
  • serial_if_debugger
    If True, then if the Python debugger is detected, the node will run in serial mode, even if multiprocess_backend is set to something else. This is useful for debugging, since the debugger does not work well with parallel processes. This can be disabled if certain steps should nevertheless run in parallel (e.g., to reach a breakpoint more quickly).

    • verbose name: Serial If Debugger
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • materialize
    Materialize parallel instances of the loop body such that they can retain prior state. This is equivalent to creating one parallel copy of the loop body for each of the inputs in the iterable, which therefore persist across successive (re-)entries into the loop. Regardless of the values in the iterable, the k'th value will then be processed by the k'th materialized instance of the loop body. Note that this is currently not compatible with true parallel computation, and requires the multiprocess backend to be set to 'serial'.

    • verbose name: Materialize
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • conserve_memory
    The auto option defaults to 'on' if the backend is set to multiprocessing or nestable, and 'off' otherwise. If enabled, then the memory will be cleared periodically across iterations. This is useful if you are running out of memory during a run, but it will slow down the computation somewhat. The aggressive mode will force memory to be cleared after each iteration, which has the highest overhead, but guarantees a minimal memory footprint. However, the multiprocessing backend is known to occasionally hang in this mode so this may be reverted to on if you experience hangs. The off mode will leave memory reclamation to the individual nodes and scheduler behavior within the loop body, which may be further tuned using e.g. the default_conserve_memory config-file setting or per-node settings.

    • verbose name: Conserve Memory
    • default value: auto
    • port type: EnumPort
    • value type: str (can be None)
  • verbose
    Whether to print verbose output.

    • verbose name: Verbose
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • log_errors
    If True, log exceptions occurring in the loop body.

    • verbose name: Log Errors
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

Placeholder

Declare a named placeholder in a graph.

Using a placeholder means that the downstream node(s) are dependent on a value that will be supplied by another node. As such, the downstream graph becomes parametrically dependent on the placeholder (or multiple placeholders). This means the downstream tree does not immediately execute. Instead, the only way to execute it is to wire it into a node that accepts not a value but a graph, such as the Repeat node. A node that accepts graphs can then execute the graph that it receives zero or more times, and can do so each time with a different value for the placeholder (for example the loop counter in the case of a Repeat node). The graph-accepting node defines what placeholders it wants in its received graph; this is called the signature of the graph. The graph that is received by the function in question is in its initial state (nodes are in their initial state), and the graph-accepting node receives a copy of it. When the node evaluates the graph repeatedly, the state evolves, but when the node itself is re-evaluated, it receives again the graph in its initial state. Note that anything that is wired into the parametric graph "from the side" (i.e., coming from nodes whose value is not dependent on a placeholder in the signature) is "captured" and is evaluated at the time the graph is received by the graph-accepting node, and is not re- evaluated each time the node executes the graph (this is usually what one wants). The customary CS terms for these concepts are as follows: a placeholder is a variable, the graph plus signature is a lambda function (or anonymous function), the graph-accepting node is a higher-order function (HOF), and a graph that captures additional values from side inputs is called a closure.

More Info...

Version 1.0.0

Ports/Properties

  • metadata
    User-definable meta-data associated with the node. Usually reserved for technical purposes.

    • verbose name: Metadata
    • default value: {}
    • port type: DictPort
    • value type: dict (can be None)
  • value
    Data received.

    • verbose name: Value
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: OUT
  • slotname
    Name of the placeholder slot. This needs to be matched to whatever the node that will fill in the placeholder (e.g., a Repeat) will use.

    • verbose name: Name
    • default value: unnamed
    • port type: StringPort
    • value type: str
  • default
    Optional default value.

    • verbose name: Default
    • default value: (unassigned)
    • port type: Port
    • value type: object (can be None)
  • value_type
    Type of the value. This only needs to be specified in function declarations.

    • verbose name: Value Type
    • default value: object
    • port type: ComboPort
    • value type: str (can be None)
  • desc
    Description for the placeholder. Only needs to be specified in function declarations. This is typically a single sentence.

    • verbose name: Description
    • default value:
    • port type: StringPort
    • value type: str (can be None)
  • verbose_name
    Optional verbose name for the placeholder. Can be used in function declarations for a more descriptive name.

    • verbose name: Verbose Name
    • default value: None
    • port type: StringPort
    • value type: str (can be None)
  • supports_unassigned
    Whether this node supports formally unassigned values. If True, the special value Unassigned will be returned when the placeholder is not bound to a value, otherwise an UnevaluableError is raised. This is required for placeholders that are wired into, e.g., CreateList or similar nodes. This must also be set to True for placeholders that reference an option set with WithOptions.

    • verbose name: Supports Unassigned
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • verbose
    Under what conditions to generate verbose output. Default is the current NeuroPype default, which may change in the future. Value prints the value itself and disabled prints nothing. Debug prints the value with a "debug" log level, which allows one to selectively filter the logs out by viewing only the "info" log level.

    • verbose name: Verbose
    • default value: default
    • port type: EnumPort
    • value type: str (can be None)
  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

Result

Declare an output as the result of the graph.

The Result node can be used to designate the value on a particular branch as the output value of the graph, which is returned when the graph is invoked as a function with some arguments. There can only be one Result node the graph, and a Result node cannot be followed by other nodes. If you need to return a combination of multiple values, you can precede it by a CreateStructure node.

Version 1.0.0

Ports/Properties

  • metadata
    User-definable meta-data associated with the node. Usually reserved for technical purposes.

    • verbose name: Metadata
    • default value: {}
    • port type: DictPort
    • value type: dict (can be None)
  • result
    Result of the graph.

    • verbose name: Result
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: INOUT
  • result_asgraph
    Result of the graph, as a graph.

    • verbose name: Result Asgraph
    • default value: None
    • port type: GraphPort
    • value type: Graph (can be None)
  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

Sequential

Ensure that the paths wired into the node are evaluated in a specified order.

Note that the Sequential node does not have a single output, but there are as many outputs as there are inputs. This node is intentionally kept trivial and mainly occurs implicitly in graphs that are automatically generated (e.g., lambda graphs with loose ends). See also NWayPassthrough for a node with more user and quality of life options.

Version 1.0.0

Ports/Properties

  • metadata
    User-definable meta-data associated with the node. Usually reserved for technical purposes.

    • verbose name: Metadata
    • default value: {}
    • port type: DictPort
    • value type: dict (can be None)
  • data1
    Input 1 to evaluate.

    • verbose name: Data1
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: INOUT
  • data2
    Input 2 to evaluate.

    • verbose name: Data2
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: INOUT
  • data3
    Input 3 to evaluate.

    • verbose name: Data3
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: INOUT
  • data4
    Input 4 to evaluate.

    • verbose name: Data4
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: INOUT
  • data5
    Input 5 to evaluate.

    • verbose name: Data5
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: INOUT
  • data6
    Input 6 to evaluate.

    • verbose name: Data6
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: INOUT
  • data7
    Input 7 to evaluate.

    • verbose name: Data7
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: INOUT
  • data8
    Input 8 to evaluate.

    • verbose name: Data8
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: INOUT
  • data9
    Input 9 to evaluate.

    • verbose name: Data9
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: INOUT
  • data10
    Input 10 to evaluate.

    • verbose name: Data10
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: INOUT
  • data11
    Input 11 to evaluate.

    • verbose name: Data11
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: INOUT
  • data12
    Input 12 to evaluate.

    • verbose name: Data12
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: INOUT
  • data13
    Input 13 to evaluate.

    • verbose name: Data13
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: INOUT
  • data14
    Input 14 to evaluate.

    • verbose name: Data14
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: INOUT
  • data15
    Input 15 to evaluate.

    • verbose name: Data15
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: INOUT
  • data16
    Input 16 to evaluate.

    • verbose name: Data16
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: INOUT
  • data17
    Input 17 to evaluate.

    • verbose name: Data17
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: INOUT
  • data18
    Input 18 to evaluate.

    • verbose name: Data18
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: INOUT
  • data19
    Input 19 to evaluate.

    • verbose name: Data19
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: INOUT
  • elidable
    If True, the node will be elided when instantiated in graph mode. The result is a graph with loose ends, but the ends have been added to the graph in sequential order (and thus will evaluate in that order).

    • verbose name: Elidable
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • data0
    Input 0 to evaluate; defunct.

    • verbose name: Data0
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: INOUT
  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

Switch

Switch expression for conditional control.

This node is used as follows: a value is wired into test_value, and then that value is compared with the values set in the match 1 through math 10 parameters, in 1-10 order. If a match is found, the corresponding branch is executed and its value returned and remaining conditions are not evaluated. If no conditions match, then the branch wired into else_branch is executed and its value (the output of its execution) is returned through the result port, which can be wired into a downstream node for processing. If nothing was wired into the condition port that was selected for execution, then the result outputs None. The Switch and ConditionSwitch nodes differ in their use in that the Switch node is used to control the execution of an upstream branch and return its result downstream, whereas the ConditionSwitch is used to control the execution of downstream nodes (usually by wiring it into the Passthrough node to "turn off" execution at a given point in the pipeline).

Version 1.0.0

Ports/Properties

  • metadata
    User-definable meta-data associated with the node. Usually reserved for technical purposes.

    • verbose name: Metadata
    • default value: {}
    • port type: DictPort
    • value type: dict (can be None)
  • test_value
    Value to test.

    • verbose name: Test Value
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: IN
  • condition1
    Test value for branch 1.

    • verbose name: Branch 1 Match
    • default value: None
    • port type: Port
    • value type: object (can be None)
  • branch1
    Branch 1.

    • verbose name: Branch1
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • condition2
    Test value for branch 2.

    • verbose name: Branch 2 Match
    • default value: None
    • port type: Port
    • value type: object (can be None)
  • branch2
    Branch 2.

    • verbose name: Branch2
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • condition3
    Test value for branch 3.

    • verbose name: Branch 3 Match
    • default value: None
    • port type: Port
    • value type: object (can be None)
  • branch3
    Branch 3.

    • verbose name: Branch3
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • condition4
    Test value for branch 4.

    • verbose name: Branch 4 Match
    • default value: None
    • port type: Port
    • value type: object (can be None)
  • branch4
    Branch 4.

    • verbose name: Branch4
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • condition5
    Test value for branch 5.

    • verbose name: Branch 5 Match
    • default value: None
    • port type: Port
    • value type: object (can be None)
  • branch5
    Branch 5.

    • verbose name: Branch5
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • condition6
    Test value for branch 6.

    • verbose name: Branch 6 Match
    • default value: None
    • port type: Port
    • value type: object (can be None)
  • branch6
    Branch 6.

    • verbose name: Branch6
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • condition7
    Test value for branch 7.

    • verbose name: Branch 7 Match
    • default value: None
    • port type: Port
    • value type: object (can be None)
  • branch7
    Branch 7.

    • verbose name: Branch7
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • condition8
    Test value for branch 8.

    • verbose name: Branch 8 Match
    • default value: None
    • port type: Port
    • value type: object (can be None)
  • branch8
    Branch 8.

    • verbose name: Branch8
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • condition9
    Test value for branch 9.

    • verbose name: Branch 9 Match
    • default value: None
    • port type: Port
    • value type: object (can be None)
  • branch9
    Branch 9.

    • verbose name: Branch9
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • condition10
    Test value for branch 10.

    • verbose name: Branch 10 Match
    • default value: None
    • port type: Port
    • value type: object (can be None)
  • branch10
    Branch 10.

    • verbose name: Branch10
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • else_branch
    Else branch (no match).

    • verbose name: Else Branch
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • result
    Result of the branch that was evaluated.

    • verbose name: Result
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: OUT
  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

Throw

Throw an Exception.

This will interrupt the execution at the current point and fall through to the nearest enclosing Exception Guard node that lists the thrown exception type (or a more general one) in its list of exceptions to catch. For the hierarchy of throwable exceptions, see also Python's documentation.

More Info...

Version 1.0.0

Ports/Properties

  • metadata
    User-definable meta-data associated with the node. Usually reserved for technical purposes.

    • verbose name: Metadata
    • default value: {}
    • port type: DictPort
    • value type: dict (can be None)
  • result
    Result of the graph.

    • verbose name: Result
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: OUT
  • exception_type
    Exception type to throw. You can find documentation about these types in the Python documentation.

    • verbose name: Exception Type
    • default value: RuntimeError
    • port type: ComboPort
    • value type: str (can be None)
  • message
    Message to include in the exception.

    • verbose name: Message
    • default value: Exception message
    • port type: StringPort
    • value type: str (can be None)
  • cause
    Causing exception. If this is set, the provided exception will be recorded as the cause of the exception being thrown. This is useful for chaining exceptions together.

    • verbose name: Cause
    • default value: None
    • port type: Port
    • value type: Exception (can be None)
  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

While

Evaluate the loop body while the condition holds.

This node receives two graphs, each driven by a loop state parameter -- a condition graph and a loop body graph. The basic principle of the while loop is that the entire variable state that the loop acts on is captured in the state variable, which can have any internal structure that the user desires, e.g., a number, list, dictionary, or any other object. The while loop requires two separate graphs wired into it, each of which must accept the state variable as input (via an appropriately named placeholder). The first of these is the "loop condition" -- a graph that takes the state and computes whether the loop should make another loop cycle or terminate. The second graph is the "loop body" -- a graph that takes also the state and computes the new state that will be passed into the next iteration. At the next iteration, again the condition is evaluated first, and if it is True, the loop body is executed. Once the condition no longer holds, the loop terminates, and the node returns the final state value that it received from the loop body on the most recent (final) iteration when it still ran. Note that, if the condition is False from the start, the loop body will not be executed at all. The loop state is by default called "state", but can be overridden using the parameter showing as "Body [loop state]" in a GUI, or equivalently via the Node's body__signature= constructor argument when creating a While node in a Python script. Note that when you rename the state variable, you also need to rename the slot name in the Placeholder nodes used by the condition and body graphs. Note also that of course the loop body (or condition) can depend on any number of other inputs, but these are not allowed to vary over the course of the loop (these are, as is customary in NeuroPype, evaluated before the loop starts). The loop state, on the other hand, is allowed to vary, and it is best practice to keep the structure of the loop state consistent over the course of the loop (i.e., ideally the initial state already has the same structure as the state returned by the loop body). An advanced technique to control execution of the loop is to use the Break and Continue nodes, which can be placed in the loop body to terminate the loop early or skip the rest of the current iteration and proceed to the next one (of these, the Continue node will cause the same state to be passed in again, which only makes sense if the loop body contains nodes that themselves are stateful). When the loop is used in a compiled context (high-performance nodes such as deep learning or convex optimization), this is required and enforced. In this case it is also not allowed to use Break or Continue nodes in the loop body. Note that it is possible to have nodes in the loop body which themselves maintain state (e.g., filters). This is fine, but note that these nodes will be re-initialized when the While loop is encountered again (e.g., due to an outer enclosing loop); also note that stateful nodes of this sort are also not currently compatible with compiled loops. It is possible to omit the loop condition graph, in which case the loop terminates when the loop body declares itself as finished (for example, when all iterators are exhausted); this is however not recommended, since the finished state can at times be somewhat complex to reason about and may not be what you expect.

More Info...

Version 1.0.0

Ports/Properties

  • metadata
    User-definable meta-data associated with the node. Usually reserved for technical purposes.

    • verbose name: Metadata
    • default value: {}
    • port type: DictPort
    • value type: dict (can be None)
  • condition
    Loop condition.

    • verbose name: Condition
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • condition__signature
    Name of the loop variable received by the loop body. In a While loop, the loop body depends on only a single loop variable, namely the current state. Consequently, your loop body should contain a single Placeholder node whose slotname must be set to the name chosen here (e.g., state). Then, anything downstream of (i.e., depending on the value of) that Placeholder will constitute your loop body. The final output node of your loop body is then wired into the "body" input port of the loop node. In graphical UIs, the edge will be drawn in dotted style to indicate that the preceding graph itself is given to the loop node as the loop body (which will then execute it zero or more times), which is in contrast to a regular (solid) edge that indices normal data flow wherein first the preceding node runs, and then its output is passed to the subsequent node. The While node will then run your loop body for as long as the condition evaluates to True, and eventually returns the value received from the final node of the loop body on the last iteration.

    • verbose name: Condition [Loop State]
    • default value: (state)
    • port type: Port
    • value type: object (can be None)
  • body
    Graph to repeat (loop body).

    • verbose name: Body
    • default value: None
    • port type: GraphPort
    • value type: Graph
  • body__signature
    Name of the loop variable received by the loop body. In a While loop, the loop body depends on only a single loop variable, namely the current state. Consequently, your loop body should contain a single Placeholder node whose slotname must be set to the name chosen here (e.g., state). Then, anything downstream of (i.e., depending on the value of) that Placeholder will constitute your loop body. The final output node of your loop body is then wired into the "body" input port of the loop node. In graphical UIs, the edge will be drawn in dotted style to indicate that the preceding graph itself is given to the loop node as the loop body (which will then execute it zero or more times), which is in contrast to a regular (solid) edge that indices normal data flow wherein first the preceding node runs, and then its output is passed to the subsequent node. The While node will then run your loop body for as long as the condition evaluates to True, and eventually returns the value received from the final node of the loop body on the last iteration.

    • verbose name: Body [Loop State]
    • default value: (state)
    • port type: Port
    • value type: object (can be None)
  • result
    Result of last operation.

    • verbose name: Result
    • default value: None
    • port type: DataPort
    • value type: object (can be None)
    • data direction: OUT
  • initial
    Initial value for the state. It is recommended that this has the same structure (e.g., dictionary fields) as what is returned by the loop body. For compiled loops, this is required.

    • verbose name: Initial
    • default value: None
    • port type: Port
    • value type: object (can be None)
  • compile
    Whether and how to compile the loop body for more efficient execution. Auto will compile the loop only if it occurs in a context where compilation is necessary, for example inside a model passed to nodes such as DeepLearning, ConvexModel, or one of the Inference nodes. The 'jax' option will generally attempt to compile the loop to run on the jax backend; this comes with a series of limitations -- all nodes in the body have to work with jax data types (numeric values only) and operations, and break/continue nodes cannot be used. 'Off' is the appropriate setting for anything except for the most performance-critical computations consisting of mainly math operations, possibly with some data reformatting. Off can also be useful in a situation where the loop occurs in a compiled context, but the iterable is a static constant (e.g., fixed list) and relatively short; in this case, the loop will be completely unrolled, which can be more efficient than actually looping.

    • verbose name: Compile
    • default value: auto
    • port type: EnumPort
    • value type: str (can be None)
  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • log_errors
    If True, log exceptions occurring in the loop body.

    • verbose name: Log Errors
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)