Module: utilities
Utility nodes and miscellanea.
Includes a variety of nodes that do not fall into any of the other categories, such as nodes that fulfill certain special roles in data-flow pipelines, such as editing the meta-data, or maintaining real-time update rates.
AlignInstances
Line up streams with instance axes such that their instances and their order match.
This node will delete and sort instances in multiple streams (for example, collections of segments from different parts of a trial), so that the instance meta-data in one or more desired fields matches at every index for all streams. For example, if in one stream some instances are missing and in another stream some others are missing, this node will delete those instances to make sure there's a 1:1 correspondence between the two streams. Note that currently the node will always sort by the given fields, so if order matters, you may include some trial counter.
Version 0.7.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data1
Data to process 1.- verbose name: Data1
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
data2
Data to process 2.- verbose name: Data2
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
data3
Data to process 3.- verbose name: Data3
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
data4
Data to process 4.- verbose name: Data4
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
data5
Data to process 5.- verbose name: Data5
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
data6
Data to process 6.- verbose name: Data6
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
data7
Data to process 7.- verbose name: Data7
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
data8
Data to process 8.- verbose name: Data8
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
data9
Data to process 9.- verbose name: Data9
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
data10
Data to process 10.- verbose name: Data10
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
dataN
Additional input data.- verbose name: Datan
- default value: None
- port type: DataPort
- value type: list (can be None)
- data direction: INOUT
-
matched_fields
Instances that have the same combination of values for these fields are considered matched. In the result, instance k of each stream will have the same combination of values across all streams (note that the values in each field can be, and likely are, different).- verbose name: Matched Fields
- default value: ['Marker']
- port type: ListPort
- value type: list (can be None)
-
handle_duplicates
How to handle duplicate instances (with same value in the matched fields) in some stream. In 'take-first' mode, the first of the duplicated instances is taken from each stream, so that each unique combination occurs only once per stream. In the 'repeat-others' mode, if a stream has duplicate instances, and all other streams have each either at most one matched instance, or the same number of instances, then those instances are repeated where necessary.- verbose name: Handle Duplicates
- default value: take-first
- port type: EnumPort
- value type: str (can be None)
-
skip_marker_streams
Whether to skip processing marker streams.- verbose name: Skip Marker Streams
- default value: True
- port type: BoolPort
- value type: bool (can be None)
-
sort_by
Sort instances by this key.- verbose name: Sort By
- default value: matched-fields
- port type: EnumPort
- value type: str (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
AppendSyntheticChannels
Append channels defined by a time-dependent formula to a time series.
This node can be used to append certain types of artificial channels to the data, for example test data, or nuisance regressors (e.g., for subsequent use in a convolutional regression model). The node can instead also create a new stream with such channels that inherits the time axis from a source stream stream.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
family
function family. Constant is a simple constant value, multiplied by the given parameter, otherwise a constant 1. Linear is a linear trend that starts at the first parameter value and ends at second parameter value. If not given, it goes from -sqrt(3) to +sqrt(3) so as to have unit variance and zero mean. dct-2 (Type-II DCT) form an orthonormal basis of cosine functions up to a given limit frequency (the given parameter), or between a low and a high cutoff frequency (if two parameters given). Legendre polynomials are orthonormal polynomials up to (and including) a given degree (e.g., 3), which can model non-linear trends in the data.- verbose name: Family
- default value: constant
- port type: EnumPort
- value type: str (can be None)
-
params
Parameters for the function family. If a family requires only one parameter, this is a one-element list. See family.- verbose name: Params
- default value: None
- port type: ListPort
- value type: list (can be None)
-
reference_stream
Name (or name prefix) of the stream from which to take the time axis. The first matching stream that also has a time axis will be used. Note that if no new stream is created and channels are appended to this stream, then the stream also needs to have a space axis.- verbose name: Reference Stream
- default value: None
- port type: StringPort
- value type: str (can be None)
-
channel_axis
Axis to use to enumerate the newly-generated channels. If not specified, this defaults to feature if the ref data has a feature axis, and otherwise space.- verbose name: Channel Axis
- default value: auto
- port type: EnumPort
- value type: str (can be None)
-
new_stream
Name of the new stream to create, or empty if channels shall be appended to the reference stream. In either case, the newly generated data will be on the same compute backend as the reference stream's data.- verbose name: Optional New Stream
- default value: synthetic
- port type: StringPort
- value type: str (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
ApproximateChunkRate
Approximate a particular rate of chunks per second in the output.
This node can be used to reduce the effective update rate of a streaming pipeline, for example to 5 Hz. As a result, the node will only output new data 5 times a second. However, since the input to this node comes in at a certain update rate, if the two rates are not divisors of each other, the output will not be exactly 5 Hz (e.g., every 3 cycles, the data may be empty). To get a more regular chunk rate, the node Regulate Chunk Sizes can be used after this node.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
desired_rate
Number of chunks per second. This node will output approximately this many chunks per second.- verbose name: Chunks Per Second
- default value: 1
- port type: FloatPort
- value type: float (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
AssignStandardChannelNames
Assigns channel names to incoming channels based on known montages of specific EEG devices, which can be selected from a list.
This acts on all streams in the data packet. If the selected device is 'none', data will pass through unchanged.
Version 1.1.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
device
Select the device to use as the channel name template. The number of incoming channels must match the number shown in the device name. Set to none to skip this node.- verbose name: Device
- default value: none
- port type: EnumPort
- value type: str (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
BatchPackets
Concatenate multiple successive packets to maintain real-time rates.
Many processing nodes in NeuroPype have a fixed per-chunk overhead, and therefore, if fewer but longer chunks are being processed, this overhead goes down, which can make the difference between the pipeline running in real time or not (note though that, in some cases, even when all overhead is eliminated, the pipeline can still run slower than real time if the cost of processing the actual streaming data is high enough, especially with high sampling rates, many channels, and heavy processing nodes; in such cases, using this node will not be sufficient). Normally, the pipeline is driven by input plugins that output all data since the last update (e.g., LSL), which naturally increases the chunk size as necessary to maintain real-time rates, and in these cases this node is not necessary. However, when the pipeline is driven with a fixed chunk size that is too small for real time, this node can be inserted to merge successive chunks in order to achieve real-time behavior. Since this node concatenates chunks, the meta-data properties of the output chunk are picked from the last chunk that was concatenated. In general it is recommended to use the fixed
batching mode and pick a batch size that is enough to make the pipeline real time. It is also possible to use the realtime
mode, in which the first signal stream with time axis and no instance axis is used to infer by how much the stream is lagging relative to the current clock, and the chunk size is chosen adaptively. However, this requires that this node uses the same clock as the one from which the time stamps in the stream's time axis were read off -- otherwise the batching will misbehave badly due to clock drift and offset. This node offers multiple different clocks that you can choose from for this purpose. The simplest alternative to using this node is to instead use an input node that does return all new data since the last update, instead of a fixed chunk size.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
batching
Batching mode. If set to 'fixed', then a fixed number of successive packets (defined inbatch size
) are concatenated to form a single packet. If set torealtime
, then as many packets are concatenated as necessary to maintain real-time processing (batch size is adaptive). Note that you need to select the correct clock (inclock type
) forrealtime
mode.- verbose name: Batching Mode
- default value: fixed
- port type: EnumPort
- value type: str (can be None)
-
batchsize
Number of successive packets to merge. Only used if batching is set to 'fixed'.- verbose name: Batch Size (Fixed Mode)
- default value: 1
- port type: IntPort
- value type: int (can be None)
-
realtime_clock
Type of real-time clock to use when using 'realtime' batching mode. This must be the same clock that was used to time-stamp the data that is received by this node. The two available clocks are the UTC clock (wall clock) and the LSL clock. When using UTC and the data comes from another computer, both clocks must be synchronized, e.g., using NTP. The LSL clock is synchronized across computers, if LSL was used to transmit the data.- verbose name: Clock Type (Realtime Mode)
- default value: UTC
- port type: EnumPort
- value type: str (can be None)
-
max_input_lag
Maximum acceptable input packet lag. If the lag is larger than this, the node will stash the packet and batch it with the next one (inrealtime
mode). Unit is seconds.- verbose name: Max Input Lag (Realtime Mode)
- default value: 0.1
- port type: FloatPort
- value type: float (can be None)
-
max_output_lag
Maximum acceptable delay incurred by batching packets, in seconds. If this amount of lag between the stream's data and the selected clock is exceeded, a packet will be emitted. This limits the maximum chunk size that can be generated, but if this threshold is reached, your processing system has problems (either your pipeline is not fast enough for real time, or you use the wrong clock).- verbose name: Max Output Lag (Realtime Mode)
- default value: 1
- port type: FloatPort
- value type: float (can be None)
-
axis
Axis along which to concatenate the data packets. If set toauto
, concatenation is along the instance axis if present, or time axis otherwise.- verbose name: Concatenate Axis
- default value: auto
- port type: EnumPort
- value type: str (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
BlockReset
Prevents subsequent nodes from receiving a reset signal from the preceding node.
This is useful when a node emits data marked as requiring a reset (e.g., because a property such as an axis changed), but we do not want a subsequent node to clear its state in response, e.g. perhaps because the reset is benign.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
ConcatSuccessivePackets
Concatenate successive packets along some axis.
This node will buffer any packets it receives (to its data
port), until it receives a None, at which time it will concatenate the buffered packets into a single packet and pass that on (it has no output prior to that). The packets must all have the same shape along all axes except the concatenation axis (though if this is not the case, you can choose to pad the incongruent axes with a zero or NaN value using the padding_value
property). By default this will create a new axis of the desired type (e.g., space
or time
) and concatenate along that axis, but you can reuse an existing axis (i.e., if you don't need to retain the original axis or don't want two axes of the same type), by setting create_new
to False. This node also has a no_buffering
property which allows you to send it a list of packets to immediately concatenate and pass on (primarily for use in an offline pipeline).
Version 1.1.2
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
dataN
A list of packets to concatenate (if no_buffering==True).- verbose name: Datan
- default value: None
- port type: DataPort
- value type: list (can be None)
- data direction: IN
-
axis
Axis along which to concatenate data.- verbose name: Concatenate Along Axis
- default value: time
- port type: ComboPort
- value type: str (can be None)
-
create_new
Whether to create a new axis. If enabled, a new axis of the desired type will be inserted and data will be concatenated along that axis.- verbose name: Append New Axis
- default value: True
- port type: BoolPort
- value type: bool (can be None)
-
new_axis_label
Label for the new axis along which to concatenate. Only applies if append new axis is set.- verbose name: New Axis Label
- default value: concatenated
- port type: StringPort
- value type: str (can be None)
-
properties
Values for the primary property of the new axis, if any. For instance, this would be a list of channel names if the axis is 'space'. If not given and a new axis is inserted, successive integers starting from 0 will be used as the values (so if this is e.g., a frequency axis, then the frequencies will be 0, 1, 2, and so on up to the number of packets that are concatenated.- verbose name: Values For New Axis
- default value: None
- port type: ListPort
- value type: list (can be None)
-
padding_value
Value to use for padding, optional. If tensors to be concatenated differ in shape, concatenation is not possible unless each tensor is padded as necessary to make the dimensions compatible. If this is set toraise
, tensors of different sizes will raise an error in this node. If set toinvalid
, the data will be filled with invalid data (NaN
). If set tozero
, the fill value will be zeroes.- verbose name: Padding Value
- default value: raise
- port type: EnumPort
- value type: str (can be None)
-
nominal_gap
If axis istime
, number of seconds to use as a gap between concatenated packets. This allows for algorithms such as filtering to be aware of a discontinuity in the time series.- verbose name: Nominal Gap
- default value: 0.5
- port type: FloatPort
- value type: float (can be None)
-
handle_markers
If axis is 'time' and markers chunks are found with no 'time' axis but with 'instance' axis, then concatenate along the marker chunks' 'instance' axis.- verbose name: Handle Markers
- default value: True
- port type: BoolPort
- value type: bool (can be None)
-
no_buffering
If true, and a list of packets is passed to thedataN
port, no buffering will occur and the packets received will be immediately concatenated and passed on. Any single packets received in thedataN
port will be passed along without any concatenation (and a warning will be triggered). This feature is primarily useful when running an "offline" Neuropype pipeline.- verbose name: No Buffering
- default value: False
- port type: BoolPort
- value type: bool (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
CountAlongAxis
Returns the number of items contained in an axis.
Can be used, for example, to return the number of trials or time points in the data. This can also be used to count the number of events in a marker stream. Note that this will return None if there is no matching stream.
Version 1.2.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: AnyNumeric (can be None)
- data direction: IN
-
num_items
Number of items in the given axis- verbose name: Num Items
- default value: None
- port type: DataPort
- value type: int (can be None)
- data direction: OUT
-
axis
Axis along which to count. Select the space axis for channels, instance axis for trials or segments, time for the number of time points, feature for the number of features in the stream, frequency for the number of frequencies, etc. You can also enter the axis label here as in feature.mylabel to further narrow down along which axis to count if there are multiple of the same type. If the incoming data is a plain array instead of a packet, or you know the order of axes, you can also pass an integer as the index of the target axis. The default value will resolves to the 0th axis in a plain array.- verbose name: Axis
- default value: instance
- port type: ComboPort
- value type: str (can be None)
-
stream
The name of the stream on which to operate. If unspecified, first stream containing the specified axis will be counted. Wildcard characters (? *) are allowed for pattern matching, i.e.,mystream*
will match all streams that start with mystream.- verbose name: Stream
- default value:
- port type: StringPort
- value type: str (can be None)
-
stream_name
- verbose name: Stream Name
- default value: None
- port type: AliasPort
- value type: str (can be None)
-
allow_markers
If True and axis='instance' this will count the number of event markers if the first matching stream is an event stream.- verbose name: Allow Markers
- default value: True
- port type: BoolPort
- value type: bool (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
CreateDatestamp
Create a new datestamp from the current time, or a given timestamp, or constituent parts.
To retrieve the current time in a desired time zone, leave all fields unspecified except for the time zone. To convert a POSIX (UNIX) time stamp (integer or float) into a datestamp, pass the timestamp and the desired time zone in which you want your datestamp to be expressed. To create a datestamp from constituent parts, populate some or all input fields with the desired values (at least year, month, and day must then be provided, the others assume default values). The resulting object can also be interrogated and updated via the GetField and GetNested or SetNested nodes, where the same fields, i.e., :year, :hour, etc are available. The timezone can be retrieved via :timezone_name and written via :tz. This can also be applied to data structure(s) of timestamps or e.g., arrays of hours, minutes, etc.
Version 0.5.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
datestamp
Datestamp(s).- verbose name: Datestamp
- default value: None
- port type: DataPort
- value type: AnyNumeric (can be None)
- data direction: OUT
-
timestamp
POSIX timestamp(s).- verbose name: Timestamp
- default value: None
- port type: DataPort
- value type: AnyNumeric (can be None)
- data direction: IN
-
timezone
Desired timezone. If a POSIX timestamp is given (which is always in UTC), then the stamp is translated into the desired time zone. If the current time is requested, this is the current time in that time zone. Time zones are formatted as in US/Pacific, Europe/London, and so forth.- verbose name: Timezone
- default value: UTC
- port type: ComboPort
- value type: str
-
year
Year.- verbose name: Year
- default value: None
- port type: Port
- value type: int (can be None)
-
month
Month.- verbose name: Month
- default value: None
- port type: Port
- value type: int (can be None)
-
day
Day.- verbose name: Day
- default value: None
- port type: Port
- value type: int (can be None)
-
hour
Hour.- verbose name: Hour
- default value: None
- port type: Port
- value type: int (can be None)
-
minute
Minute.- verbose name: Minute
- default value: None
- port type: Port
- value type: int (can be None)
-
second
Second.- verbose name: Second
- default value: None
- port type: Port
- value type: int (can be None)
-
microsecond
Microsecond.- verbose name: Microsecond
- default value: None
- port type: Port
- value type: int (can be None)
-
fold
DST Fold (0 or 1). For times that occur twice in a given year (when the clock is reversed for daylight savings time), this identifies whether this is the first or second occurrence of that time. When a datestamp is constructed from scratch, and this is not specified, it is assumed to be the second occurrence (i.e., 1).- verbose name: Fold (Dst)
- default value: None
- port type: Port
- value type: int (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
DejitterTimestamps
De-jitter the time-stamps of the given data.
This can be useful (or necessary) when working with streams whose time stamps are jittered (e.g., because they were not time-stamped by the hardware but stamps were read off the computer clock at receive time, in which case they are subject to jitter from the operating system and driver stack). This is particularly important when multiple streams need to be time-synchronized, such as event markers versus data, and if you plan to segment the data into epochs around events (e.g., using the Segmentation node). Note that this filter should be used before the removal of bad time windows from the data (e.g., using the RemoveBadTimeWindows node), as otherwise the time stamps will be skewed. This node will work on both streaming and non-streaming data, the latter being processed incrementally (similar to how streaming data is processed). If you're importing an XDF file, the recommended approach is to apply de-jittering with the ImportXDF node (on by default) which is more accurate on non-streaming data since it can use the full data set, instead of using this node after import. (Note that the ImportFile node does not do de-jittering, only ImportXDF.)
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
forget_halftime
Exponential forget factor. This is the time, in seconds, until a past measurement will be weighted by 1/2. This value ideally roughly corresponds to the rate at which the sampling rate changes due to things like room temperature fluctuations, but excessively large values can cause numeric problems.- verbose name: Forget Factor (Halftime)
- default value: 90
- port type: FloatPort
- value type: float (can be None)
-
force_monotonic
Enforce monotonic timestamps. This is needed for certain subsequent filters that assume that time-stamps are monotonically increasing (you will see warnings in subsequent filters when data does not satisfy this requirement).- verbose name: Enforce Monotonic Timestamps
- default value: True
- port type: BoolPort
- value type: bool (can be None)
-
max_updaterate
Max internal update rate in Hz. If the signal's sampling rate is higher than this, updates will only be performed on every k'th sample (k being an integer). If this is set very high and a very high rate signal is dejittered (e.g., audio), this node can consume a significant amount of CPU.- verbose name: Max Internal Update Rate
- default value: 500
- port type: IntPort
- value type: int (can be None)
-
warmup_samples
Number of warmup samples. The number of samples for which we warm up the timing statistics. The first few samples will be updated in a blockwise manner, because there will be significant uncertainty in the first few samples, followed by regular sample-wise updates once statistics have settled. If set to -1, this will equal the forget_halftime in samples.- verbose name: Number Of Warmup Samples
- default value: -1
- port type: IntPort
- value type: int (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
DiscardEmptyData
Discard data that is empty.
This includes empty data structures such as lists, dicts (structures), arrays, axes, blocks, chunks, packets with no chunks, packets with only empty chunks, and graphs with no nodes. In all those cases this node will output None.
Version 0.5.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: INOUT
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
DiscardInvalidData
Remove slices of the incoming packets that contain invalid data.
This node can use various criteria to determine whether some data in a chunk is invalid (e.g., contains any infinite values or NaN values). However, since the data is a multi-dimensional array, one cannot just cut out a single number -- instead, a whole slice needs to be removed, and you can select whether you want to remove the affected channel(s), time range, instances, and so on, by selecting the desired axis along which to prune. It is important to note that this node will not remove a different set of e.g., channels on each successive chunk that it is getting (this would break subsequent processing). Instead, this node determines what to remove when it receives a non-streaming chunk ( e.g., an imported recording or some calibration data), and then carries that removal mask over unchanged to any subsequent streaming chunk. Note that these are some relatively basic criteria to identify completely invalid data. This node does not do the kind of sophisticated statistical analysis that dedicated bad-channel removal or bad-time window removal nodes perform.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
axis
Axis along which to remove data. For instance, if this is set to space, then channels with invalid data will be removed.- verbose name: Prune Along Axis
- default value: space
- port type: ComboPort
- value type: str (can be None)
-
criteria
Pruning criteria. This is a list of multiple possible criteria, formatted as in all or some of: ['has-inf', 'has-nan', 'all-nan', 'all-zero'].- verbose name: Criteria
- default value: ['has-inf', 'has-nan', 'all-nan', 'all-zero']
- port type: ListPort
- value type: list (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
DiscardLongChunks
Drop chunks whose length exceeds a threshold.
This can be useful to drop excessively long chunks that may cause a hitch if they enter a slow processing function. Such chunks can occur, for instance, after a node spent a long time calibrating itself.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
threshold
Threshold in seconds. Chunks that are longer than this will be dropped.- verbose name: Threshold
- default value: 10
- port type: FloatPort
- value type: float (can be None)
-
entire_packet
Discard whole packet instead of individual chunks.- verbose name: Entire Packet
- default value: True
- port type: BoolPort
- value type: bool (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
DiscardNaNChunks
Drop chunks that contain NaNs from the data.
This is for streaming processing.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
entire_packet
Discard whole packet instead of individual chunks.- verbose name: Entire Packet
- default value: True
- port type: BoolPort
- value type: bool (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
ExtractChannels
Extract a list of the channel names from a data stream.
Version 1.0.1
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: IN
-
channel_names
Channel Names- verbose name: Channel Names
- default value: None
- port type: DataPort
- value type: list (can be None)
- data direction: OUT
-
stream
Name of stream from which to extract the channel names. If blank will extract from the first stream.- verbose name: Stream
- default value:
- port type: StringPort
- value type: str (can be None)
-
verbose
Verbose output.- verbose name: Verbose
- default value: False
- port type: BoolPort
- value type: bool (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
ExtractStreamProperty
Extract a given property from one of the streams of an incoming data packet.
In NeuroPype, data packets have one or more streams (internally of class Chunk), and each of those streams has meta-data properties (for example, file path from which the data was originally imported, whether the data is streaming or not, and so on). Some properties are set when a Packet is created, but you can set your own properties using the SetStreamProperties node. This node can extract (retrieve) a desired property from a particular stream in a data packet, by specifying the stream name (with wildcard support). Properties may also be stored as nested dictionaries in the stream. To extract a nested property, use periods in the property name, i.e.: 'experiment.task.label'. This node Returns None if the target property is not found in the specified stream (or in the Packet if not stream is specified).
Version 1.2.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Input packet.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: IN
-
property
Extracted property.- verbose name: Property
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: OUT
-
stream
The name of the stream containing the target property. If not specified, the first stream in the packet will be used. Wildcards are supported (somystream*
will use the first stream that starts with mystream.- verbose name: Stream
- default value: None
- port type: StringPort
- value type: str (can be None)
-
stream_name
- verbose name: Stream Name
- default value: None
- port type: AliasPort
- value type: str (can be None)
-
property_name
Stream property to extract (retrieve) the value of.- verbose name: Property Name
- default value: source_url
- port type: StringPort
- value type: str (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
ExtractTimestamps
Extract the timestamps of the given signal.
This extracts the timestamps of the signal as a feature vector. This will create a feature axis.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
FormatDatestamp
Format a datestamp into a string using either a named date form, or a custom format string using placeholders.
Formatting can additionally be localized using the locale parameter. This can also be applied to data structures(s) of datestamps.
Version 0.5.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
datestamp
Input datestamp(s).- verbose name: Datestamp
- default value: None
- port type: DataPort
- value type: AnyNumeric (can be None)
- data direction: IN
-
string
Formatted result string(s).- verbose name: String
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: OUT
-
format
Format for the generated time stamps. This can either be one of the named formats, or a custom format string using placeholders. The basic placeholders are Y, Q, M, D for the numeric year, quarter, month, and day, respectively, Qo, Mo, Do for their ordinal forms (1st, 2nd etc), H, h, m, s for the hour (24- or 12-hour based), minute, and second, S for fractional seconds, d for day of the week, and A for AM/PM. Repeat characters like YYYY, ddd yield respectively more detailed forms of these components. For the time zone, use Z or ZZ for the time zone offset, and z or zz for the time zone name or abbreviation. A full listing of these placeholders can be found in the Pendulum documentation (https://pendulum.eustace.io/docs/#tokens).- verbose name: Format
- default value: default
- port type: ComboPort
- value type: str (can be None)
-
locale
Locale for parsing of month and day names. Note that additional locales may be supported by the implementation.- verbose name: Locale For Date Format
- default value: en
- port type: ComboPort
- value type: str (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
FusePredictionsAndMarkers
""Fuse a BCI's predictions and some markers together into a stream that can be evaluated by MeasureLoss.
This node can be used to rejoin a stream of numeric predictions that were generated in response to event markers (e.g., in a standard machine-learning setup involving marker-locked segmentation followed by feature extraction and an ML node) with a second stream that holds the original event markers. The node copies any per-marker metadata (in the instance axis of the marker stream) to the instance axis of the predictions stream. The underlying assumption of the node is that the predictions have the same or sufficiently similar per-instance time stamps as the event markers to allow for unambiguous matching. Crucially, the node seamlessly handles predictions that arrive much later, in a wallclock sense, than the event markers (despite being labeled with the original time stamps), which is typical when the predictions depend on data that lags the event markers by several seconds (e.g., because the segment extracted relative to an event marker extends into the future). This node can be used, for example, in simulated-online test setups where a BCI pipeline's marker-locked outputs are checked against known ground truth event markers. The resulting stream can be fed into MeasureLoss (or other nodes) to evaluate for instance the accuracy of the predictions. Note that this node considers only event markers that have their TargetValue field set, i.e., you should have called AssignTargetMarkers on your marker stream prior to feeding into this node, in order to select the subset of markers that should have generated predictions.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
max_delay
Max delay of predictions with respect to markers. We assume that predictions cannot appear later than this following a time-locking event marker.- verbose name: Max Delay
- default value: 2
- port type: FloatPort
- value type: float (can be None)
-
min_delay
Min delay of predictions with respect to markers. We assume that predictions cannot appear earlier than this following a time-locking event marker.- verbose name: Min Delay
- default value: -2
- port type: FloatPort
- value type: float (can be None)
-
warn_tolerance
Max timing tolerance. Will warn if the discrepancy in time stamp between prediction and marker was larger than this value. This can be used to catch unexplained discrepancies in logical timing, as well as possible overlap of multiple (perhaps unexpected) candidate event markers or predictions, resulting in possible ambiguity.- verbose name: Warn Tolerance
- default value: 0.1
- port type: FloatPort
- value type: float (can be None)
-
drop_delay
Drop markers older than this. Any event markers that were not successfully associated with a prediction and which are too old will be dropped (and a warning will be generated).- verbose name: Drop Delay
- default value: 30
- port type: FloatPort
- value type: float (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
GetSamplingRate
Get the sampling rate of the given data.
This will retrieve the sampling rate from the first qualifying stream, which is the first stream that has a non-empty time axis, and optionally whose name starts with the given prefix. If no stream is found, or if the sampling rate is irregular, None is returned.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: IN
-
srate
Sampling rate- verbose name: Srate
- default value: None
- port type: DataPort
- value type: float (can be None)
- data direction: OUT
-
stream
Name of stream from which to extract the channel names or prefix thereof. If blank will extract from the first qualifying stream.- verbose name: Stream
- default value:
- port type: StringPort
- value type: str (can be None)
-
verbose
Generate verbose output.- verbose name: Verbose
- default value: False
- port type: BoolPort
- value type: bool (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
GetSourceFilename
Get the full path (filename) of the source of a packet.
This node extracts source_url property of the stream. If the source_url property has a file:// prefix (typical if originally stored via LSL), it will be stripped. The path is output as a string through the filename port. (For more flexibility or for extracting a filename or other information stored in other stream properties, use the GetStreamProperty node, followed by the StringFormat node if string manipulation is needed.)
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: IN
-
filename
The filename extracted from the source_url property.- verbose name: Filename
- default value: None
- port type: DataPort
- value type: str (can be None)
- data direction: OUT
-
stream
Optionally, the name of stream from which to fetch the source filename. Leave empty to use the stream.- verbose name: Stream
- default value: None
- port type: StringPort
- value type: str (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
GetStreamProperty
Get (fetch) a metadata property of a given stream.
In NeuroPype, each data Packet has one or more streams (internally of type "Chunk") carrying data such as EEG or event markers. Each such stream (chunk) has a dictionary of metadata properties, holding information such as the modality of the data (EEG, NIRS, etc.), the origin of the data (file path that the data was originally loaded from), the name of the original LSL stream, the delay incurred by the applied filters, and so on. There are preset properties such as 'is_streaming', 'source_url', 'modality', 'subject_id', etc., but any custom property can also be set using the SetStreamProperties node (or when creating the originating LSL stream). You may specify a stream name or modality (i.e., EEG) to fetch properties from that stream, or else the first stream will be searched. Multi-level properties can be specified using a period (.) as a delimiter: subject.hair.length
will fetch nested property <desc><subject><hair><length>
. The value will be available through the prop_value
output port. If the property does not exist, the output will be None.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: IN
-
prop_value
Value of the fetched property.- verbose name: Prop Value
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: OUT
-
property
Property to get (fetch). I.e., 'is_streaming', 'source_url', 'modality', `subject_id', 'meta_unit', etc. See Flags, Origin and Metadata constants in the documentation for a list of commonly used properties, which are also available from the drop-down menu. You can also type in a custom property name. Use a dot (.) to indicate nested property names, i.e.: 'metadata.cap.size'. Flags (i.e., 'is_streaming') will be returned as a boolean value.- verbose name: Property
- default value: None
- port type: ComboPort
- value type: str (can be None)
-
stream
Optionally, the name of stream from which to fetch the property. Leave empty to find the first stream matching the modality (if specified), or first overall (if modality is not specified).- verbose name: Stream
- default value: None
- port type: StringPort
- value type: str (can be None)
-
modality
Optionally, the modality of the stream to fetch properties from. The first stream matching this modality will be returned. Ignored if stream name is specified.- verbose name: Modality
- default value: None
- port type: ComboPort
- value type: str (can be None)
-
only_nonempty
Only consider streams with non-empty data.- verbose name: Only Non-Empty Streams
- default value: False
- port type: BoolPort
- value type: bool (can be None)
-
handle_missing
Action to take if the property is not found. If 'none', the output will be None with no log message (nodes such as If/Else will interpret None as False). If 'warn', the output will be None with a warning. If 'error', an error will be raised.- verbose name: Handle Missing
- default value: warn
- port type: ComboPort
- value type: str (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
GetTimeShifts
Retrieve the time shifts between a chunk's time axis and another set of timestamps.
Version 0.1.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: IN
-
offset
TimeShift offset- verbose name: Offset
- default value: None
- port type: DataPort
- value type: float (can be None)
- data direction: OUT
-
scale
Timeshift scale- verbose name: Scale
- default value: None
- port type: DataPort
- value type: float (can be None)
- data direction: OUT
-
timestamp_pairs
Pairs of timestamps used to calculate piecewise interpolation. Used by ShiftTimestamps node. Returns a nested array of floats: [[other, this], [other, this], ...].- verbose name: Timestamp Pairs
- default value: None
- port type: ListPort
- value type: list (can be None)
-
other_source
The source of timestamps from the 'other' clock that will be compared to the timestamps in the time axis. Set this to 'data' to use the values in the data tensor (chunk.block.data). Any other string will be assumed to be the name of an instance axis field containing the timestamps.- verbose name: Other Source
- default value: data
- port type: StringPort
- value type: str (can be None)
-
allow_scaling
If the other timestamp source was in different units (i.e . not in seconds) then set this to True to allow the scale-factor to be != 1.- verbose name: Allow Scaling
- default value: False
- port type: BoolPort
- value type: bool (can be None)
-
return_inverse
When False (default), this node returns the offset (and scale factor) to transform a time vector from the clock used for the timestamps defined inother_source
to the clock used for the time axis timestamps. If set to True, then the offset (and scale factor) will transform the time vector from the time axis clock, into theother_source
clock.- verbose name: Return Inverse
- default value: False
- port type: BoolPort
- value type: bool (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
HoldLastPacket
Holds and passes on the most recently received packet.
Use this node to continuously repeat the output of the preceding node, which would otherwise only be sent through the pipeline only once (for example, the output of an import node, or of a node that can be configured to generate and transmit a single buffer, such as AccumulateCalibrationData
). Only non-empty streams are passed through.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
HoldLastValue
Holds the most recently received (non-empty/zero/None) value.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: INOUT
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
InterpolateMissingTimestamps
Interpolate data to repair gaps and jitter in the timestamps.
The timestamps of the signal data is inspected to detect missing timestamps (i.e. timestamps of gaps of at least 2 samples - 2 * nominal_rate). Spline interpolation is then used to fill in the gaps between the missing timestamps in order to create a continuous timeseries. This node is currently designed for offline usage only.
Version 0.9.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
ignore_gap
Minimum length of gap in seconds to ignore during interpolation. All gaps larger than this length will be carried over into the interpolated timeseries.- verbose name: Ignore Gap
- default value: 1
- port type: FloatPort
- value type: float (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
MoveToBackend
Move the data to the specified compute backend.
Any subsequent node that has a backend option will then by default use the same backend (i.e., the one specified here) without any conversion overhead. However, BEWARE that currently, most nodes will only work if the data is on the numpy backend (you will usually get crashes as a result). For this reason, the main use of this node is for the case where you run a chain of nodes that all have a backend option; in this case, you can first convert the data to the desired backend with this node, then run the chain of nodes, and then convert the data back to numpy before continuing with "regular" nodes. This avoids the unnecessary overhead of converting the data back and forth between numpy and your backend of choice in each of the nodes along that chain.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: INOUT
-
backend
Compute backend to use. Keep means to keep the current backend. Numpy will force the data to NeuroPype's default numpy backend. Cupy is a fast GPU backend with very high numpy compatibility. Jax has nearly as good numpy compatibility as cupy, but is often a bit slower in most operations. Torch is often the fastest, but may not have all operations available. Torch-CPU runs on the CPU, but typically makes better use of multicore than Numpy (however, either of them will use no more cores than currently configured via, e.g., environment variables and context managers). TensorFlow is another fast GPU backend, but has at the time of this writing the lowest numpy compatibility. Dask is a multi-core CPU backend that aims to handle very large datasets. The remaining options are short-hand aliases.- verbose name: Backend
- default value: numpy
- port type: ComboPort
- value type: str (can be None)
-
precision
Numeric precision to use. There is no difference between N-bit and floatN; these are just convenience aliases. Can be reduced to save memory (e.g. if running on GPU).- verbose name: Precision
- default value: keep
- port type: EnumPort
- value type: str (can be None)
-
allow_markers
Allow processing of marker chunks, if any.- verbose name: Also Process Marker Stream
- default value: False
- port type: BoolPort
- value type: bool (can be None)
-
inconvertible
What to do if the data cannot be converted to the desired backend. Preserve will keep the affected piece of data on the current backend. Blank will replace the data with None. Raise will raise an exception.- verbose name: Inconvertible
- default value: preserve
- port type: EnumPort
- value type: str (can be None)
-
recursive
Whether to recursively move all data in a nested structure. If False, only the top-level data will be moved, and if it is a list, it will be interpreted as an array.- verbose name: Recursive
- default value: False
- port type: BoolPort
- value type: bool (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
OneHotCoding
Convert the given integer data to a one-hot coding.
This will create a new array with one dimension more than the input, where the new dimension is of size index_size. The output data will have a one in the location corresponding to the value of the input data, and zeros everywhere else.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to encode.- verbose name: Data
- default value: None
- port type: DataPort
- value type: AnyNumeric (can be None)
- data direction: INOUT
-
index_size
Number of positions along the one-hot encoded axis. If not given, this defaults to the highest value in the data plus one; however, this can be brittle (if not all indices always occur), so it is recommended to set this value explicitly.- verbose name: Index Size
- default value: None
- port type: IntPort
- value type: int (can be None)
-
precision
Numeric precision to use. Keep resolves to the precision of the template array if one is provided, and otherwise to the current default (usually 64-bit). Can be reduced to save memory (e.g. if running on GPU).- verbose name: Precision
- default value: float32
- port type: EnumPort
- value type: str (can be None)
-
axis
Type of the new axis to append. This may also bake in a custom label, as in feature.mylabel.- verbose name: Axis
- default value: statistic
- port type: ComboPort
- value type: str (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
OneSampleBuffer
One sample buffer for control signals.
This is primarily meant to handle default values for multi-channel user-provided control signals that update at most once per tick (i.e., there is only one sample per tick). The initial value of the control signal is sourced from the initi_values argument, and thereafter the initial value is optionally updated based on the data of the input signal. There is a distinct 'null' value, which will not override the default values, to allow users to override the defaults only partially.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
initial_values
These values are the initial values assigned to the output. The dict use channel names as the key values.- verbose name: Initial Values
- default value: None
- port type: DictPort
- value type: dict (can be None)
-
null_value
The null value. If the input signal contains this value in any of its channels, the current value in the buffer will not be updated.- verbose name: Null Value
- default value: nan
- port type: FloatPort
- value type: float (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
OverrideDataType
Override the data precision type of the given data (i.e
., float64 to float32).
Version 0.5.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
precision
New data precision (bit depth) to use. Float32 uses half the storage space but can run into numeric precision issues with some nodes.- verbose name: Precision
- default value: float64
- port type: EnumPort
- value type: str (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
OverrideSamplingRate
Override the sampling rate in the given data's time axis.
This is useful when you know that the sampling rate is incorrect for some reason.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
sampling_rate
New sampling rate to set. If None, no change is made.- verbose name: New Sampling Rate
- default value: None
- port type: FloatPort
- value type: float (can be None)
-
change_triggers_state_reset
Changing the rate triggers a reset on subsequent nodes.- verbose name: Change Triggers State Reset
- default value: True
- port type: BoolPort
- value type: bool (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
OverrideTimestamps
Override the timestamps in the given data based on its nominal sampling rate, optionally pegged to the current "wall clock" reading.
If you peg the stamps to the clock, you will get jitters in the time stamps, because the times at which the clock is read out will not be exactly periodic -- however, you can easily dejitter the time stamps by inserting a Dejitter Timestamps node after this node. The main reason to use this node is when you have a data stream that has time stamps that are either garbage due to buggy data acquisition software, or which happen to be read off some other clock that is not synchronous with the current clock (e.g., playback of a previous recording, or data coming from another PC with no clock synchronization or drift compensation). In these cases you can achieve a moderate degree of synchronization using the wallclock (the timing will still be off by the amount of transmission latency, which can be on the order of 10s of miliseconds, but this is still better than nothing).
Version 1.1.2
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
timebase
Time base to use. The wallclock setting uses the current computer clock, specifically the same clock as used by the Lab Streaming Layer, which allows you to achieve approximate synchronization to other LSL streams within a few 10s of miliseconds. Note that you may need to dejitter the time stamps using the Dejitter Timestamps node. UTC uses UTC time relative to the current computer clock (equivalent to wallclock + the UTC offset of the current computer clock). The abstract clock has no jitter, but is not synchronized to anything. Keep retains the current timestamps.- verbose name: Time Base
- default value: wallclock
- port type: EnumPort
- value type: str (can be None)
-
include_markerstreams
Handling of marker stream: if set to 'brute', markers are simply re-timestamped upon arrival. This will be somewhat inaccurate, especially if the tick rate is low and or a tick covers a lot of time (e.g., during playback).- verbose name: Include Markerstreams
- default value: skip
- port type: EnumPort
- value type: str (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
ParseDatestamp
Parse a string into a datestamp according to some format.
This will convert a formatted date string into a datestamp object. The format can be either a standard format such as ISO 8601, RFC 3339, a manually specified format using placeholders, or a heuristic mode that tries to recognize a variety of formats. Be aware that the heuristic mode can be ambiguous and should only be used when the range of possible inputs is known in advance. For cases where the time zone is not given in the string, the set timezone is used.
Version 0.5.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
string
Input string to parse.- verbose name: String
- default value: None
- port type: DataPort
- value type: str (can be None)
- data direction: IN
-
datestamp
Datestamp.- verbose name: Datestamp
- default value: None
- port type: DataPort
- value type: datetime (can be None)
- data direction: OUT
-
format
Assumed string format for parsing. Besides the two predefined formats, you can also use a date format string using placeholders such as YYYY-MM-DD HH:MM:SS.sssZ. Strict recognizes unambiguous formats such as ISO 8601, RFC 3339, and a few other common formats. The heuristic mode is more lenient and tries to recognize a wider range of formats, but be aware that these are often ambiguous with respect to the time zone, and can even be ambiguous about the order of months, days, and so forth. Be sure to then set the assumed time zone as is appropriate for your use case.- verbose name: Format
- default value: strict
- port type: ComboPort
- value type: str (can be None)
-
timezone
Time zone if not provided in string. Time zones are formatted as in US/Pacific, Europe/London, and so forth.- verbose name: Assumed Time Zone (If Not Provided)
- default value: UTC
- port type: ComboPort
- value type: str
-
locale
Locale for parsing of month and day names. Note that additional locales may be supported by the implementation.- verbose name: Locale For Month/day Names
- default value: en
- port type: ComboPort
- value type: str (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
ParseTriggerChannel
Extract event markers from a trigger channel.
The traditional way to mark events in sensor time series is using trigger channels that can take on a number of discrete numeric values (where 0 means "no event"). However, NeuroPype treats event markers as a separate data stream, where each event has a string to identify it and a timestamp of when it happened. This node lets you convert from a trigger channel to the NeuroPype format (markers will be the string versions of the numeric values in the channel). This node does not currently remove the trigger channel itself, so you need to remove it by hand if you are doing machine learning of any kind on the data.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
channel
Index of the trigger channel, counting from 0. Negative values count from the end, e.g., -1 is the last channel. If this is None, the data is passed through unmodified.- verbose name: Trigger Channel Index
- default value: None
- port type: IntPort
- value type: int (can be None)
-
debounce_ttl
Only mark onset times. If a trigger value is held for multiple successive samples, a marker will only be emitted for the first sample. This is typically what you want, since the hold time on the trigger channel usually has to be multiple samples long to ensure that spike is not missed. Disabling this allows you to generate a marker for every non-zero sample.- verbose name: Only Mark Onset Times
- default value: True
- port type: BoolPort
- value type: bool (can be None)
-
print_markers
Print extracted markers. This is for diagnostic and debugging purposes.- verbose name: Print Markers
- default value: False
- port type: BoolPort
- value type: bool (can be None)
-
previous_markers
Handling mode for previous marker streams. In replace mode, any previous NeuroPype marker stream (e.g., from LSL) will be removed and replaced by the trigger-based marker stream. In ignore mode, a new marker stream will be added and the old one will be kept. Note that if you do this, some subsequent processing nodes that handle marker streams may be confused and will only pick up one of the streams.- verbose name: Existing Marker Stream Handling
- default value: replace
- port type: EnumPort
- value type: str (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
PipelineDone
Special purpose node used to force a pipeline to report that it is finished.
An example are streaming pipelines that also have one or more offline components (i.e., calling a node that generates a report or exports files to disk), or pipelines that have complex looping structures where NeuroPype cannot otherwise determine whether the pipeline is finished. Nodes which write to disk should have an "done" port that is wired into this node and which returns True when they are done. When all nodes wired into this node report they are finished (=True), this node will report itself as "finished" (is_finished=True). Neuropype will then set the status of the pipeline, which can be queried over the API, to completed (completed=True) even if there are other nodes which are not "finished" (i.e., LSLInput, StreamData, etc.) The inputs to this node are ideally booleans (True/False values), but can be any value that can be interpreted as a boolean (e.g., that can be wired into an If/Else node). For some data types (e.g., Packets) this would be ambiguous, so you will get an error if you wire such data into the node without first reducing it to a boolean value. The number of inputs that must evaluate to True can be specified in the required_inputs property. You can also have data passed unmodified through this node that you may wire into a Result node to have the Pipeline return a value when it is is finished.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data1
Input Condition 1.- verbose name: Data1
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
-
data2
Input Condition 2.- verbose name: Data2
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
-
data3
Input Condition 3.- verbose name: Data3
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
-
data4
Input Condition 4.- verbose name: Data4
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
-
data5
Input Condition 5.- verbose name: Data5
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
-
dataN
Additional conditions.- verbose name: Datan
- default value: None
- port type: DataPort
- value type: list (can be None)
- data direction: IN
-
passthrough
Pass through data. Can be wired into a final Result node.- verbose name: Passthrough
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: INOUT
-
required_inputs
Number of inputs which must evaluate to True for pipeline to complete. If 0, the pipeline will unconditionally complete when the node is reached (regardless of the value). If set to the special value (use default), then each wired-in input must evaluate to True.- verbose name: Required Inputs
- default value: 0
- port type: IntPort
- value type: int (can be None)
-
verbose
Print verbose output. This will log when the node was triggered.- verbose name: Verbose
- default value: True
- port type: BoolPort
- value type: bool (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
PipelineFilepath
Returns the filepath of the pipeline containing the node (if any), or an empty string if there is no associated file (e.g
. unsaved pipeline).
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
result
File path- verbose name: Result
- default value: None
- port type: DataPort
- value type: str (can be None)
- data direction: OUT
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
PropagateNaNs
Propagate NaNs within a data packet following a specified rule.
This can be used, for example, to propagate bad-sample masks between different NIRS wavelengths, or to carry over a NaN mask from one stream to another.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
other
Secondary stream.- verbose name: Other
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: IN
-
rule
Propagation rule. Thesame_position
mode will propagate to channels that have the same position, e.g., wavelengths in NIRS, including dark measurements if present. Thefrom_other
mode will propagate NaNs from the other packet to the data packet.- verbose name: Rule
- default value: from_other
- port type: EnumPort
- value type: str (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
RegulateChunkSizes
Fix up chunk size irregularities in streaming data using a buffer.
This node maintains a running estimate of the average number of samples that it receives on each update (e.g., 135 samples), and adjusts the amount of data that it outputs on each update to be this quantity of samples. The node maintains a buffer of configurable length to be able to absorb "hitches" in the input pipeline. As a result, the node outputs a very regular stream of samples even if the input processing chain has very irregular latency characteristics. One use of this node is to drive smooth-scrolling plots and other visualizations that shall be smoothly updating.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
buffer_length
Buffer length. This is the length of the buffer to absorb hitches, in seconds.- verbose name: Buffer Length (Seconds)
- default value: 1
- port type: FloatPort
- value type: float (can be None)
-
reset_on_gap
Reset the state if a gap is encountered that is longer than this. In seconds.- verbose name: Reset On Gap
- default value: 5
- port type: FloatPort
- value type: float (can be None)
-
also_reset_buffer
Also reset buffer on gap.- verbose name: Also Reset Buffer
- default value: False
- port type: BoolPort
- value type: bool (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
RepeatAlongAxis
Repeat the data along the given axis.
This node basically repeats n copies of the data along the desired axis. One of the most common use cases is to extend an array to match the size of another array to then perform an elementwise operation between the two, such as addition.
Version 1.1.2
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: AnyNumeric (can be None)
- data direction: INOUT
-
axis
Axis along which to repeat. Select the space axis for channels, instance axis for trials or segments, time for the number of time points, feature for the number of features in the stream, etc.- verbose name: Axis
- default value: instance
- port type: ComboPort
- value type: str (can be None)
-
create_axis
Create the axis if it doesn't exist in the data.- verbose name: Create Axis If Not Present
- default value: True
- port type: BoolPort
- value type: bool (can be None)
-
num_repeats
Number of repetitions. The data will be repeated this many times along the given axis. The repeat operation is a multiplication of the data along the axis, so if the number of repeats is 1, the data will be passed through unchanged.- verbose name: Num Repeats
- default value: 1
- port type: IntPort
- value type: int (can be None)
-
verbose
Print warnings for edge cases.- verbose name: Verbose
- default value: True
- port type: BoolPort
- value type: bool (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
ReplaceAxisValues
Replace the values in an array of an axis (i.e
., channel names in a space axis) using a regular expression search and replace. For example, to rename channels in the space axis with the format EEG Xx-LE
to Xx
, use (EEG )(.*[^-LE])(.*)
as the search expression and (\2)
as the replacement expression. (For a regular expression cheatsheet and to easily test regular expressions, see https://pythex.org.).
Version 1.1.4
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
streams
A list of stream names, for the streams containing axis array elements to replace. Leave empty to replace values in all streams. Note: This parameter may be deprecated in the future. To operate on only a subset of streams, please use theExtractStreams
node first.- verbose name: Streams To Act On
- default value: []
- port type: ListPort
- value type: list (can be None)
-
axis
Axis to modify.- verbose name: Axis
- default value: space
- port type: ComboPort
- value type: str (can be None)
-
axis_occurrence
Index of the occurrence of the axis of desired type. (0
is the first axis of the desired type,1
is the second,-1
is the last,-2
is the second-to-last, and so on).- verbose name: Axis Occurrence
- default value: 0
- port type: IntPort
- value type: int (can be None)
-
array_key
The name of the axis array containing the elements to search and replace. For example, on the Space or Feature axis this is typically thenames
array, which holds the channel names on the Space axis and feature names in a Feature axis.- verbose name: Axis Array To Modify
- default value: names
- port type: StringPort
- value type: str (can be None)
-
search_expr
A regular expression to search for in the axis array elements. Example:(EEG )(.*[^-LE])(.*)
.- verbose name: Search Expression
- default value:
- port type: StringPort
- value type: str (can be None)
-
replace_expr
A regular expression of the element to replace with. To specify groups, use\n
where n is the group number. Example:\2
with the above search expression would replace each element with the second matched group.- verbose name: Replacement Expression
- default value:
- port type: StringPort
- value type: str (can be None)
-
verbose
Output a log entry with the results of the change.- verbose name: Verbose
- default value: False
- port type: BoolPort
- value type: bool (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
RetainChannelMetadata
Store and recover space axis (channel) meta-data upon save/load of a model.
This node is occasionally useful in conjunction with pipelines for which models are being saved and then later loaded and used, such as certain real-time processing contexts where the pipeline is calibrated on some recording, a model is stored, and then later that model is used in a real-time context. In such a context, this node can ensure that whatever per-channel meta-data was available offline (e.g., loaded from an XDF file or generated via curation), will be retrieved and then associated with streaming data (e.g., coming from LSL). Note that, since this node supports multiple streams, the stream name(s) upon use of the model must match the stream names(s) that were used during calibration.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
ignore_resets
Ignore signal resets. If enabled, the meta-data will persist across resets of the data (e.g., loading of a new file) and will only be cleared if the channel count changes.- verbose name: Ignore Resets
- default value: False
- port type: BoolPort
- value type: bool (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
SelectActiveInput
Pick the value from the active input port.
This node accepts multiple inputs of any type and one output. The node assumes that at most one of the inputs is not None, and it will then output that value, or None if all inputs are None. This node is typically used to rejoin multiple conditionally forking paths, for instance different processing chains that are selectively enabled by a ConditionSwitch node.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
input1
Input alternative 1.- verbose name: Input1
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
-
input2
Input alternative 2.- verbose name: Input2
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
-
input3
Input alternative 3.- verbose name: Input3
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
-
input4
Input alternative 4.- verbose name: Input4
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
-
input5
Input alternative 5.- verbose name: Input5
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
-
output
The value of the one input that is active.- verbose name: Output
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: OUT
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
SelectInstanceFields
Select (retain) or drop fields from an instance axis.
This can be used to drop extraneous variables from instance axes that may otherwise be included automatically by some nodes (i.e., plot nodes). Note that this will act on all instance axes found in all streams (except for marker streams if include_markers
is False). Note that dropping (or adding) fields to the instance axis does not change the number of elements (items) along the instance axis, (rather these fields act as attributes of each element along the instance axis), and therefore does not change the shape of the data tensor in a stream.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
retain_fields
Fields in the instance axis to select; all other fields will be dropped. An empty list will drop all fields. Note however that the 'Marker', 'TargetValue', and 'IsGood' fields cannot be dropped so in that case their values will be set to null instead.- verbose name: Fields To Select
- default value: ['Marker', 'TargetValue', 'IsGood']
- port type: ListPort
- value type: list (can be None)
-
inverse_selection
Inverse the selection. If True, the fields listed inretain_fields
will be dropped and all others present in the instance axis will be retained.- verbose name: Inverse Selection
- default value: False
- port type: BoolPort
- value type: bool (can be None)
-
include_markers
Also select (or drop) fields from marker streams. If false, only signal streams are processed.- verbose name: Include Markers
- default value: False
- port type: BoolPort
- value type: bool (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
SetAxisLabel
Set a custom label on a given axis.
The existing custom label will be replaced, if it exists. In NeuroPype, each axis has a type (for example, space, time, frequency). Additionally, any axis can be given a "label". Axis labels are most useful to distinguish between multiple axes of same type in a data packet, for example estimates of some non-symmetric quantity between all pairs of channels (e.g., information flow). Such data may have two space axes, and without a label it would not be possible to tell which of them is which (e.g., "from" and "to" channels). Neuropype does not assume or enforce a certain fixed order for axes, and some nodes may reorder axes in non-obvious ways. For this reason, axes are uniquely identifiable by their type, and if needed by their label. Note that this node will set or override the label of the first axis of the desired type in the data unless the 'axis_occurrence' property is set.
Version 1.1.1
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
new_label
New axis label (custom_label) to set.- verbose name: New Label
- default value: mylabel
- port type: StringPort
- value type: str (can be None)
-
axis_type
The axis to modify. You can additionally specify an axis label, using the format "space.mylabel", to select a specific axis.- verbose name: Axis Type
- default value: space
- port type: ComboPort
- value type: str (can be None)
-
axis_occurrence
Index of the occurrence of the axis of desired type. (0
is the first axis of the desired type,1
is the second,-1
is the last,-2
is the second-to-last, and so on). If not set, the first axis of the desired type will be used.- verbose name: Axis Occurrence
- default value: 0
- port type: IntPort
- value type: int (can be None)
-
only_signals
If set to true, non-signal streams (i.e ., marker streams) will be ignored.- verbose name: Only Signals
- default value: True
- port type: BoolPort
- value type: bool (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
SetStreamProperties
Set (override or add) meta-data properties of a given stream.
In NeuroPype, each data packet has one or more streams (internally of type Chunk) carrying data such as EEG or event markers. Each such stream (chunk) has a dictionary of meta-data properties, holding information such as the modality of the data (EEG, NIRS, etc.), the origin of the data (file path that the data was originally loaded from), the name of the original LSL stream, the delay incurred by the applied filters, and so on. This node will override any specified properties in the stream which already exist (properties not specified are not deleted). Any specified properties which do not exist in the stream will be set (added). Multi-level properties can be specified using a period (.) as a delimiter: desc.cap.size
will create a nested property that is <desc><cap><size>
. You may specify a stream name to only override properties on that stream, or else all streams. To operate on select multiple streams (filtered by property, for example), use ExtractStreams before this node.
Version 1.5.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
val1
Property value 1.- verbose name: Val1
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
-
val2
Property value 2.- verbose name: Val2
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
-
val3
Property value 3.- verbose name: Val3
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
-
val4
Property value 4.- verbose name: Val4
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
-
val5
Property value 5.- verbose name: Val5
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
-
valN
Additional property values.- verbose name: Valn
- default value: None
- port type: DataPort
- value type: list (can be None)
- data direction: IN
-
stream
Name of stream on which to set or override properties. Leave empty to override properties on all streams. Wildcards are supported (i.e.,mystream*
will match the first stream that begins with mystream.- verbose name: Stream To Override
- default value: None
- port type: StringPort
- value type: str (can be None)
-
stream_name
- verbose name: Stream Name
- default value: None
- port type: AliasPort
- value type: str (can be None)
-
properties
Dictionary of properties to assign (where each key is the name of the property and the value is the value to assign to that property), i.e . {'modality': 'nirs'}. Use a dot (.) to indicate nested property names, i.e.: {'metadata.cap.size': '56'}. Note that, if a property of same name is provided both in this dictionary and passed via the property name/value ports,the latter take precedence and will override the value.- verbose name: Properties
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
name1
Property name 1.- verbose name: Property Name 1
- default value: None
- port type: StringPort
- value type: str (can be None)
-
name2
Property name 2.- verbose name: Property Name 2
- default value: None
- port type: StringPort
- value type: str (can be None)
-
name3
Property name 3.- verbose name: Property Name 3
- default value: None
- port type: StringPort
- value type: str (can be None)
-
name4
Property name 4.- verbose name: Property Name 4
- default value: None
- port type: StringPort
- value type: str (can be None)
-
name5
Property name 5.- verbose name: Property Name 5
- default value: None
- port type: StringPort
- value type: str (can be None)
-
nameN
Additional property names in list form.- verbose name: Additional Property Names
- default value: None
- port type: ListPort
- value type: list (can be None)
-
only_signals
If set to True, non-signal streams (i.e ., marker streams) will be ignored.- verbose name: Only Signals
- default value: False
- port type: BoolPort
- value type: bool (can be None)
-
allow_markers
If set to False, marker streams will be ignored.- verbose name: Allow Markers
- default value: True
- port type: BoolPort
- value type: bool (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
ShiftDatestamp
Shift a datestamp by some offset.
This allows the datestamp to be offset by a specified number of years, months, days, and so forth. One may also shift to a different time zone (without changing the moment in time).
Version 0.5.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
datestamp
Datestamp.- verbose name: Datestamp
- default value: None
- port type: DataPort
- value type: AnyNumeric (can be None)
- data direction: INOUT
-
delta
Time.- verbose name: Delta
- default value: None
- port type: DataPort
- value type: AnyNumeric (can be None)
- data direction: INOUT
-
years
Year offset.- verbose name: Years
- default value: None
- port type: IntPort
- value type: int (can be None)
-
months
Month offset.- verbose name: Months
- default value: None
- port type: IntPort
- value type: int (can be None)
-
weeks
Week offset.- verbose name: Weeks
- default value: None
- port type: IntPort
- value type: int (can be None)
-
days
Day offset.- verbose name: Days
- default value: None
- port type: IntPort
- value type: int (can be None)
-
hours
Hour offset.- verbose name: Hours
- default value: None
- port type: IntPort
- value type: int (can be None)
-
minutes
Minute offset.- verbose name: Minutes
- default value: None
- port type: IntPort
- value type: int (can be None)
-
seconds
Second offset.- verbose name: Seconds
- default value: None
- port type: FloatPort
- value type: float (can be None)
-
microseconds
Microsecond offset.- verbose name: Microseconds
- default value: None
- port type: IntPort
- value type: int (can be None)
-
timezone
Optionally the desired time zone into which to translate the datestamp. This will translate the same moment in time into the other time zone.- verbose name: In Time Zone
- default value: keep
- port type: ComboPort
- value type: str
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
ShiftTimestamps
Shift the timestamps in the given signal by some delta.
This is mostly useful to compensate for shifts in the signal introduced by the data acquisition process, or shifts that were introduced by some preceding filters. In the latter case, the shift is known and can be accounted for by enabling the compensate for filter lags
property. The timestamp_pairs
port can receive pairs of timestamps (arranged as [[original, new], ...]
), for piecewise interpolation of the data packet's timestamps using the timestamp pairs. This node can also be used to reset the timestamps so that the first timestamp in the signal is 0 (by enabling the beginning_from_zero
property), which is useful if you wish to downstream trim or select a time range relative to the start of the file.
Version 1.2.2
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
offset
Offset to add to time stamps. In seconds. This parameter is mostly useful if your data acquisition process introduces some known consistent shift.- verbose name: Time Offset
- default value: 0
- port type: FloatPort
- value type: float (can be None)
-
use_time_axis
In case of having both instance and time axes, force timeshift using the time axis instead of instance axis. This is useful in the case of having an instance axis with in which time is not being used. For example, using an instance axis to store session/subject meta data in grouping fields.- verbose name: Shift Time Axis Not Instance
- default value: False
- port type: BoolPort
- value type: bool (can be None)
-
timestamp_pairs
Pairs of timestamps for piecewise shifting as a nested array of floats: [[original, new], [original, new], .. .].- verbose name: Timestamp Pairs
- default value: None
- port type: ListPort
- value type: list (can be None)
-
compensate_filter_lag
Compensate for filter delay. If enabled, this node will in addition compensate for the accumulated logical (signal) delay of the preceding filters. Note that this is unrelated to how long these filters actually take to compute -- it only measures by how much they shift the data content relative to its time axis.- verbose name: Compensate For Filter Lags
- default value: False
- port type: BoolPort
- value type: bool (can be None)
-
include_markers
Also shift time stamps of markers. If this is enabled, then this will not apply a relative shift between signal and markers, but apply to all streams in the data in the same way. This should be set to True if you are resetting the timestamps to 0 (withbeginning_from_zero
set to True), so that the markers do not change their relative position to the signal.- verbose name: Shift Whole Packet
- default value: False
- port type: BoolPort
- value type: bool (can be None)
-
marker_fields
List of marker (instance axis data) fields that should also be shifted. e.g. events may have a 'stop' column to indicate the stop time of each event. If the event time gets shifted then the stop time should get shifted too.- verbose name: Shift Marker Fields
- default value: None
- port type: ListPort
- value type: list (can be None)
-
beginning_from_zero
Shift the timestamps such that the first stamp in the signal is 0. Other streams (e.g., markers) will be shifted consistently with it. Note that for streaming data, this will only reset the starting timestamp to 0 when the pipeline is restarted, not if there is some other change in the data at runtime that triggers a signal_changed event.- verbose name: Beginning From Zero
- default value: False
- port type: BoolPort
- value type: bool (can be None)
-
offset_triggers_state_reset
Changing the offset triggers a reset on subsequent nodes.- verbose name: Offset Triggers State Reset
- default value: True
- port type: BoolPort
- value type: bool (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
StripMetadata
Strip all except for a desired subset of meta-data fields from the packet.
This can be used to deidentify data before recording to a file or database.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Data to process.- verbose name: Data
- default value: None
- port type: DataPort
- value type: Packet (can be None)
- data direction: INOUT
-
retain_properties
Meta-data properties to retain. All other chunk meta-data will be dropped. An empty list will drop all meta-data. If this is set to None, no meta-data is stripped.- verbose name: Retain Properties
- default value: ['source_id', 'source_url', 'source_paths', 'stream_name', 'subject_id', 'session_id']
- port type: ListPort
- value type: list (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
TaskNumber
Retrieve the number of this task if run as part of a parallel collection of tasks.
This is 0-based and is -1 if not running as part of a parallel run. Note that even if a parallel run is configured to run in serial, this will still return the corresponding 0-based task number.
Version 1.0.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
data
Current task number.- verbose name: Data
- default value: None
- port type: DataPort
- value type: int (can be None)
- data direction: OUT
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
WithBackend
Run the given computational graph with a specified compute backend and precision.
This will override the default backend used by any node that has its backend and/or precision setting set to "keep" and which has, no template array input whose backend/precision to keep. As a result, this node primarily affects the source nodes of a given computation (e.g., RandomNormal) whose results trickle down through the rest of the graph. You can also specify the device (usually the GPU index if you have multiple GPUs) to run the computation on (if the selected backend supports multiple devices). By default this node will run the entire graph that is wired into it under the given backend context. However, you can also tag a subgraph that shall run under the given context by placing a dummy Placeholder node in your graph, giving it a unique slotname (one would usually name it "with" as a reminder that it pairs with a downstream "With..." node), and then and selectively wiring that placeholder's update output port to select nodes of your graph. Then only the nodes downstream of that placeholder will run under the given context. You will also need to set the "Graph [tag]" option (graph__signature in code) to name the tag that you used (e.g. "(with)" in this example). Note that, when you use a node that uses multiple processes (like the Parallel Foreach loop via its multiprocess_backend option), then this context will not automatically apply the nodes running under the control of such a node (although you can nest a WithBackend node inside such a loop to apply the context there as needed).
Version 0.9.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
graph
Computation to run.- verbose name: Graph
- default value: None
- port type: GraphPort
- value type: Graph
-
graph__signature
Optionally the slotname of a Placeholder node that shall mark the starting point of the graph that runs under the with context. As with all "With" context manager nodes in NeuroPype, the graph that runs under the context managed by this node is preceding the context manager node in the graph, rather than following it. This is accomplished by wiring the final node of the graph that shall execute under the governance of the context manager into the "graph" input port of the subsequent "With" node. In graphical UIs, this edge will be drawn in dotted style to indicate that this is not normal forward data flow where first the preceding node executes and then the subsequent node receives its output, but that instead the subsequent node (the context manager) receives a portion of the preceding graph, which it then runs in such a way that the context applies. You can use a "tilde" binding, spelled as (~), which affects the same nodes that a then or else branch in an IfElse node would control, which is normally what one would expect. However, note that this can have sometimes the unexpected consequence that simply wiring a value into your graph may "infect" an upstream portion of the graph with unintentionally running under the context, too. As an alternative, you can instead rely on a named Placeholder node to mark the beginning of the managed graph. The typical way to do this is to list a single placeholder name, e.g., (with), and then to create a Placeholder in your graph whose slotname is set to "with". Then, only the nodes that are downstream of (i.e., formally depending on) that placeholder will run under the context defined by the "With" node. In this case you need to make everything that shall be part of the context dependent on the placeholder, which can be accomplished by wiring the update output port of the Placeholder into the update input port of any nodes that shall run under the context (this dependency propagates down the graph, so you don't need to wire it to every node, just to the first node(s) that you want to be part of the context). Any nodes in this graph that have a backend and/or precision argument set to "keep" will be overridden by the backend and/or precision setting of this node.- verbose name: Graph [Tag]
- default value: (~)
- port type: Port
- value type: object (can be None)
-
result
Result of the computation.- verbose name: Result
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: OUT
-
backend
Optional compute backend to use. Keep means to not override, which causes operations to resolve the backend of any input or template array if one is provided and otherwise numpy (unless overridden by an outer WithBackend). Numpy is the standard CPU backend that underpins most of NeuroPype's operations. The others require one or more GPUs to be present on the system, except for torch-cpu. For best performance, keep all arrays that interact with each other (via processing nodes) on the same backend.- verbose name: Backend
- default value: keep
- port type: EnumPort
- value type: str (can be None)
-
precision
Numeric precision to use. Keep means to not override, which causes nodes to default to the precision of the input/template array if one is provided, and otherwise to whatever is the default for the specific backend, which is usually 64-bit, unless there is an outer WithBackend that overrides it. Can be reduced to save memory and improve throughput (e.g. if running on GPU).- verbose name: Precision
- default value: keep
- port type: EnumPort
- value type: str (can be None)
-
compile_loops
Compile any loops that have a compile option set to 'auto'. Note that this requires the loop bodies to be very clean (typically mainly math operations, optionally with data reformatting). See the respective loop node(s) for more details on applicable caveats.- verbose name: Compile Loops
- default value: False
- port type: BoolPort
- value type: bool (can be None)
-
device
Device index to use. This is only relevant for backends that support multiple devices. If None, no change to the current default is made.- verbose name: Device
- default value: None
- port type: IntPort
- value type: int (can be None)
-
is_global
If True, the context implicitly applies to the entire graph (except for those parts that are configuring the context itself). In this case the node must not have anything wired into its graph input and must not have any successors.- verbose name: Is Global
- default value: False
- port type: BoolPort
- value type: bool (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
WithCache
Evaluate a computational graph with caching of the results.
This will attempt to reuse results from a disk cache if the computation is being re-run with the same explicit (and optionally implicit) arguments. This will also automatically invalidate the cache if the graph or any of its nodes have changed, if any of the listed dependencies have changed, or if the version is incremented. The cache is by default stored in a directory, named after the topic, under the cache_root directory of NeuroPype. This defaults to ~/.cache/neuropype, but can be overridden either by setting the environment variable NP_CACHE_ROOT, or by overriding the cache_root variable in a script that uses NeuroPype. The cacheable computation is expected to have a single input placeholder, defaulting to 'data', which receives the main input data to the computation. This argument is passed into WithCache as the data port. However, the graph may have additional Placeholders or ParameterPorts, that may be bound downstream of the WithCache node or which may be free parameters of the graph, and these will be accounted for in the cache key, UNLESS they do not occur directly in the graph but in other graphs that are indirectly invoked by function calls (via the Call node). In such cases, the user must explicitly list the names of these arguments in the implicit_args port, otherwise results may be incorrect. Note that any placeholders in the graph whose value is externally determined at the time the graph executes will automatically factor in the cache key, because their values will be implicitly captured as constants into the graph (lambda capture). It is possible to prevent this from happening, which may be useful when these parameters are irrelevant to the result (e.g., verbosity, num_procs settings etc). This can be done by making the placeholders part of the graph, by making them formally depedent on the data placeholder, by drawing an 'update' edge from the data placeholder to the parameter/placeholder node in question. Note that this node will not delete the cache; it is the user's responsibility to delete the cache when it becomes stale or excessively large. This can be done by deleting the respective topic directory or the whole cache_root directory if desired.
Version 0.9.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
graph
Cached computation.- verbose name: Graph
- default value: None
- port type: GraphPort
- value type: Graph
-
graph__signature
Optionally the name of an argument accepted by the cached computation. If this is not specified, the cached computation is everything that runs under the control of the WithCache node (this can be viewed by Ctrl+clicking on the control (dotted) edge going into this node in a visual editor). If a name is given, then the graph wired into this node must then start with a single Placeholder node whose slotname must match the name listed here (e.g., data), and anything downstream of that placeholder is part of the cacheable computation and may be skipped if the result is already in the cache. In either case, the final node of that graph, which produces the cacheable result, is then connected to the WithCache node by connecting it to the "graph" input port of the WithCache node). As such, the WithCache node follows the cached computation graph, rather than preceding it. In graphical UIs, this edge is drawn in dotted style to indicate that this is not normal forward data flow but that the cached computation graph runs under the control of the WithCache node, which may or may not execute it (depending on whether the result is already in the cache). When your graph executes and a name was specified, the placeholder receives whichever value is passed into the "data" input of the WithCache node, which can, for example, be a filename to load and preprocess or a packet that shall be further processed, and the graph's final node outputs the result that is then written to the cache, and returned as the output of the WithCache node itself. Note that your cached computation is allowed to depend on other external inputs, for example, the content of some imported files or the value of Parameter Ports, that are wired into the graph "from the side", meaning that the nodes producing these potentially variable outputs may not themselves part of the graph that is being cached. The criterion here is that these nodes cannot be downstream of the placeholder that marks the beginning of the cacheable graph. The principle that enables this is that the cache key is computed by hashing the graph (including any of its structure and settings), plus the values of anything that flows into it from the side and the values of any contained placeholders and parameter ports, as well as the primary cache key (if any, i.e., the data argument), and only if all these details match a previously cached result, the cache will be used.- verbose name: Graph [Signature]
- default value: (~)
- port type: Port
- value type: object (can be None)
-
data
Input data to the computation. Can only be provided if a placeholder name is specified.- verbose name: Data
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: IN
-
result
Result of the cached computation.- verbose name: Result
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: OUT
-
enabled
Enable caching. This can be unchecked to bypass the caching mechanism, e.g., when debugging.- verbose name: Cache Enabled
- default value: True
- port type: BoolPort
- value type: bool (can be None)
-
topic
Topic to use for caching. Data is stored in a directory under this topic; this helps when the data for this topic becomes stale and the user wants to delete it.- verbose name: Cache Topic
- default value: default
- port type: StringPort
- value type: str (can be None)
-
dependencies
Optional list of Python code files that the computation depends on. This is used to invalidate the cache when these files change. For beginners or for computations that depend on custom nodes that you are actively editing, [''] is a good choice, which will include the code for all nodes that are part of the graph being cached. However, when you are mainly caching the output of mature nodes that are unlikely to receive bugfixes that affect your outputs, you can also set this to the empty list [] in order to avoid unnecessary cache invalidations resulting from such minor updates, for example when you are updating to a newer NeuroPype release. A good rule of thumb is to use [] for preprocessing and [''] for more advanced analysis that may use node marked beta or that you are actively developing. This list can also include fully specified paths to additional files, for example from utility code that your nodes are using, like ['~/my_code/my_utility_function.py', '']. The default value resolves to [''], but incurs a reminder to make an explicit choice.- verbose name: Graph Dependencies
- default value: ['*']
- port type: ListPort
- value type: list (can be None)
-
ver
Optional version of the graph. This can be used to invalidate the cache when the computation implemented by the graph or one of its nodes changes. The WithCache node will automatically snapshot the graph, its parameters, and the source code of the nodes it employs and make sure that the cache key factors in these details; however, if a node calls some external code (e.g., a library or utility function), then the user needs to make sure that the version is updated when the external code changes. See also dependencies for a tool to protect against this.- verbose name: Cache Version
- default value: 1.0.0
- port type: StringPort
- value type: str (can be None)
-
implicit_args
Optional list of implicit arguments to the computation. These are arguments that are not explicitly passed to the computation but are used by the computation. For example, if the computation calls a node which contains a ParameterPort whose value dynamically resolves to a bound value in an outer call context, then that value is an implicit argument, and the name of the ParameterPort should be listed here.- verbose name: Additional Implicit Arguments
- default value: []
- port type: ListPort
- value type: list (can be None)
-
extra_context
Optional additional context to include in the cache key. This is useful when the computation depends on some external data or code that is not captured by the implicit_args listing, such as the current date or the return value of some system call, if used in the computation.- verbose name: Additional Cache Context
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
capture_functions
If enabled, the structure of function graphs that are being called (using the Call node) are included in the cache key. This is important when portions of the cached computation are factored into functions, and those functions may be changing or are being updated.- verbose name: Capture Functions
- default value: True
- port type: BoolPort
- value type: bool (can be None)
-
verbose
Verbosity level of diagnostic outputs.- verbose name: Verbosity
- default value: 1
- port type: EnumPort
- value type: str (can be None)
-
no_graph_tracking
If enabled, structural changes to the graph whose output is being cached are not tracked, and will not invalidate the cache. Instead, it is your responsibility to update the version field when you make substantive changes that justify recomputation and recaching. Note that dependencies will still be respected unless you clear that field. WARNING: it is very easy to forget that this field is enabled when you are making changes to the cached graph and expect the computation to be re-run.- verbose name: No Graph Tracking
- default value: False
- port type: BoolPort
- value type: bool (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
WithCoreLimit
Run the given computational graph with a limit on the number of cores that can be used by the process.
This can be used to prevent churn due to math backends that would otherwise use excessively many threads / cores, and can also help control the total number of cores used when multiple processes are involved, by allotting each process a subset of the total number of cores. The graph has a dummy input named "with", which is used to tag the start of the graph that is running under control of this node, which is all nodes downstream of the Placeholder that accepts the "with" input, up to but excluding the WithCoreLimit node itself. However, note that this only applies to a certain set of math libraries, including BLAS, OpenMP, and libraries using them. Also note that, if your computation involves multi-process parallelism (such as when using a Parallel ForEach loop with any multiprocess backend except for 'serial' or the rarely-used 'threading' backend), then this setting has no effect on what happens in those sub-processes, which would need their own WithCoreLimit node to limit their use of cores.
Version 0.9.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
graph
Computation to run.- verbose name: Graph
- default value: None
- port type: GraphPort
- value type: Graph
-
graph__signature
Optionally the slotname of a Placeholder node that shall mark the starting point of the graph that runs under the with context. As with all "With" context manager nodes in NeuroPype, the graph that runs under the context managed by this node is preceding the context manager node in the graph, rather than following it. This is accomplished by wiring the final node of the graph that shall execute under the governance of the context manager into the "graph" input port of the subsequent "With" node. In graphical UIs, this edge will be drawn in dotted style to indicate that this is not normal forward data flow where first the preceding node executes and then the subsequent node receives its output, but that instead the subsequent node (the context manager) receives a portion of the preceding graph, which it then runs in such a way that the context applies. You can use a "tilde" binding, spelled as (~), which affects the same nodes that a then or else branch in an IfElse node would control, which is normally what one would expect. However, note that this can have sometimes the unexpected consequence that simply wiring a value into your graph may "infect" an upstream portion of the graph with unintentionally running under the context, too. As an alternative, you can instead rely on a named Placeholder node to mark the beginning of the managed graph. The typical way to do this is to list a single placeholder name, e.g., (with), and then to create a Placeholder in your graph whose slotname is set to "with". Then, only the nodes that are downstream of (i.e., formally depending on) that placeholder will run under the context defined by the "With" node. In this case you need to make everything that shall be part of the context dependent on the placeholder, which can be accomplished by wiring the update output port of the Placeholder into the update input port of any nodes that shall run under the context (this dependency propagates down the graph, so you don't need to wire it to every node, just to the first node(s) that you want to be part of the context). Any computation in this graph will then use at most this many cores, but note that this does not apply to any sub-processes that may be spawned by the computation (e.g., when using a Parallel ForEach loop with a multiprocess backend). The WithCoreLimit node can also be nested, such that the managed graph itself may contain other WithCoreLimit nodes and their associated "with" Placeholders (these nested placeholders are distinguished by being wired (indirectly) into the "graph" port of the nested WithCoreLimit node, while the outer "with" Placeholder should only be wired into the update port of the nested WithCoreLimit, but you can also rename the tag name to something other than 'with' to avoid ambiguity).- verbose name: Graph [Tag]
- default value: (with)
- port type: Port
- value type: object (can be None)
-
result
Result of the computation.- verbose name: Result
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: OUT
-
num_cores
Maximum number of cores available to the computation. This can also override a limit imposed by an outer caller context. If set to -1, the computation will be able to use as many cores as there are on the system. If set to a positive integer, then the computation will be limited to use this many threads / cores. If left unspecified (None/use default), then this node does nothing.- verbose name: Number Of Cores
- default value: -1
- port type: IntPort
- value type: int (can be None)
-
is_global
If True, the context implicitly applies to the entire graph (except for those parts that are configuring the context itself). In this case the node must not have anything wired into its graph input and must not have any successors.- verbose name: Is Global
- default value: False
- port type: BoolPort
- value type: bool (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)
WithOptions
Run the wired-in computational graph with the specified options applied to it.
This node can be used to distribute a single value to a large number of nodes in a graph, which can either be one or more instances of Placeholder nodes of the same name in the graph, or any ports of a specified name on nodes of a certain class. See the options property for more details. By default this node will run the entire graph that is wired into it under the given backend context. However, you can also tag a subgraph that shall run under the given context by placing a dummy Placeholder node in your graph, giving it a unique slotname (one would usually name it "with" as a reminder that it pairs with a downstream "With..." node), and then and selectively wiring that placeholder's update output port to select nodes of your graph. Then only the nodes downstream of that placeholder will run under the given context. You will also need to set the "Graph [tag]" option (graph__signature in code) to name the tag that you used (e.g. "(with)" in this example). The full set of options can be accessed within the graph by the name __options.
Version 0.5.0
Ports/Properties
-
metadata
User-definable meta-data associated with the node. Usually reserved for technical purposes.- verbose name: Metadata
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
graph
Computation to run.- verbose name: Graph
- default value: None
- port type: GraphPort
- value type: Graph
-
graph__signature
Optionally the slotname of a Placeholder node that shall mark the starting point of the graph that runs under the with context. As with all "With" context manager nodes in NeuroPype, the graph that runs under the context managed by this node is preceding the context manager node in the graph, rather than following it. This is accomplished by wiring the final node of the graph that shall execute under the governance of the context manager into the "graph" input port of the subsequent "With" node. In graphical UIs, this edge will be drawn in dotted style to indicate that this is not normal forward data flow where first the preceding node executes and then the subsequent node receives its output, but that instead the subsequent node (the context manager) receives a portion of the preceding graph, which it then runs in such a way that the context applies. You can use a "tilde" binding, spelled as (~), which affects the same nodes that a then or else branch in an IfElse node would control, which is normally what one would expect. However, note that this can have sometimes the unexpected consequence that simply wiring a value into your graph may "infect" an upstream portion of the graph with unintentionally running under the context, too. As an alternative, you can instead rely on a named Placeholder node to mark the beginning of the managed graph. The typical way to do this is to list a single placeholder name, e.g., (with), and then to create a Placeholder in your graph whose slotname is set to "with". Then, only the nodes that are downstream of (i.e., formally depending on) that placeholder will run under the context defined by the "With" node. In this case you need to make everything that shall be part of the context dependent on the placeholder, which can be accomplished by wiring the update output port of the Placeholder into the update input port of any nodes that shall run under the context (this dependency propagates down the graph, so you don't need to wire it to every node, just to the first node(s) that you want to be part of the context).. The options apply to any matching nodes in this context.- verbose name: Graph [Tag]
- default value: (~)
- port type: Port
- value type: object (can be None)
-
result
Result of the computation.- verbose name: Result
- default value: None
- port type: DataPort
- value type: object (can be None)
- data direction: OUT
-
options
Dictionary of named options to apply. The graph may contain one or more placeholders whose slotname matches the name of the option, which then resolve to the option. Additionally, if the option is named like "node:port" then the option applies to any port with the given name on any node of the given node class (node: these are the CamelCase node classes and the under_score, port names as they show up in the hover documentation, not the friendly names with spaces that show under each node). Both node and port may contain the * or ? wildcard characters to apply to multiple nodes and/or multiple ports matching a certain name pattern. You can mix and match different kinds of options. For brevity you may also omit the node or port part, and use options formatted like node: or :port.- verbose name: Options
- default value: {}
- port type: DictPort
- value type: dict (can be None)
-
is_global
If True, the context implicitly applies to the entire graph (except for those parts that are configuring the context itself). To use multiple global With context managers, you need to chain them together using their respective graph ports since they are required to apply in a well-defined order.- verbose name: Is Global
- default value: False
- port type: BoolPort
- value type: bool (can be None)
-
set_breakpoint
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.- verbose name: Set Breakpoint (Debug Only)
- default value: False
- port type: BoolPort
- value type: bool (can be None)