BatchPackets¶
Concatenate multiple successive packets to maintain real-time rates.
Many processing nodes in NeuroPype have a fixed per-chunk overhead, and therefore, if fewer but longer chunks are being processed, this overhead goes down, which can make the difference between the pipeline running in real time or not (note though that, in some cases, even when all overhead is eliminated, the pipeline can still run slower than real time if the cost of processing the actual streaming data is high enough, especially with high sampling rates, many channels, and heavy processing nodes; in such cases, using this node will not be sufficient). Normally, the pipeline is driven by input plugins that output all data since the last update (e.g., LSL), which naturally increases the chunk size as necessary to maintain real-time rates, and in these cases this node is not necessary. However, when the pipeline is driven with a fixed chunk size that is too small for real time, this node can be inserted to merge successive chunks in order to achieve real-time behavior. Since this node concatenates chunks, the meta-data properties of the output chunk are picked from the last chunk that was concatenated. In general it is recommended to use the fixed batching mode and pick a batch size that is enough to make the pipeline real time. It is also possible to use the realtime mode, in which the first signal stream with time axis and no instance axis is used to infer by how much the stream is lagging relative to the current clock, and the chunk size is chosen adaptively. However, this requires that this node uses the same clock as the one from which the time stamps in the stream's time axis were read off -- otherwise the batching will misbehave badly due to clock drift and offset. This node offers multiple different clocks that you can choose from for this purpose. The simplest alternative to using this node is to instead use an input node that does return all new data since the last update, instead of a fixed chunk size.
Version 1.0.0
Ports/Properties¶
data¶
Data to process.
batching¶
Batching mode. If set to 'fixed', then a fixed number of successive packets (defined in batch size) are concatenated to form a single packet. If set to realtime, then as many packets are concatenated as necessary to maintain real-time processing (batch size is adaptive). Note that you need to select the correct clock (in clock type) for realtime mode.
batchsize¶
Number of successive packets to merge. Only used if batching is set to 'fixed'.
realtime_clock¶
Type of real-time clock to use when using 'realtime' batching mode. This must be the same clock that was used to time-stamp the data that is received by this node. The two available clocks are the UTC clock (wall clock) and the LSL clock. When using UTC and the data comes from another computer, both clocks must be synchronized, e.g., using NTP. The LSL clock is synchronized across computers, if LSL was used to transmit the data.
max_input_lag¶
Maximum acceptable input packet lag. If the lag is larger than this, the node will stash the packet and batch it with the next one (in realtime mode). Unit is seconds.
max_output_lag¶
Maximum acceptable delay incurred by batching packets, in seconds. If this amount of lag between the stream's data and the selected clock is exceeded, a packet will be emitted. This limits the maximum chunk size that can be generated, but if this threshold is reached, your processing system has problems (either your pipeline is not fast enough for real time, or you use the wrong clock).
axis¶
Axis along which to concatenate the data packets. If set to auto, concatenation is along the instance axis if present, or time axis otherwise.
set_breakpoint¶
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.
metadata¶
User-definable meta-data associated with the node. Usually reserved for technical purposes.