Module: markers

Nodes related to event marker handling and formatting.

The nodes in this category are used to add or manipulate event or other data markers. Upon import, are stored in the instance.data axis of a 'markers' data chunk. Markers are typically used to identify events, for which the data can be segmented using the Segmentation node for event-related processing.

AddEventMarker

Find trials matching a particular set of event markers, and add a new marker with a time stamp that matches one of those events.

Such new markers can then be used for purposes such as segmentation to events where the surrounding trial satisfies certain desired conditions etc. The node uses regular expressions, a very powerful language for string matching to identify which markers comprise a trial of interest. See the below link for more details. To add markers matching different criteria (e.g., 'left', and 'right'), use multiple copies of this node in succession.

More Info...

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • stream
    Name of stream containing markers to use for trial selection.

    • verbose name: Stream
    • default value:
    • port type: StringPort
    • value type: str (can be None)
  • trial_criteria
    List of marker patterns that constitute a matching trial. When a trial is found, the event markers in the trial will be matched against this list in order. If all conditions are true, a new event marker will be added to the trial. These conditions are assumed to be regular expressions.

    • verbose name: Required Trial Markers
    • default value: []
    • port type: ListPort
    • value type: list (can be None)
  • trial_start
    The event marker that indicates the start of a trial. Can be a regular expression.

    • verbose name: Trial Start
    • default value:
    • port type: StringPort
    • value type: str (can be None)
  • trial_end
    The event marker that indicates the end of a trial, used to move to the next trial. Can be a regular expression.

    • verbose name: Trial End
    • default value:
    • port type: StringPort
    • value type: str (can be None)
  • event_to_match
    The event within a matching trial that the new event marker should be time-locked to. This should be one of the items in the 'Trial Criteria' parameter. When a matching trial is found a new event marker will be written with the same time stamp as this event marker.

    • verbose name: Event To Match
    • default value:
    • port type: StringPort
    • value type: str (can be None)
  • new_event_marker
    The name of the new event marker to be added to matching trials.

    • verbose name: New Event Marker
    • default value:
    • port type: StringPort
    • value type: str (can be None)
  • verbose
    Verbose debugging output.

    • verbose name: Verbose
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

ExportMarkers

Write all markers and matching timestamps to a 2-column CSV file.

Useful for exporting behavioral data for further processing. The CSV file is written to the same folder as the file being processed. (Not for streaming data.)

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: IN
  • stream
    Name of stream containing markers. Leave empty to auto-select first marker stream.

    • verbose name: Stream
    • default value: None
    • port type: StringPort
    • value type: str (can be None)
  • filename
    File name to record to (csv).

    • verbose name: Filename
    • default value: markers.csv
    • port type: StringPort
    • value type: str (can be None)
  • output_root
    Path for the output root folder.

    • verbose name: Output Root
    • default value:
    • port type: StringPort
    • value type: str (can be None)
  • cloud_host
    Cloud storage host to use (if any). You can override this option to select from what kind of cloud storage service data should be downloaded. On some environments (e.g., on NeuroScale), the value Default will be map to the default storage provider on that environment.

    • verbose name: Cloud Host
    • default value: Default
    • port type: EnumPort
    • value type: object (can be None)
  • cloud_account
    Cloud account name on storage provider (use default if omitted). You can override this to choose a non-default account name for some storage provider (e.g., Azure or S3.). On some environments (e.g., on NeuroScale), this value will be default-initialized to your account.

    • verbose name: Cloud Account
    • default value:
    • port type: StringPort
    • value type: str (can be None)
  • cloud_bucket
    Cloud bucket to read from (use default if omitted). This is the bucket or container on the cloud storage provider that the file would be written to. On some environments (e.g., on NeuroScale), this value will be default-initialized to a bucket that has been created for you.

    • verbose name: Cloud Bucket
    • default value:
    • port type: StringPort
    • value type: str (can be None)
  • cloud_credentials
    Secure credential to access cloud data (use default if omitted). These are the security credentials (e.g., password or access token) for the the cloud storage provider. On some environments (e.g., on NeuroScale), this value will be default-initialized to the right credentials for you.

    • verbose name: Cloud Credentials
    • default value:
    • port type: StringPort
    • value type: str (can be None)

FuseMarkerStreams

Fuse the given marker streams into a single interleaved stream.

When a node such as Merge Streams is used, this will merely bundle multiple streams side-by-side in a single packet. In the case of marker streams, this will result in multiple marker streams. Since many NeuroPype nodes will only process the first marker stream that they find, this node can be used to fuse all marker streams down into a single marker stream that has all markers in it.

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT

InsertMarkers

Insert markers into time-series data.

This node is not applicable to streaming data, that is, you can only apply it to whole recordings like those imported by one of the import nodes. A single application of this node will insert zero or more occurrences of a given marker string (or other payload) into the recording, according to the selected criteria. If you need to insert multiple different marker types, you need to apply this node multiple times in succession. The criteria supported by this node are quite sophisticated, and it is for instance possible to insert markers in time ranges relative to other markers, or between pairs of other markers, and to insert multiple markers randomly or regularly spaced in such time ranges. Some of the common use cases of this node are: inserting calibration-begin and calibration-end markers for use with the accumulate calibration data node, inserting multiple markers into longer data ranges for subsequent extraction of segments/trials from these ranges, or creating jittered placements of markers relative to other markers to simulate timing uncertainty. This node will operate on the first non-empty marker stream that it finds in the data, or fall back to the first empty marker stream, or append one if no marker stream is present.

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • payload
    New marker type to insert. This is the data payload (usually a string) that shall be used for every newly inserted marker. If this is None, the node will pass through the data unchanged.

    • verbose name: Marker Type/payload To Insert
    • default value: mynewevent
    • port type: Port
    • value type: object (can be None)
  • mode
    Type of insertion interval. Events can be inserted either in a fixed absolute time window (absolute), or relative to the beginning/end of the data (relative), or in time windows relative to reference events of a certain type (marker-locked), or in time window spanned by two subsequent events of certain types A and B, optionally with ignored events in between (marker-spanned).

    • verbose name: Insertion Mode
    • default value: relative
    • port type: EnumPort
    • value type: object (can be None)
  • time_limits
    Lower and upper time limit for the insertion range(s) (in seconds). In absolute mode, this is the beginning and end of the time range, regardless of where the signal (if any) begins or ends (unless you know the begin/end time of the signal, using this rarely makes sense); in relative mode, this range is instead relative to the range covered by the signal, where positive numbers count from the beginning of the data, and negative numbers count from the end (e.g., [10, -5] inserts 10 seconds after the beginning up to 5 seconds prior to the end). An empty interval (i.e., []) in this mode is interpreted as the whole data range. In marker-locked mode, this is the beginning and end of the range extracted relative to each marker (i.e., [-1, 2] would begin 1 second before each marker and end 2 seconds after). In marker-spanned mode, for every matching pair of successive markers of types A and B, the range begins x seconds relative to A and ends y seconds relative to B if the time limits are [x, y]. Marker A is called the anchor marker, and marker B is called the second anchor marker in the subsequent settings.

    • verbose name: Time Limits
    • default value: [0, 1]
    • port type: ListPort
    • value type: list (can be None)
  • anchor_marker
    Marker/event type relative to which insertion ranges shall be generated. Applies to marker-locked and marker-spanned mode. If mode is marker-spanned, this is the first marker in a pair of successive markers (referred to as A in other places of the node documentation), while the second marker is given by the second anchor marker.

    • verbose name: Anchor Marker
    • default value: myeventA
    • port type: Port
    • value type: object (can be None)
  • anchor_marker_B
    Second anchor marker (B) if insertion mode is marker-spanned. This marker indicates the end of a spanned range.

    • verbose name: Second Anchor Marker (Spanned Mode)
    • default value: myeventB
    • port type: Port
    • value type: object (can be None)
  • intermittent_markers
    If in spanned mode, this determines how to handle intermittent markers that occur between successive markers A and B by default. If set to allow, any marker type may occur between marker A and the next marker B, and a spanned range will still be generated. If set to forbid, no other marker may occur in between, or else the spanned range will not be generated. Exceptions can be listed in the below parameter for exceptions to this setting.

    • verbose name: Handling Of Intermittent Markers (Spanned Mode)
    • default value: allow
    • port type: EnumPort
    • value type: object (can be None)
  • except_markers
    List of markers types for which to make an exception from the default chosen in intermittent markers. For instance, if the default is allow, then the marker types in this list will be forbidden, and if the default is forbid, then the marker types in this list will be allowed.

    • verbose name: Exceptions To Intermittent Markers (Spanned Mode)
    • default value: []
    • port type: ListPort
    • value type: list (can be None)
  • count
    Number of markers to insert. This is either per interval or per second, depending on the setting chosen in the counting parameter.

    • verbose name: Marker Count To Insert
    • default value: 1
    • port type: FloatPort
    • value type: float (can be None)
  • counting
    Counting measure. The desired number of markers can be inserted either per interval or per second.

    • verbose name: Counting Measure
    • default value: perinterval
    • port type: EnumPort
    • value type: object (can be None)
  • placement
    Marker placement scheme. Narkers can be inserted into ranges either at equal (regular) distance from each other or at random positions.

    • verbose name: Spacing Between Inserted Markers
    • default value: equidistant
    • port type: EnumPort
    • value type: object (can be None)
  • random_seed
    Random seed (for random placement).

    • verbose name: Random Seed
    • default value: 12345
    • port type: IntPort
    • value type: int (can be None)
  • extra_limits
    Extra time limits. These are additional absolute time limits outside of which markers will never be generated (in seconds). Can be used to drop markers outside a particular segment in a recording.

    • verbose name: Extra Limits
    • default value: [-inf, inf]
    • port type: ListPort
    • value type: list (can be None)
  • min_length
    Minimum insertion range length. Ranges shorter than this will be dropped (mostly relevant for marker-spanned mode).

    • verbose name: Min Range Length
    • default value: 0
    • port type: FloatPort
    • value type: float (can be None)
  • max_length
    Maximum insertion range length. Ranges longer than this will be dropped (mostly relevant for marker-spanned mode).

    • verbose name: Max Range Length
    • default value: inf
    • port type: FloatPort
    • value type: float (can be None)

RemoveIntersectingMarkerSpans

Remove spans of markers that intersect signal boundaries.

This node identifies spans of markers that are delimited by a start marker and a subsequent end marker (e.g., trial_start and trial_end). The node will then check if the signal between those markers has any gaps (e.g., due to packet loss, or artifacts having been removed), and if so, it will remove the stretch of markers, optionally with all in-between markers, as well. This can be used to remove all markers for a trial. One of the main uses of this node is in combination with the segmentation node: in a case where one segments around multiple places in a trial (e.g., stimulus and response), this will involve two uses of the Segmentation node. However, if the trial contains a gap on only one of the segments (e.g., near the stimulus but not near the response), then only one of the segments gets dropped by the Segmentation node, and one ends up with different numbers of trials in the two pathways. By preceding instead removing the entire trial if it contains any gap anywhere, it can be ensured that the Segmentation nodes will yield the same number of retained trials. (note: if the segments reach outside the trial bounds, this is currently not guaranteed, however, since this node does not check for gaps outside the span).

Version 0.5.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • start_marker
    Marker that indicates the beginning of a span.

    • verbose name: Start Marker
    • default value: marker1
    • port type: StringPort
    • value type: str (can be None)
  • end_marker
    Marker that indicates the end of a span.

    • verbose name: End Marker
    • default value: marker1
    • port type: StringPort
    • value type: str (can be None)
  • max_gap_length
    Maximum length of time-stamp gaps that are allowed not considered a gap. If a gap between successive samples is larger than this duration in seconds, then any marker whose bounds intersect the gap will be dropped. Optional. Note that if your time stamps are jittered, this will generate a lot of false positives unless you use a very generous cutoff. You can use the DejitterTimestamps node early during your processing to fix that.

    • verbose name: Max Gap Length
    • default value: 0.2
    • port type: FloatPort
    • value type: float (can be None)
  • remove_other_markers
    Remove any other markers that lie in the affected segments.

    • verbose name: Remove Other Markers
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)

RemoveIntersectingMarkers

Remove markers whose neighborhoods intersect gaps in the data and/or specified other markers.

This node can be useful when segments are extracted relative to event markers, and some of these segments may intersect these gaps. The Segmentation node will then drop any segments that cross gaps (since all segments emitted by it must have the same length, i.e., number of samples), and therefore the number of segments retained will be lower than the number of event markers to begin with. For a single Segmentation run this may not be a problem, but if one uses multiple Segmentation operations in parallel with different segment lengths, then one may end up with different numbers of trials in the resulting data. This can be a problem when the segments shall later be 1:1 associated with each other, and this removal node can be used to prevent that case. One simply removes any markers that are within a generous distance from any nearby gaps, and then all subsequent Segmentation runs are guaranteed to produce the same number of segments. For simplicity, the time range checked relative to each marker by this node is specified in the same manner as in the segmentation node.

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • time_bounds
    Time limits around the markers that are checked for intersection with gaps, in seconds. The first number is the start time of the range relative to each marker, and the second number is the end time of the range. If any of these numbers is negative, it means that the time point lies this many seconds before the marker, while positive numbers mean that the point lies after the marker. This is the same as in the Segmentation node. So a range of [-2,2] means that any markers that are within 2 seconds of a subsequent marker are dropped from the data.

    • verbose name: Marker Safety Bounds
    • default value: [-1, 2]
    • port type: ListPort
    • value type: list (can be None)
  • max_gap_length
    Maximum length of time-stamp gaps that are allowed not considered a gap. If a gap between successive samples is larger than this duration in seconds, then any marker whose bounds intersect the gap will be dropped. Optional. Note that if your time stamps are jittered, this will generate a lot of false positives unless you use a very generous cutoff. You can use the DejitterTimestamps node early during your processing to fix that.

    • verbose name: Max Gap Length
    • default value: 0.2
    • port type: FloatPort
    • value type: float (can be None)
  • sample_offset
    Optional additional shift of the bounds position, in samples. This setting allows to shift all bounds by this many samples relative to the marker, which can be used to match the behavior of other systems (e.g., BCILAB or EEGLAB) in a sample-exact fashion. Same as in the Segmentation node.

    • verbose name: Additional Sample Offset
    • default value: 0
    • port type: IntPort
    • value type: int (can be None)
  • avoid_markers
    Avoid these other markers. If a list of markers is given (which may include wildcards), then markers are removed whose neighborhoods cross any of those matching markers to avoid.

    • verbose name: Avoid Markers
    • default value: None
    • port type: ListPort
    • value type: list (can be None)

RewriteMarkers

Rewrite event markers according to one or more mapping rules.

This node allows one to rewrite one or more event markers to new markers. The node also optionally allows to use pattern-matching syntax, including wildcard syntax and regular expression syntax, which allows to match a large set of markers and then to remap those to a simpler, canonical marker name. An empty dictionary {} in the rewriting rules field will skip this node.

Version 0.9.3

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • rules
    Event marker rewriting rules. This is given as one or more mappings of some input marker string(s) or pattern(s) to some specific output marker string. This is given as in {some_input_markers1: 'mynewmarker1', some_input_marker2: 'mynewmarker2'}, where some_input_markers can be either a plain string, if just one marker shall be rewritten to a given new output marker, or a string pattern with * and/or ? wildcard characters (wildcard pattern syntax), or a regular expression (regex pattern syntax). It is also possible to use a list of input markers in round braces, as in {('myoldmarkerA', 'myoldmarkerB'): 'mynewmarker', ...}, or a list of string patterns. Pro Tip: if the input and output are identical no rewriting will take place. Use this combined with the 'remove all others' option to drop all markers except those matching the rewrite rule, without doing any actual rewriting. (Use regex as the pattern syntax to retain all markers that match a particular regex pattern.)

    • verbose name: Rewriting Rule
    • default value: {}
    • port type: Port
    • value type: object (can be None)
  • pattern_syntax
    Optionally the syntax to use for the input marker patterns. If set to none, the input strings are not used as patterns but used to match a specific marker string verbatim. If set to wildcards, then the special character * can be used to match a string of any length, including empty, and the character ? can be used to match a any character.

    • verbose name: Pattern Syntax
    • default value: wildcards
    • port type: EnumPort
    • value type: object (can be None)
  • regex_sub
    Check this box if you want to use regex substitution with groups. Remember to escape any groups like this: \2 instead of .

    • verbose name: Regex Sub
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • remove_all_others
    Remove all other markers. If enabled, all markers other than those matched by the rules will be removed from the data.

    • verbose name: Remove All Others
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • iv_column
    If the marker stream instance axis data is a recarray then here we choose which column of the recarray to use for mapping. If the marker stream instance axis data is an array of strings then this field does nothing.

    • verbose name: Iv Column
    • default value: Marker
    • port type: StringPort
    • value type: str (can be None)

StripLongMarkers

Removes (replaces) event markers that exceed a maximum length.

Version 0.5.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • removed
    Stripped markers.

    • verbose name: Removed
    • default value: None
    • port type: DataPort
    • value type: list (can be None)
    • data direction: OUT
  • max_length
    Maximum marker length.

    • verbose name: Max Length
    • default value: 512
    • port type: IntPort
    • value type: int (can be None)
  • replace_with
    Replace removed markers with this string.

    • verbose name: Replace With
    • default value: (removed-marker-too-long)
    • port type: StringPort
    • value type: str (can be None)

TrimMarkerSegment

Trim data before and after specified begin/end markers.

The incoming data must have a time axis and a marker streamwith an instance axis. Data is trimmed along the time axis based on the timestamps of the given markers.

Version 1.1.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • event_start
    Name of starting event marker. If empty, data will be retained from the start of the data until the event_end marker

    • verbose name: Event Start
    • default value: start
    • port type: StringPort
    • value type: str (can be None)
  • start_occurrence
    In case of multiple events of same name, index of desired event occurrence (0 is first index, -1 is last index, -2 is second to last, etc.) .

    • verbose name: Start Occurrence
    • default value: 0
    • port type: IntPort
    • value type: int (can be None)
  • event_end
    Name of ending event marker. If empty, data will be retained " from the event_start marker to the end of the data.

    • verbose name: Event End
    • default value: end
    • port type: StringPort
    • value type: str (can be None)
  • end_occurrence
    In case of multiple events of same name, index of desired event occurrence (0 is first index, -1 is last index, -2 is second to last, etc.) .

    • verbose name: End Occurrence
    • default value: -1
    • port type: IntPort
    • value type: int (can be None)
  • ignore_errors
    Continue operation if either marker is not found (no trimming will occur but the pipeline will not terminate).

    • verbose name: Ignore Errors
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • verbose
    Show informational messages.

    • verbose name: Verbose
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)