Module: machine_learning

Machine learning algorithms and related nodes.

Machine learning methods typically expect their inbound chunks to have an instance axis. Most existing nodes implement supervised machine learning algorithms: these nodes will only output predictions after they have been trained. To train a machine learning node, it needs to receive a packet that has both a data chunk, and a chunk of target instances (must have the flag has_targets set to True). Naturally, the number of instances in the data and target chunk must match. A node will usually output the predictions for every chunk that it receives once trained, including for the initial training chunk. Also, note that most of these nodes, with few exceptions, will only adapt themselves on non-streaming chunks (flag is_streaming is not set to True). Some nodes in here provide important related functionality, such as nodes for annotation with target labels and and management of training/calibration data.

AccumulateCalibrationData

Accumulate calibration data and emit it in one large chunk.

This node is for setups where you have streaming data and some of your processing nodes need to be calibrated, and you intend to collect the calibration data on the fly at the beginning of the live session, instead of using a recording of previously collected calibration data (in the latter case you would use the Inject Calibration Data node instead of this node). If all your processing nodes are capable of incremental calibration on streaming data, you do not need this node -- however, only very few nodes can do that: instead, most adaptive processing nodes require a long chunk that has all the calibration data, on which they then perform a one-step calibration calculation. This node handles the task of buffering up your streaming calibration data into one chunk, and then emits it all in one go, so that subsequent nodes can adapt themselves on it. For this to work, the accumulate node needs to know where the calibration data begin, and where they end: you are responsible for telling it by inserting special marker in the data at the beginning of the calibration period, and another special marker at the end (the marker strings to look for can be customized). This node also supports a few options only visible in expert mode to decide what happens to any streaming data prior to the beginning of the calibration section: usually you want to drop such data since your pipeline would not be able to process it yet anyway, but this can be overridden with a parameter. The other option is what happens when there is another calibration section some time after the first one: you may choose to either ignore that section, or to trigger another recalibration. Also, other than the mere length of the chunk, the way in which the calibration data that this node emits is distinguished from the regular streaming data which it also emits, is that this node marks the calibration chunk that it outputs as 'non-streaming' by setting the corresponding flag on the packet (since most adaptive nodes will only be able to update themselves on non-streaming data). Limitations: While it is possible to use this node to demarcate multiple successive calibration windows, and it will then emit a calibration chunk for each of them, this node basically assumes that a single input chunk is intersected by at most one such calibration time window. If that assumption is violated, this node will default to fusing the successive calibration windows into a longer one that covers all calibration data. Warnings will be generated in such cases.

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • begin_marker
    Marker string that indicates that calibration data is beginning. As explained in the help text for the node, a marker is necessary to inform the node about when the data that it should accumulate begins (and ends). The recommended way to get this marker into the data is by emitting it from the same program that also generates any other calibration-relevant markers (e.g., those that are picked up in the Assign Targets Markers node). Can also be a time offset, see Marker mode argument.

    • verbose name: Begin Marker
    • default value: calib-begin
    • port type: Port
    • value type: object (can be None)
  • end_marker
    Marker string that indicates that calibration data is ending. As for the begin marker, the best way to get it embedded into the data stream is to emit it from the program that manages the calibration process (that program would usually emit markers that are used e.g., by the Segmentation or the machine learning nodes). Can also be a time offset, see Marker mode argument.

    • verbose name: End Marker
    • default value: calib-end
    • port type: Port
    • value type: object (can be None)
  • marker_mode
    How to interpret the begin_marker and end_marker. In the default setting 'markers', the are matched to event markers that are assumed to be present in the data. In the 'relative-times' mode, they are interpreted as time offsets counting from the beginning of the data, in seconds.

    • verbose name: Marker Mode
    • default value: markers
    • port type: EnumPort
    • value type: object (can be None)
  • print_markers
    Print markers. This prints markers during the calibration period for debugging/inspection.

    • verbose name: Print Markers
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • verbose
    Verbose output.

    • verbose name: Verbose
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • emit_calib_data
    Emit the calibration data. If set to False, the calibration data portion is dropped.

    • verbose name: Emit Calibration Data
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • emit_predict_data
    Emit the non-calibration ('streaming') data. If set to False, the data outside the calibration markers will be dropped.

    • verbose name: Emit Streaming Data
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • calibration_first
    Do not emit any streaming data before the first calibration chunk has ended. This is needed for many methods that can only predict after they have been calibrated.

    • verbose name: No Output Before Calibration Data
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • can_recalibrate
    Allow re-calibration. If false, only a single calibration period will be allowed and subsequent calibration markers will be ignored.

    • verbose name: Allow Recalibration
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

AssignTargets

Select which markers contain event-related signal activity, and optionally assign numeric target values to these markers for use in machine learning.

This node is part of a standard workflow for analysis of event-related signal activity, and optionally for machine learning on that activity. First of all, this node allows you to define which of the (possibly many) markers in the data should be used for event-related analysis. Second, you can assign different numeric values to different events (since each event has an associated string, you can give a mapping that assigns different values to different strings). There is also an option to accept any marker strings that can be converted into numbers, and take these numbers as the target values (this is useful for the case where regression targets are being specified, instead of classification). Once your markers are annotated in this way in the data, subsequent nodes will act on that subset of markers (e.g., the Segmentation node will extract segments around only the target markers), and if you have assigned numeric target values to specific markers, any subsequent machine learning nodes will interpret these values as the desired output values (or "labels") that the machine learning node is supposed to predict whenever it sees data that looks like what it observed around those markers in its training data. Thus, a typical workflow is to have an Assign Target Markers node, followed by a Segmentation node, optionally followed by some segment processing, followed by a machine learning node; usually you also need a way to feed both training and test/live data into this chain of nodes, e.g., using the Inject Calibration Data node or the Accumulate Calibration Data node prior to the Assign Target Markers node. Tip: since this node only distinguishes markers based on exact string matching, you may need to preprocess your marker strings beforehand using other nodes. It can also be helpful to insert new markers in the data based on custom criteria using, e.g., the Insert Markers node, before applying this node.

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • mapping
    Mapping of matching criteria (e.g ., marker strings) to target values. Any instances that match a given criterion will be assigned the associated target value. The format of the criteria can be overridden by mapping_format. The mapping can be given either fully explicit as a dictionary of {crit1: target-value1, crit2: target-value2, ...}, or using the shorthand list notation [crit1, crit3, crit3], which is equivalent to {crit1:0, crit2:1, crit3:3, ...}. An unorderered set {crit1, crit2, crit3} can be given to simply set target values for any matching criterion to 1.

    • verbose name: Value Assignment
    • default value: {'M1': 0, 'M2': 1}
    • port type: Port
    • value type: object (can be None)
  • mapping_format
    Format of the criterion strings. If set to 'names', each instance (e.g., marker string) needs to match the provided string exactly. If set to 'wildcards', the criterion is a wildcard expression that may include * or ? characters. If set to 'conditions', the mapping can be a restricted Python expression that may refer to other instance fields (e.g., "Marker == 'left' and Duration > 4.0", provided that the the instances have fields named Marker and Duration). See NeuroPype's QueryGrammar for more details on the available functions. The special format 'passthrough-numbers' ignores the mapping entirely, and simply converts the marker strings to numbers, and uses those as target values. The 'compat' format is primarily for backwards compatibility with the settings of some deprecated fields. It is recommended to instead always select the syntax that you're using explicitly.

    • verbose name: Mapping Format
    • default value: compat
    • port type: EnumPort
    • value type: object (can be None)
  • iv_column
    Choose which column of the instance axis data table to use for mapping, if mapping is 'names' or 'wildcards'. This will almost always be 'Marker' (the default).

    • verbose name: Default Condition Field
    • default value: Marker
    • port type: StringPort
    • value type: str (can be None)
  • is_categorical
    If set then the TargetValue column in the IV table will be marked as categorical.

    • verbose name: Is Categorical
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • also_legacy_output
    Also write the target values in the legacy location. The target values will also be written into the .data field that is indexed by the instance axis.

    • verbose name: Also Legacy Output
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • use_numbers
    Alternatively convert number strings to target values. If this is checked, the marker assignment is ignored, and the node will treat any marker string that can be converted to a number as a target marker, and use the corresponding number as the target value. This is useful when regression targets are encoded in marker strings.

    • verbose name: Use Numbers Instead (Regression)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • support_wildcards
    Support wildcard matching.

    • verbose name: Support Wildcards
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • verbose
    Enable verbose output.

    • verbose name: Verbose
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

BalanceClasses

Balance the per-class trial counts in the data.

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • strategy
    Strategy to apply. Duplicate means to duplicate under-represented trials, and drop means to drop excess over-represented trials. Mixed is a mix in which the overall number of trials is approximately kept, but relative frequencies are rebalanced (involves both and dropping and duplication).

    • verbose name: Strategy
    • default value: duplicate
    • port type: EnumPort
    • value type: object (can be None)
  • field_name
    Name of field to rebalance. The values of this field will occur approximately equally in the given instances.

    • verbose name: Field Name
    • default value: TargetValue
    • port type: StringPort
    • value type: str (can be None)
  • binning_field
    Optional name of field to bin on. This will, for each unique value in this field, perform the balancing within all instances where the field takes on the same value.

    • verbose name: Binning Field
    • default value:
    • port type: StringPort
    • value type: str (can be None)
  • randseed
    Optionally the random seed to use to get deterministic results.

    • verbose name: Randseed
    • default value: 12345
    • port type: IntPort
    • value type: int (can be None)

BayesianRidgeRegression

Estimate a continuous output value from features using Bayesian Ridge Regression.

Bayesian Ridge regression is a an elegant Bayesian method to learn a linear mapping between input data and desired output values from training data, which is closely related to ridge regression. The main difference to ridge regression is that in ridge regression, there is a tunable parameter that controls how strongly the method should be regularized. This parameter controls effectively how flexible or complex the solution may be, to prevent over-fitting to random details of the data and therefore to improve the generalization to new data. In ridge regression, this parameter is tuned using cross-validation, that is, by empirical testing on held-out data. In the Bayesian variant, there is such a parameter as well, however, the optimal degree of regularization is estimated from the data as well in a theoretically clean and principled fashion. This method assumes that both inputs and outputs are Gaussian distributed, that is, have no or very few major statistical outliers. If the output follows a radically different distribution, for instance between 0 and 1, or nonnegative, or discrete values, then different methods may be more appropriate (for instance, classification methods for discrete values). To ameliorate the issue of outliers in the data, the raw data can be cleaned of artifacts with various artifact removal methods. To the extent that the assumptions hold true, this method is highly competitive with other linear methods. Like all machine learning methods, this method needs to be calibrated ("trained") before it can make any predictions on data. For this, the method requires training instances and associated training labels. The typical way to get such labels associated with time-series data is to make sure that a marker stream is included in the data, which is usually imported together with the data using one of the Import nodes, or received over the network alongside with the data, e.g., using the LSL Input node (with a non-empty marker query). These markers are then annotated with target labels using the Assign Targets node. To generate instances of training data for each of the training markers, one usually uses the Segmentation node to extract segments from the continuous time series around each marker. Since this machine learning method is not capable of being trained incrementally on streaming data, the method requires a data packet that contains the entire training data; this training data packet can either be accumulated online and then released in one shot using the Accumulate Calibration Data node, or it can be imported from a separate calibration recording and then spliced into the processing pipeline using the Inject Calibration Data, where it passes through the same nodes as the regular data until it reaches the machine learning node, where it is used for calibration. Once this node is calibrated, the trainable state of this node can be saved to a model file and later loaded for continued use.

More Info...

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • max_iter
    Maximum number of iterations. This is one of the stopping criteria to limit the compute time. The default is usually fine, and gains from increasing the number of iterations will be minimal (it can be worth experimenting with lower iteration numbers if the algorithm must finish in a fixed time budget, at a cost of potentially less accurate solutions).

    • verbose name: Maximum Number Of Iterations
    • default value: 300
    • port type: IntPort
    • value type: int (can be None)
  • tolerance
    Convergence tolerance. This is the desired errors tolerance or acceptable inaccuracy in the solution. Using larger values gives less accurate results, but will lead to faster compute times. Note that, for biosignal-driven machine learning systems, one often does not need very small tolerances.

    • verbose name: Tolerance
    • default value: 0.001
    • port type: FloatPort
    • value type: float (can be None)
  • verbosity
    Verbosity level. Higher numbers will produce more extensive diagnostic output.

    • verbose name: Verbosity Level
    • default value: 0
    • port type: IntPort
    • value type: int (can be None)
  • include_bias
    Include a bias term. If false, your features need to be centered, or include a dummy feature set to 1.

    • verbose name: Include Bias Term
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • normalize_features
    Normalize features. Should only be disabled if the data comes with a predictable scale (e.g., normalized in some other way).

    • verbose name: Normalize Features
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • alpha_shape
    Alpha shape parameter. This is only included for completeness and usually does not have to be adjusted. This is the shape parameter for the Gamma distribution prior over the alpha parameter. By default, this is an uninformative prior.

    • verbose name: Alpha Shape Parameter
    • default value: 1e-06
    • port type: FloatPort
    • value type: float (can be None)
  • alpha_rate
    Alpha rate parameter. This is only included for completeness and usually does not have to be adjusted. This is the rate parameter for the Gamma distribution prior over the alpha parameter. By default, this is an uninformative prior.

    • verbose name: Alpha Rate Parameter
    • default value: 1e-06
    • port type: FloatPort
    • value type: float (can be None)
  • lambda_shape
    Lambda shape parameter. This is only included for completeness and usually does not have to be adjusted. This is the shape parameter for the Gamma distribution prior over the lambda parameter. By default, this is an uninformative prior.

    • verbose name: Lambda Shape Parameter
    • default value: 1e-06
    • port type: FloatPort
    • value type: float (can be None)
  • lambda_rate
    Lambda rate parameter. This is only included for completeness and usually does not have to be adjusted. This is the rate parameter for the Gamma distribution prior over the lambda parameter. By default, this is an uninformative prior.

    • verbose name: Lambda Rate Parameter
    • default value: 1e-06
    • port type: FloatPort
    • value type: float (can be None)
  • initialize_once
    Calibrate the model only once. If set to False, then this node will recalibrate itself whenever a non-streaming data chunk is received that has both training labels and associated training instances.

    • verbose name: Calibrate Only Once
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • dont_reset_model
    Do not reset the model when the preceding graph is changed. Normally, when certain parameters of preceding nodes are being changed, the model will be reset. If this is enabled, the model will persist, but there is a chance that the model is incompatible when input data format to this node has changed.

    • verbose name: Do Not Reset Model
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

ClearTargets

Clear the target information from the markers in the given data.

This node undoes the operation of the Assign Target Markers node.

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • subset
    Indices of previously assigned markers to clear target value from. Only non-nan markers are counted in the indexing. This may be a convenient way to eliminate bad trials.

    • verbose name: Subset
    • default value: None
    • port type: ListPort
    • value type: list (can be None)

CovRiemann

Classify covariance matrices based on a Riemannian minimum-distance-to-mean criterion.

This method assumes that the input features are not feature vectors but instead covariance matrices (i.e., as computed using one of the covariance estimation nodes). This node will find the average (centroid) of of trials in each class, and then classify new trials (each a covariance matrix) based on what class mean they are closest to. This distance metric is non-Euclidean and instead follows the Riemannian geometry of covariance matrices. This method is moderately sensitive to major outliers, but less so than, for instance, LDA or QDA. To minimize outlier issues, the raw data can be cleaned of artifacts with various artifact removal methods. This method is quite fast and requires no regularization or other tunable parameters; for classifying data based on covariance information, this method is competitive with a number of other state-of-the-art methods that rely on traditional off-the-shelf classifiers, such as LDA, QDA, SVM, logistic regression, etc. This method will return not the most likely class label for each trial it is given, but instead estimated probabilies for each class (=category of labels), that the trial is actually of that class However, these probabilities are currently not precisely calibrated, so that, while they are fine for continuous control in non-critical systems or for classification purposes, they should not be interpreted literally, e.g., in a decision-theoretic context (this will be addressed in a future version). Like all machine learning methods, this method needs to be calibrated ("trained") before it can make any predictions on data. For this, the method requires training instances and associated training labels. The typical way to get such labels associated with time-series data is to make sure that a marker stream is included in the data, which is usually imported together with the data using one of the Import nodes, or received over the network alongside with the data, e.g., using the LSL Input node (with a non-empty marker query). These markers are then annotated with target labels using the Assign Targets node. To generate instances of training data for each of the training markers, one usually uses the Segmentation node to extract segments from the continuous time series around each marker. Since this machine learning method is not capable of being trained incrementally on streaming data, the method requires a data packet that contains the entire training data; this training data packet can either be accumulated online and then released in one shot using the Accumulate Calibration Data node, or it can be imported from a separate calibration recording and then spliced into the processing pipeline using the Inject Calibration Data, where it passes through the same nodes as the regular data until it reaches the machine learning node, where it is used for calibration. Once this node is calibrated, the trainable state of this node can be saved to a model file and later loaded for continued use.

Version 0.1.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • max_iter
    Max number of iterations for mean estimate. This value serves as an additional stopping criterion for the learning algorithm, and can be used to ensure that the method runs in a fixed time budget. This rarely has to be tuned.

    • verbose name: Max Number Of Iterations
    • default value: 50
    • port type: IntPort
    • value type: int (can be None)
  • tolerance
    Convergence tolerance for mean estimate. A lower tolerance will lead to longer running times and can result in more accurate solutions -- however, note that the actual difference in the outputs will be minimal at best, unless a very coarse tolerance is used.

    • verbose name: Convergence Tolerance
    • default value: 1e-08
    • port type: FloatPort
    • value type: float (can be None)
  • dont_reset_model
    Do not reset the model when the preceding graph is changed. Normally, when certain parameters of preceding nodes are being changed, the model will be reset. If this is enabled, the model will persist, but there is a chance that the model is incompatible when input data format to this node has changed.

    • verbose name: Do Not Reset Model
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

ElasticNetRegression

Estimate a continuous output value from features using linear regression with Elastic Net regularization.

Elastic net regression is a principled statistical technique to learn a linear mapping between input data and desired output values from training data. Elastic net combines the regularization terms of ridge regression and LASSO regression, which combines some of the benefits of both techniques. Most importantly, elastic net overcomes a shortcoming of LASSO in the presence of multiple highly correlated features, which is that such features will usually not receive similar weights, but instead often rather arbitrary relative weights. As such, elastic net is a preferable technique if sparse feature selection as provided by LASSO is desirable, but if features may also be correlated. Elastic net includes a tuning parameter that allows to blend between l1 (LASSO) and l2 (ridge) regularization terms, which is automatically tuned together with the overall regularization strength parameter using cross-validation. For this reason, this method can at the extremes imitate either of the two algorithms if that is beneficial given the data. Due to the need for tuning the additional parameter, the running time will however be significantly larger than using either method directly. If there are very few trials, or some extensive stretches of the data exhibit only one class, the procedure used to find the regularization parameters (cross-validation) can fail with an error that there were too few or no trials of a given class present. Elastic net regression assumes that both inputs and outputs are Gaussian distributed, that is, have no or very few major statistical outliers. If the output follows a radically different distribution, for instance between 0 and 1, or nonnegative, or discrete values, then different methods may be more appropriate (for instance, classification methods for disrete values). To ameliorate the issue of outliers in the data, the raw data can be cleaned of artifacts with various artifact removal methods. To the extent that the assumptions hold true, this method is highly competitive with other (sparse) linear methods. Like all machine learning methods, this method needs to be calibrated ("trained") before it can make any predictions on data. For this, the method requires training instances and associated training labels. The typical way to get such labels associated with time-series data is to make sure that a marker stream is included in the data, which is usually imported together with the data using one of the Import nodes, or received over the network alongside with the data, e.g., using the LSL Input node (with a non-empty marker query). These markers are then annotated with target labels using the Assign Targets node. To generate instances of training data for each of the training markers, one usually uses the Segmentation node to extract segments from the continuous time series around each marker. Since this machine learning method is not capable of being trained incrementally on streaming data, the method requires a data packet that contains the entire training data; this training data packet can either be accumulated online and then released in one shot using the Accumulate Calibration Data node, or it can be imported from a separate calibration recording and then spliced into the processing pipeline using the Inject Calibration Data, where it passes through the same nodes as the regular data until it reaches the machine learning node, where it is used for calibration. Once this node is calibrated, the trainable state of this node can be saved to a model file and later loaded for continued use.

More Info...

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • num_alphas
    Number of values in regularization strength search grid. This method determines the optimal regularization strength by testing a number of different strength values between the minimum and maximum value. The running time increases with a finer search grid, but the found solution may be slightly better regularized when using a fine grid (e.g., 100 values) instead of a coarse grid (e.g., 20 values).

    • verbose name: Number Of Values In Reg. Search Grid
    • default value: 100
    • port type: IntPort
    • value type: int (can be None)
  • num_folds
    Number of cross-validation folds for parameter search. Cross-validation proceeds by splitting up the data into this many blocks of trials, and then tests the method on each block. For each fold, the method is re-trained on all the other blocks, excluding the test block (therefore, the total running time is proportional to the number of folds). This is not a randomized cross-validation, but a blockwise cross-validation, which is usually the correct choice if the data stem from a time series. If there are few trials in the data, one can use a higher number here (e.g., 10) to ensure that more data is available for training.

    • verbose name: Number Of Cross-Validation Folds
    • default value: 5
    • port type: IntPort
    • value type: int (can be None)
  • l1_ratio
    Tradeoff parameter between between l1 and l2 penalties. This parameter controls the balance between the l1 regularization term (as in LASSO) and the l2 regularization term (as in ridge regression). A value of 0 leads to exclusive use of l2, and a value of 1 leads to exlusively use of l1. This is a list of candidate values, the best of which is found via an exhaustive search (i.e., each value is tested one by one, and therefore the total running time is proportional to the number of values listed here). In fact, for each setting of this parameter, the entire list of possible values for the regularization strength is tested, so the running is also proportional to the number of those values. The details of the parameter search can be controlled via the search metric and number of folds parameters. The default search range is intentionally coarse for quick running times; refine it to smaller steps to obtain potentially better solutions, but do not expect massive gains from refining.

    • verbose name: Regularization Type Tradeoff Parameter
    • default value: [0.1, 0.5, 0.7, 0.9, 0.95, 0.99, 1]
    • port type: ListPort
    • value type: list (can be None)
  • max_iter
    Maximum number of iterations. This is one of the stopping criteria to limit the compute time. The default is usually fine, and gains from increasing the number of iterations will be minimal (it can be worth experimenting with lower iteration numbers if the algorithm must finish in a fixed time budget, at a cost of potentially less accurate solutions).

    • verbose name: Maximum Number Of Iterations
    • default value: 1000
    • port type: IntPort
    • value type: int (can be None)
  • tolerance
    Convergence tolerance. This is the desired errors tolerance or acceptable inaccuracy in the solution. Using larger values gives less accurate results, but will lead to faster compute times. Note that, for biosignal-driven machine learning systems, one often does not need very small tolerances.

    • verbose name: Tolerance
    • default value: 0.0001
    • port type: FloatPort
    • value type: float (can be None)
  • num_jobs
    Number of parallel compute jobs. This value only affects the running time and not the results. Values between 1 and twice the number of CPU cores make sense to expedite computation, but may temporarily reduce the responsiveness of the machine. The value of -1 stands for all available CPU cores.

    • verbose name: Number Of Parallel Jobs
    • default value: 1
    • port type: IntPort
    • value type: int (can be None)
  • verbosity
    Verbosity level. Higher numbers will produce more extensive diagnostic output.

    • verbose name: Verbosity Level
    • default value: 0
    • port type: IntPort
    • value type: int (can be None)
  • include_bias
    Include a bias term. If false, your features need to be centered, or include a dummy feature set to 1.

    • verbose name: Include Bias Term
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • normalize_features
    Normalize features. Should only be disabled if the data comes with a predictable scale (e.g., normalized in some other way).

    • verbose name: Normalize Features
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • min_alpha
    Minimum regularization strength. This is expressed as a factor of the maximum regularization strength, which is calculated from the data. By default, the optimal strength will be searched between this value and the maximum.

    • verbose name: Minimum Regularization Strength
    • default value: 0.001
    • port type: FloatPort
    • value type: float (can be None)
  • alphas
    Override regularization strength. Optionally the default regularization search grid can be overridden by giving an explicit list of values here, although this is rarely necessary or very helpful. This is a list of candidate values, the best of which is found via an exhaustive search (i.e., each value is tested one by one, and therefore the total running time is proportional to the number of values listed here). The details of the parameter search can be controlled via the search metric and number of folds parameters. Larger values cause stronger regularization, that is, less risk of the method over-fitting to random details of the data, and thus better generalization to new data. A value of 0 means no regularization, and there is no upper limit to how large the values that can be given here may be -- however, depending on the scale of the data and the number of trials, there is a cutoff beyond which all features are weighted by zero, and are thus unused. Often one covers a range between 0.1 and 10, and at times 0.01 to 100. Typically the values here are not linearly spaced, but follow an exponential progression (e.g., 0.25, 0.5, 1, 2, 4, 8, ... etc). The default search range is intentionally coarse for quick running times; refine it to smaller steps to obtain potentially better solutions, but do not expect massive gains from refining.

    • verbose name: Override Regularization Strength
    • default value: None
    • port type: ListPort
    • value type: list (can be None)
  • selection
    Parameter update schedule. Using random here can be significantly faster when higher tolerance values are used.

    • verbose name: Parameter Update Schedule
    • default value: cyclic
    • port type: EnumPort
    • value type: object (can be None)
  • positivity_constraint
    Constrain weights to be positive. This is a special (and rarely-used) flavor of this method, in which the learned weights are constrained to be positive.

    • verbose name: Constrain Weights To Be Positive
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • precompute
    Precompute shared data. Precompute some shared data that is reused during parameter search. Aside from 'auto', this can be set to True or False. Auto attempts to determine the best choice automatically.

    • verbose name: Precompute Shared Data
    • default value: auto
    • port type: Port
    • value type: object (can be None)
  • random_seed
    Random seed. Different values may give slightly different outcomes.

    • verbose name: Random Seed
    • default value: 12345
    • port type: IntPort
    • value type: int (can be None)
  • initialize_once
    Calibrate the model only once. If set to False, then this node will recalibrate itself whenever a non-streaming data chunk is received that has both training labels and associated training instances.

    • verbose name: Calibrate Only Once
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • dont_reset_model
    Do not reset the model when the preceding graph is changed. Normally, when certain parameters of preceding nodes are being changed, the model will be reset. If this is enabled, the model will persist, but there is a chance that the model is incompatible when input data format to this node has changed.

    • verbose name: Do Not Reset Model
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

HierarchicalDiscriminantComponentAnalysis

Use Hierarchical Discriminant Component Analysis (HDCA) to classify data instances.

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • shrinkage_within
    Regularization strength within each time slice. If using 'auto', the parameter is computed analytical (using the Ledoit-Wolf method); otherwise a number between 0 (least) and 1 (most) should be given.

    • verbose name: Shrinkage Within
    • default value: 0.01
    • port type: Port
    • value type: object (can be None)
  • shrinkage_across
    Regularization strength across the time slices. If using 'auto', the parameter is computed analytical (using the Ledoit-Wolf method); otherwise a number between 0 (least) and 1 (most) should be given.

    • verbose name: Shrinkage Across
    • default value: 0.01
    • port type: Port
    • value type: object (can be None)
  • probabilistic
    Output probabilities instead of class labels.

    • verbose name: Probabilistic
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • class_weights
    Per-class weight (dictionary). This is formatted as a Python-style dictionary that assigns to each numeric class (e.g., 0, 1) a weighting (e.g., 1.0). The weights do not have to add up to 1. Example syntax: {'0': 0.5, '1': 0.5} (note the quotes before the colons).

    • verbose name: Class Weights
    • default value: None
    • port type: Port
    • value type: object (can be None)
  • tolerance
    Threshold for rank estimation in SVD.

    • verbose name: Tolerance
    • default value: 0.0001
    • port type: FloatPort
    • value type: float (can be None)
  • initialize_once
    If set to True, the model is trained once.

    • verbose name: Initialize Once
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • dont_reset_model
    Do not reset the model when the preceding graph is changed. Normally, when certain parameters of preceding nodes are being changed, the model will be reset. If this is enabled, the model will persist, but there is a chance that the model is incompatible when input data format to this node has changed.

    • verbose name: Do Not Reset Model
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • verbose
    Enable verbose output.

    • verbose name: Verbose
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

InjectCalibrationData

Insert calibration data into the stream.

This node is useful when you have a pipeline that receives some data from a data source (e.g., an LSL Input node) and performs some processing on the data (e.g., make predictions using some machine learning nodes) -- and this pipeline requires calibration on some previously recorded calibration data. This node can be inseted between the data source and the first adaptive processing step in your pipeline, and what it allows you to do is as follows: the node has a second input port, which you can wire to some node(s) that import your calibration data (e.g., Import XDF). Then, the Inject Calibration Data node will first let through the calibration recording, which will then trickle down your subsequent processing pipeline, giving every node a chance to calibrate itself. After that, the Inject Calibration Data node will let through the regular streaming data from your actual data source, so that the pipeline can do its regular processing, now that it is calibrated. Note that, since on the first tick this node outputs the calibration data, the question arises what happens to any streaming data that also came into the Inject Calibration Data node on that same first tick -- you can choose to either emit it on the next tick, and thus delay this and all subsequent streaming packets by one tick (1/25th of a second at NeuroPype's default update rate), or you can choose to drop it. Tip: if you want to collect the calibration data on the fly at the beginning of a real-time session instead of using a previous recording, you can instead use the Accumulate Calibration Data node.

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • streaming_data
    Streaming data.

    • verbose name: Streaming Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: IN
  • calib_data
    Calibration data.

    • verbose name: Calib Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: IN
  • data
    Output data.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: OUT
  • delay_streaming_packets
    Whether streaming packets should be delayed by one tick, or whether the first streaming packet should be dropped. If enabled, the first streaming packet will be buffered by this node, and emitted on the next tick (since on the first tick this node outputs the calibration data); all subsequent streaming packets will naturally also have to be delayed by one tick. If disabled, then the first streaming packet will be dropped, and there is no delay. For streaming processing, it is usually best to drop the packet, since incoming streaming data can generally not be acted on before the pipeline is finished calibrating. However, if the data on the streaming input port is actually a single packet holding a whole recording that shall be processed in an offline fashion, then it must be delayed by one tick, since dropping it would drop all the data.

    • verbose name: Delay Streaming Packets
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

LASSORegression

Estimate a continuous output value from features using LASSO Regression.

LASSO regression is a straightfoward and principled statistical technique to learn a linear mapping between input data and desired output values from training data. LASSO differs from ridge regression in that LASSO will identify and select a small number of relevant features, whereas ridge regression will usually weight every feature to some extent, however small. As a result, LASSO regression can be used when there is a very large number of irrelevant features in the data, as along as the relevant features are relatively few. This is also called sparsity regularization. If there are very few trials, or some extensive stretches of the data exhibit only one class, the procedure used to find the optimal regularization strength (cross-validation) can fail with an error that there were too few or no trials of a given class present. LASSO regression assumes that both inputs and outputs are Gaussian distributed, that is, have no or very few major statistical outliers. If the output follows a radically different distribution, for instance between 0 and 1, or nonnegative, or discrete values, then different methods may be more appropriate (for instance, classification methods for disrete values). To ameliorate the issue of outliers in the data, the raw data can be cleaned of artifacts with various artifact removal methods. To the extent that the assumptions hold true, this method is highly competitive with other (sparse) linear methods. There are several other nodes that implement similar methods that have advantages in specific circumstances. Like all machine learning methods, this method needs to be calibrated ("trained") before it can make any predictions on data. For this, the method requires training instances and associated training labels. The typical way to get such labels associated with time-series data is to make sure that a marker stream is included in the data, which is usually imported together with the data using one of the Import nodes, or received over the network alongside with the data, e.g., using the LSL Input node (with a non-empty marker query). These markers are then annotated with target labels using the Assign Targets node. To generate instances of training data for each of the training markers, one usually uses the Segmentation node to extract segments from the continuous time series around each marker. Since this machine learning method is not capable of being trained incrementally on streaming data, the method requires a data packet that contains the entire training data; this training data packet can either be accumulated online and then released in one shot using the Accumulate Calibration Data node, or it can be imported from a separate calibration recording and then spliced into the processing pipeline using the Inject Calibration Data, where it passes through the same nodes as the regular data until it reaches the machine learning node, where it is used for calibration. Once this node is calibrated, the trainable state of this node can be saved to a model file and later loaded for continued use.

More Info...

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • num_alphas
    Number of values in regularization search grid. This method determines the optimal regularization strength by testing a number of different strength values between the minimum and maximum value. The running time increases with a finer search grid, but the found solution may be slightly better regularized when using a fine grid (e.g., 100 values) instead of a coarse grid (e.g., 20 values).

    • verbose name: Number Of Values In Reg. Search Grid
    • default value: 100
    • port type: IntPort
    • value type: int (can be None)
  • num_folds
    Number of cross-validation folds for parameter search. Cross-validation proceeds by splitting up the data into this many blocks of trials, and then tests the method on each block. For each fold, the method is re-trained on all the other blocks, excluding the test block (therefore, the total running time is proportional to the number of folds). This is not a randomized cross-validation, but a blockwise cross-validation, which is usually the correct choice if the data stem from a time series. If there are few trials in the data, one can use a higher number here (e.g., 10) to ensure that more data is available for training.

    • verbose name: Number Of Cross-Validation Folds
    • default value: 5
    • port type: IntPort
    • value type: int (can be None)
  • max_iter
    Maximum number of iterations. This is one of the stopping criteria to limit the compute time. The default is usually fine, and gains from increasing the number of iterations will be minimal (it can be worth experimenting with lower iteration numbers if the algorithm must finish in a fixed time budget, at a cost of potentially less accurate solutions).

    • verbose name: Maximum Number Of Iterations
    • default value: 1000
    • port type: IntPort
    • value type: int (can be None)
  • tolerance
    Convergence tolerance. This is the desired errors tolerance or acceptable inaccuracy in the solution. Using larger values gives less accurate results, but will lead to faster compute times. Note that, for biosignal-driven machine learning systems, one often does not need very small tolerances.

    • verbose name: Tolerance
    • default value: 0.0001
    • port type: FloatPort
    • value type: float (can be None)
  • num_jobs
    Number of parallel compute jobs. This value only affects the running time and not the results. Values between 1 and twice the number of CPU cores make sense to expedite computation, but may temporarily reduce the responsiveness of the machine. The value of -1 stands for all available CPU cores.

    • verbose name: Number Of Parallel Jobs
    • default value: 1
    • port type: IntPort
    • value type: int (can be None)
  • verbosity
    Verbosity level. Higher numbers will produce more extensive diagnostic output.

    • verbose name: Verbosity Level
    • default value: 0
    • port type: IntPort
    • value type: int (can be None)
  • include_bias
    Include a bias term. If false, your features need to be centered, or include a dummy feature set to 1.

    • verbose name: Include Bias Term
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • normalize_features
    Normalize features. Should only be disabled if the data comes with a predictable scale (e.g., normalized in some other way).

    • verbose name: Normalize Features
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • min_alpha
    Minimum regularization strength. This is expressed as a factor of the maximum regularization strength, which is calculated from the data. By default, the optimal strength will be searched between this value and the maximum.

    • verbose name: Minimum Regularization Strength
    • default value: 0.001
    • port type: FloatPort
    • value type: float (can be None)
  • alphas
    Override regularization strength. Optionally the default regularization search grid can be overridden by giving an explicit list of values here, although this is rarely necessary or very helpful. This is a list of candidate values, the best of which is found via an exhaustive search (i.e., each value is tested one by one, and therefore the total running time is proportional to the number of values listed here). The details of the parameter search can be controlled via the search metric and number of folds parameters. Larger values cause stronger regularization, that is, less risk of the method over-fitting to random details of the data, and thus better generalization to new data. A value of 0 means no regularization, and there is no upper limit to how large the values that can be given here may be -- however, depending on the scale of the data and the number of trials, there is a cutoff beyond which all features are weighted by zero, and are thus unused. Often one covers a range between 0.1 and 10, and at times 0.01 to 100. Typically the values here are not linearly spaced, but follow an exponential progression (e.g., 0.25, 0.5, 1, 2, 4, 8, ... etc). The default search range is intentionally coarse for quick running times; refine it to smaller steps to obtain potentially better solutions, but do not expect massive gains from refining.

    • verbose name: Override Regularization Strength
    • default value: None
    • port type: ListPort
    • value type: list (can be None)
  • selection
    Parameter update schedule. Using random here can be significantly faster when higher tolerance values are used.

    • verbose name: Parameter Update Schedule
    • default value: cyclic
    • port type: EnumPort
    • value type: object (can be None)
  • positivity_constraint
    Constrain weights to be positive. This is a special (and rarely-used) flavor of this method, in which the learned weights are constrained to be positive.

    • verbose name: Constrain Weights To Be Positive
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • precompute
    Precompute shared data. Precompute some shared data that is reused during parameter search. Aside from 'auto', this can be set to True or False. Auto attempts to determine the best choice automatically.

    • verbose name: Precompute Shared Data
    • default value: auto
    • port type: Port
    • value type: object (can be None)
  • random_seed
    Random seed. Different values may give slightly different outcomes.

    • verbose name: Random Seed
    • default value: 12345
    • port type: IntPort
    • value type: int (can be None)
  • initialize_once
    Calibrate the model only once. If set to False, then this node will recalibrate itself whenever a non-streaming data chunk is received that has both training labels and associated training instances.

    • verbose name: Calibrate Only Once
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • dont_reset_model
    Do not reset the model when the preceding graph is changed. Normally, when certain parameters of preceding nodes are being changed, the model will be reset. If this is enabled, the model will persist, but there is a chance that the model is incompatible when input data format to this node has changed.

    • verbose name: Do Not Reset Model
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

LarsRegression

Estimate a continuous output value from features using Least-Angle (LARS) Regression.

LARS regression is a straightfoward and principled statistical technique to learn a linear mapping between input data and desired output values from training data. LARS is very similar to LASSO, but tends to be faster when the number of features is much higher than the number of data points. LASSO regression can be used when there is a very large number of irrelevant features in the data, as along as the relevant features are relatively few. This is also called sparsity regularization. If there are very few trials, or some extensive stretches of the data exhibit only one class, the procedure used to find the optimal regularization strength (cross-validation) can fail with an error that there were too few or no trials of a given class present. LARS regression assumes that both inputs and outputs are Gaussian distributed, that is, have no or very few major statistical outliers. If the output follows a radically different distribution, for instance between 0 and 1, or nonnegative, or discrete values, then different methods may be more appropriate (for instance, classification methods for disrete values). To ameliorate the issue of outliers in the data, the raw data can be cleaned of artifacts with various artifact removal methods. To the extent that the assumptions hold true, this method is highly competitive with other (sparse) linear methods. There are several other nodes that implement similar methods that have advantages in specific circumstances. Like all machine learning methods, this method needs to be calibrated ("trained") before it can make any predictions on data. For this, the method requires training instances and associated training labels. The typical way to get such labels associated with time-series data is to make sure that a marker stream is included in the data, which is usually imported together with the data using one of the Import nodes, or received over the network alongside with the data, e.g., using the LSL Input node (with a non-empty marker query). These markers are then annotated with target labels using the Assign Targets node. To generate instances of training data for each of the training markers, one usually uses the Segmentation node to extract segments from the continuous time series around each marker. Since this machine learning method is not capable of being trained incrementally on streaming data, the method requires a data packet that contains the entire training data; this training data packet can either be accumulated online and then released in one shot using the Accumulate Calibration Data node, or it can be imported from a separate calibration recording and then spliced into the processing pipeline using the Inject Calibration Data, where it passes through the same nodes as the regular data until it reaches the machine learning node, where it is used for calibration. Once this node is calibrated, the trainable state of this node can be saved to a model file and later loaded for continued use.

More Info...

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • num_folds
    Number of cross-validation folds for parameter search. Cross-validation proceeds by splitting up the data into this many blocks of trials, and then tests the method on each block. For each fold, the method is re-trained on all the other blocks, excluding the test block (therefore, the total running time is proportional to the number of folds). This is not a randomized cross-validation, but a blockwise cross-validation, which is usually the correct choice if the data stem from a time series. If there are few trials in the data, one can use a higher number here (e.g., 10) to ensure that more data is available for training.

    • verbose name: Number Of Cross-Validation Folds
    • default value: 5
    • port type: IntPort
    • value type: int (can be None)
  • num_alphas
    Number of values in regularization search grid. This method determines the optimal regularization strength by testing a number of different strength values between the minimum and maximum value. The running time increases with a finer search grid, but the found solution may be slightly better regularized when using a fine grid (e.g., 100 values) instead of a coarse grid (e.g., 20 values).

    • verbose name: Number Of Values In Reg. Search Grid
    • default value: 100
    • port type: IntPort
    • value type: int (can be None)
  • max_iter
    Maximum number of iterations. This is one of the stopping criteria to limit the compute time. The default is usually fine, and gains from increasing the number of iterations will be minimal (it can be worth experimenting with lower iteration numbers if the algorithm must finish in a fixed time budget, at a cost of potentially less accurate solutions).

    • verbose name: Maximum Number Of Iterations
    • default value: 500
    • port type: IntPort
    • value type: int (can be None)
  • num_jobs
    Number of parallel compute jobs. This value only affects the running time and not the results. Values between 1 and twice the number of CPU cores make sense to expedite computation, but may temporarily reduce the responsiveness of the machine. The value of -1 stands for all available CPU cores.

    • verbose name: Number Of Parallel Jobs
    • default value: 1
    • port type: IntPort
    • value type: int (can be None)
  • verbosity
    Verbosity level. Higher numbers will produce more extensive diagnostic output.

    • verbose name: Verbosity Level
    • default value: 0
    • port type: IntPort
    • value type: int (can be None)
  • normalize_features
    Normalize features. Should only be disabled if the data comes with a predictable scale (e.g., normalized in some other way).

    • verbose name: Normalize Features
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • include_bias
    Include a bias term. If false, your features need to be centered, or include a dummy feature set to 1.

    • verbose name: Include Bias Term
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • precompute
    Precompute shared data. Precompute some shared data that is reused during parameter search. Aside from 'auto', this can be set to True or False. Auto attempts to determine the best choice automatically.

    • verbose name: Precompute Shared Data
    • default value: auto
    • port type: Port
    • value type: object (can be None)
  • epsilon
    Degeneracy regularization. This parameter can be used to ensure that the underlying computation does not fail with singular results. Can be increased in cases where the number of features is very high compared to the number of observations.

    • verbose name: Epsilon
    • default value: 2.220446049250313e-16
    • port type: FloatPort
    • value type: float (can be None)
  • initialize_once
    Calibrate the model only once. If set to False, then this node will recalibrate itself whenever a non-streaming data chunk is received that has both training labels and associated training instances.

    • verbose name: Calibrate Only Once
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • dont_reset_model
    Do not reset the model when the preceding graph is changed. Normally, when certain parameters of preceding nodes are being changed, the model will be reset. If this is enabled, the model will persist, but there is a chance that the model is incompatible when input data format to this node has changed.

    • verbose name: Do Not Reset Model
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

LinearDiscriminantAnalysis

Classify data instances using Linear Discriminant Analysis (LDA).

The LDA method is a fast statistical method that learns a linear mapping from input data to discrete category labels. LDA assumes that the data are Gaussian-distributed, that is, have no or very few major statistical outliers. To the extent that these assumptions hold true, this method is highly competitive with other linear (or generalized linear) methods. To ameliorate the outlier issue, the raw data can be cleaned of artifacts with various artifact removal methods. This implementation uses shrinkage regularization by default, which allows it to handle large numbers of features quite gracefully, compared to the unregularized variant, which will overfit or fail on more than a few dozen features. This method can be implemented using a number of different numerical approaches which have different tradeoffs -- it is worth experimenting with different choices of the solver parameter if you cannot get the result that you seek. By default, this method will return not the most likely class label for each trial it is given, but instead the probabilies for each class (=category of labels), that the trial is actually of that class. This method can also optionally use a regularization parameter that is tuned using an internal cross-validation on the data. If there are very few trials, or some extensive stretches of the data exhibit only one class, this cross-validation can fail with an error that there were too few or no trials of a given class present. Like all machine learning methods, this method needs to be calibrated ("trained") before it can make any predictions on data. For this, the method requires training instances and associated training labels. The typical way to get such labels associated with time-series data is to make sure that a marker stream is included in the data, which is usually imported together with the data using one of the Import nodes, or received over the network alongside with the data, e.g., using the LSL Input node (with a non-empty marker query). These markers are then annotated with target labels using the Assign Targets node. To generate instances of training data for each of the training markers, one usually uses the Segmentation node to extract segments from the continuous time series around each marker. Since this machine learning method is not capable of being trained incrementally on streaming data, the method requires a data packet that contains the entire training data; this training data packet can either be accumulated online and then released in one shot using the Accumulate Calibration Data node, or it can be imported from a separate calibration recording and then spliced into the processing pipeline using the Inject Calibration Data, where it passes through the same nodes as the regular data until it reaches the machine learning node, where it is used for calibration. Once this node is calibrated, the trainable state of this node can be saved to a model file and later loaded for continued use.

More Info...

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • shrinkage
    Regularization strength. If using 'auto', then a fast automatic method is used to determine the regularization parameter (using the Ledoit-Wolf method). However, if given as a list of numbers between 0 and 1 (formatted as in, e.g., [0.1, 0.2, 0.3]) where 0 is no regularization and 1 is maximal regularization, then the best parameter is searched, which can be slow. The details of the parameter search can be controlled via the search metric and number of folds parameters.

    • verbose name: Regularization Strength
    • default value: auto
    • port type: Port
    • value type: object (can be None)
  • search_metric
    Parameter search metric. When the shrinkage parameter is given as a list of values, then the method will run a cross-validation for each possible parameter value and use this metric to score how well the method performs in each case, in order to select the best parameter. Therefore, when a search is done the running time of the method is multiplied by the number of parameter values and the number of folds in the cross-validation, which can be slow. While 'accuracy' is usually a good default, some other metrics can be useful under special circumstances, e.g., roc_auc for highly imbalanced ratios of trials from different classes.

    • verbose name: Scoring Metric If Searching Parameters
    • default value: accuracy
    • port type: EnumPort
    • value type: object (can be None)
  • num_folds
    Number of cross-validation folds for parameter search. Cross-validation proceeds by splitting up the data into this many blocks of trials, and then tests the method on each block. For each fold, the method is re-trained on all the other blocks, excluding the test block. This is not a randomized cross-validation, but a blockwise cross-validation, which is usually the correct choice if the data stem from a time series. If there are few trials in the data, one can use a higher number here (e.g., 10) to ensure that more data is available for training

    • verbose name: Number Of Cross-Validation Folds
    • default value: 5
    • port type: IntPort
    • value type: int (can be None)
  • probabilistic
    Use probabilistic outputs. If enabled, the node will output for each class the probability that a given trial is of that class; otherwise it will output the most likely class label.

    • verbose name: Output Probabilities
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • solver
    Solver to use. This node supports formulations based on a least-squares solution (lsqr), eigenvalue decomposition (eigen), and singular-value decomposition (svd). Some of these methods are known to have numerical issues under various circumstances -- consider trying different settings if you cannot achieve the desired results. Note: the svd method can handle many features, but does not support shrinkage-type regularization.

    • verbose name: Solver To Use
    • default value: eigen
    • port type: EnumPort
    • value type: object (can be None)
  • class_weights
    Per-class weights. Optionally this is a mapping from class label to weight. This is formatted as in, e.g., {0: 0.1, 1: 0.2, 2: 0.1}. Class weights are used fairly infrequently, except when the cost of false detections of different classes needs to be traded off.

    • verbose name: Per-Class Weight
    • default value: None
    • port type: Port
    • value type: object (can be None)
  • tolerance
    Threshold for rank estimation in SVD. Using a larger value will more aggressively prune features, but can make the difference between it working or not.

    • verbose name: Rank Estimation Threshold (If Svd)
    • default value: 0.0001
    • port type: FloatPort
    • value type: float (can be None)
  • initialize_once
    Calibrate the model only once. If set to False, then this node will recalibrate itself whenever a non-streaming data chunk is received that has both training labels and associated training instances.

    • verbose name: Calibrate Only Once
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • dont_reset_model
    Do not reset the model when the preceding graph is changed. Normally, when certain parameters of preceding nodes are being changed, the model will be reset. If this is enabled, the model will persist, but there is a chance that the model is incompatible when input data format to this node has changed.

    • verbose name: Do Not Reset Model
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • verbosity
    Verbosity level. Higher numbers will produce more extensive diagnostic output.

    • verbose name: Verbosity Level
    • default value: 0
    • port type: IntPort
    • value type: int (can be None)
  • cond_field
    The name of the instance data field that contains the conditions to be discriminated. This parameter will be ignored if the packet has previously been processed by a BakeDesignMatrix node.

    • verbose name: Cond Field
    • default value: TargetValue
    • port type: StringPort
    • value type: str (can be None)

LinearSupportVectorClassification

Use linear support vector machines to classify data instances.

This is the linear version of the support vector classification. As such, it will not be able to exploit non-linear structure, but on data that has little to no dominant non-linear features, it will perform just as well as non-linear ("kernel") SVMs, and do so in a computationally more efficient way. Linear SVMs are quite similar to logistic regression, and can be more robust in the presence of outliers (when using the default hinge loss), though on the other hand the probability estimates produced by SVM are not as theoretically straightforward and well-motivated as those of logistic regression. This implementation uses regularization by default, which allows it to handle large numbers of features very well. Importantly, there are two types of regularization that one can choose from: the default l2 regularization is a fine choice for most data (it is closely related to shrinkage in LDA or a Gaussian prior in Bayesian methods). The alternative l1 regularization is unique in that it can learn to identify a sparse subset of features that is relevant while pruning out all other features as irrelevant. This sparsity regularization is statistically very efficient and can deal with an extremely large number of irrelevant features. To determine the optimal regularization strength, a list of candidate parameter settings can be given, which is then searched by the method using an internal cross-validation on the data to find the best value. If there are very few trials, or some extensive stretches of the data exhibit only one class, this cross-validation can fail with an error that there were too few or no trials of a given class present. Also, the default search grid for regularization (i.e., the list of candidate values) is deliberately rather coarse to keep the method fast. For higher-quality results, use a more fine grained list of values (which will be correspondingly slower). This method can be implemented using a number of different numerical approaches which have different running times depending on the number of data points and features. If you are re-solving the problem a lot, it can make sense to try out the various solvers to find the fastest one. Like all machine learning methods, this method needs to be calibrated ("trained") before it can make any predictions on data. For this, the method requires training instances and associated training labels. The typical way to get such labels associated with time-series data is to make sure that a marker stream is included in the data, which is usually imported together with the data using one of the Import nodes, or received over the network alongside with the data, e.g., using the LSL Input node (with a non-empty marker query). These markers are then annotated with target labels using the Assign Targets node. To generate instances of training data for each of the training markers, one usually uses the Segmentation node to extract segments from the continuous time series around each marker. Since this machine learning method is not capable of being trained incrementally on streaming data, the method requires a data packet that contains the entire training data; this training data packet can either be accumulated online and then released in one shot using the Accumulate Calibration Data node, or it can be imported from a separate calibration recording and then spliced into the processing pipeline using the Inject Calibration Data, where it passes through the same nodes as the regular data until it reaches the machine learning node, where it is used for calibration. Once this node is calibrated, the trainable state of this node can be saved to a model file and later loaded for continued use.

More Info...

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • cost
    SVM cost parameter. This value determines the degree to which solutions are penalized that would mis-classify data points. Higher values result in models that are less likely to mis-classify the training data, but at the expense of potentially worse generalization to new data (less margin for error when slightly different trials are encountered in future test data). This is a list of candidate values, the best of which is found via an exhaustive search (i.e., each value is tested one by one, and therefore the total running time is proportional to the number of values listed here). The details of the parameter search can be controlled via the search metric and number of folds parameters. Larger values cause stronger regularization, that is, less risk of the method over-fitting to random details of the data, and thus better generalization to new data. A very small value means effectively no penalty, and there is no upper limit to how large the values that can be given here may be -- however, depending on the scale of the data and the number of trials. Often one covers a range between 0.1 and 10, and at times 0.01 to 100. Typically the values here are not linearly spaced, but follow an exponential progression (e.g., 0.25, 0.5, 1, 2, 4, 8, ... etc). The default search range is intentionally coarse for quick running times; refine it to smaller steps to obtain potentially better solutions, but do not expect massive gains from refining.

    • verbose name: Cost
    • default value: [0.01, 0.1, 1.0, 10.0, 100]
    • port type: Port
    • value type: list (can be None)
  • loss
    Loss function to use. This selects the data term, i.e., what assumptions are being imposed on the data. l1 is the hinge loss (standard SVM). l2 is the squared_hinge which is like hinge but is quadratically penalized.

    • verbose name: Data Term
    • default value: l2
    • port type: EnumPort
    • value type: object (can be None)
  • regularizer
    Regularization type. The default l2 regularization is a good choice for most data. The alternative l1 regularization results in a type of "feature selection", that is, only a sparse set of features will have a non-zero weight, and all other features will remain effectively unused. This sparsity regularization is very useful if only a small number of features are relevant. It is nevertheless a good idea to compare to the performance of using l2 regularization, since excessive sparsity can sometimes degrade performance.

    • verbose name: Regularization Type
    • default value: l2
    • port type: EnumPort
    • value type: object (can be None)
  • search_metric
    Parameter search metric. When the regularization parameter is given as a list of values, then the method will run a cross-validation for each possible parameter value and use this metric to score how well the method performs in each case, in order to select the best parameter. While 'accuracy' is usually a good default, some other metrics can be useful under special circumstances, e.g., roc_auc for highly imbalanced ratios of trials from different classes.

    • verbose name: Scoring Metric If Searching Parameters
    • default value: accuracy
    • port type: EnumPort
    • value type: object (can be None)
  • num_folds
    Number of cross-validation folds for parameter search. Cross-validation proceeds by splitting up the data into this many blocks of trials, and then tests the method on each block. For each fold, the method is re-trained on all the other blocks, excluding the test block (therefore, the total running time is proportional to the number of folds). This is not a randomized cross-validation, but a blockwise cross-validation, which is usually the correct choice if the data stem from a time series. If there are few trials in the data, one can use a higher number here (e.g., 10) to ensure that more data is available for training.

    • verbose name: Number Of Cross-Validation Folds
    • default value: 5
    • port type: IntPort
    • value type: int (can be None)
  • max_iter
    Maximum number of iterations. This is one of the stopping criteria to limit the compute time. The default is usually fine, and gains from increasing the number of iterations will be minimal (it can be worth experimenting with lower iteration numbers if the algorithm must finish in a fixed time budget, at a cost of potentially less accurate solutions).

    • verbose name: Maximum Number Of Iterations
    • default value: 1000
    • port type: IntPort
    • value type: int (can be None)
  • verbosity
    Verbosity level. Higher numbers will produce more extensive diagnostic output.

    • verbose name: Verbosity Level
    • default value: 0
    • port type: IntPort
    • value type: int (can be None)
  • class_weights
    Per-class weights. Optionally this is a mapping from class label to weight. This is formatted as in, e.g., {0: 0.1, 1: 0.2, 2: 0.1}. Alternatively, 'balanced' may be used which uses the values of y to automatically adjust weights inversely proportional to class frequencies in the input data as n_samples / (n_classes * np.bincount(y)). Class weights are used fairly infrequently, except when the cost of false detections of different classes needs to be traded off.

    • verbose name: Per-Class Weight
    • default value: None
    • port type: Port
    • value type: object (can be None)
  • dual_formulation
    Use dual formulation. This is an alternative way to solve the problem. If enabled, it can be faster when the number of trials is larger than the number of features, but it is not supported with l1 regularization.

    • verbose name: Use Alternative Dual Formulation
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • include_bias
    Include a bias term. If false, your features need to be centered, or include a dummy feature set to 1.

    • verbose name: Include Bias Term
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • bias_scaling
    Scale for bias term. Since this implementation applies the regularization to the bias term too (which is usually not ideal, although rarely a significant issue), you can use this scale to counter the effect.

    • verbose name: Bias Scaling
    • default value: 1.0
    • port type: FloatPort
    • value type: float (can be None)
  • random_seed
    Random seed. Different values may give slightly different outcomes.

    • verbose name: Random Seed
    • default value: 12345
    • port type: IntPort
    • value type: int (can be None)
  • tolerance
    Convergence tolerance. This is the desired errors tolerance or acceptable inaccuracy in the solution. Using larger values gives less accurate results, but will lead to faster compute times. Note that, for biosignal-driven machine learning systems, one often does not need very small tolerances.

    • verbose name: Tolerance
    • default value: 0.0001
    • port type: FloatPort
    • value type: float (can be None)
  • initialize_once
    Calibrate the model only once. If set to False, then this node will recalibrate itself whenever a non-streaming data chunk is received that has both training labels and associated training instances.

    • verbose name: Calibrate Only Once
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • dont_reset_model
    Do not reset the model when the preceding graph is changed. Normally, when certain parameters of preceding nodes are being changed, the model will be reset. If this is enabled, the model will persist, but there is a chance that the model is incompatible when input data format to this node has changed.

    • verbose name: Do Not Reset Model
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

LogisticRegression

Classify data instances using regularized Logistic Regression.

The logistic regression method is a versatile and principled statistical method that learns a generalized linear mapping from input data to the probability of that data belonging to one of several possible classes. This method is highly competitive with other linear or generalized linear methods, such as LDA and SVM. Logistic regression makes relatively weak assumptions on the distribution of the data, so that it is moderately tolerant to outliers, especially when compared to relatively brittle methods like LDA. Logistic regression is not considered to be quite as robust as SVM (therefore it can make sense to remove artifacts beforehand using the appropriate nodes if the data are very noisy) -- however, the main benefit of logistic regression over SVM is that the outputs can be straightforwardly interpreted as probabilities without having to resort to 'tricks' such as probability calibration. This implementation uses regularization by default, which allows it to handle large numbers of features very well. Importantly, there are two types of regularization that one can choose from: the default l2 regularization is a fine choice for most data (it is closely related to shrinkage in LDA or a Gaussian prior in Bayesian methods). The alternative l1 regularization is unique in that it can learn to identify a sparse subset of features that is relevant while pruning out all other features as irrelevant. This sparsity regularization is statistically very efficient and can deal with an extremely large number of irrelevant features. To determine the optimal regularization strength, the a list of candidate parameter settings can be given, which is then searched by the method using an internal cross-validation on the data to find the best value. If there are very few trials, or some extensive stretches of the data exhibit only one class, this cross-validation can fail with an error that there were too few or no trials of a given class present. Also, the default search grid for regularization (i.e., the list of candidate values) is deliberately rather coarse to keep the method fast. For higher-quality results, use a more fine grained list of values (which will be correspondingly slower). There are also several other implementations of logistic regression with different regularization terms or different performance characteristics in other nodes. This method can be implemented using a number of different numerical approaches which have different running times depending on the number of data points and features. If you are re-solving the problem a lot, it can make sense to try out the various solvers to find the fastest one. By default, this method will return not the most likely class label for each trial it is given, but instead the probabilies for each class (=category of labels), that the trial is actually of that class. Like all machine learning methods, this method needs to be calibrated ("trained") before it can make any predictions on data. For this, the method requires training instances and associated training labels. The typical way to get such labels associated with time-series data is to make sure that a marker stream is included in the data, which is usually imported together with the data using one of the Import nodes, or received over the network alongside with the data, e.g., using the LSL Input node (with a non-empty marker query). These markers are then annotated with target labels using the Assign Targets node. To generate instances of training data for each of the training markers, one usually uses the Segmentation node to extract segments from the continuous time series around each marker. Since this machine learning method is not capable of being trained incrementally on streaming data, the method requires a data packet that contains the entire training data; this training data packet can either be accumulated online and then released in one shot using the Accumulate Calibration Data node, or it can be imported from a separate calibration recording and then spliced into the processing pipeline using the Inject Calibration Data, where it passes through the same nodes as the regular data until it reaches the machine learning node, where it is used for calibration. Once this node is calibrated, the trainable state of this node can be saved to a model file and later loaded for continued use.

More Info...

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • regularizer
    Regularization type. The default l2 regularization is a good choice for most data. The alternative l1 regularization results in a type of "feature selection", that is, only a sparse set of features will have a non-zero weight, and all other features will remain effectively unused. This sparsity regularization is very useful if only a small number of features are relevant. It is nevertheless a good idea to compare to the performance of using l2 regularization, since excessive sparsity can sometimes degrade performance. If l1 is selected, the liblinear solver is automatically used, so it is of no use to adjust the solver.

    • verbose name: Regularization Type
    • default value: l2
    • port type: EnumPort
    • value type: object (can be None)
  • alphas
    Regularization strength. This is a list of candidate values, the best of which is found via an exhaustive search (i.e., each value is tested one by one, and therefore the total running time is proportional to the number of values listed here). The details of the parameter search can be controlled via the search metric and number of folds parameters. Larger values cause stronger regularization, that is, less risk of the method over-fitting to random details of the data, and thus better generalization to new data. A value of 0 means no regularization, and there is no upper limit to how large the values that can be given here may be -- however, depending on the scale of the data and the number of trials, there is a cutoff beyond which all features are weighted by zero, and are thus unused. Often one covers a range between 0.1 and 10, and at times 0.01 to 100. Typically the values here are not linearly spaced, but follow an exponential progression (e.g., 0.25, 0.5, 1, 2, 4, 8, ... etc). The default search range is intentionally coarse for quick running times; refine it to smaller steps to obtain potentially better solutions, but do not expect massive gains from refining.

    • verbose name: Regularization Strength
    • default value: [0.1, 0.5, 1.0, 5, 10.0]
    • port type: ListPort
    • value type: list (can be None)
  • search_metric
    Parameter search metric. When the regularization parameter is given as a list of values, then the method will run a cross-validation for each possible parameter value and use this metric to score how well the method performs in each case, in order to select the best parameter. While 'accuracy' is usually a good default, some other metrics can be useful under special circumstances, e.g., roc_auc for highly imbalanced ratios of trials from different classes.

    • verbose name: Scoring Metric If Searching Parameters
    • default value: accuracy
    • port type: EnumPort
    • value type: object (can be None)
  • num_folds
    Number of cross-validation folds for parameter search. Cross-validation proceeds by splitting up the data into this many blocks of trials, and then tests the method on each block. For each fold, the method is re-trained on all the other blocks, excluding the test block (therefore, the total running time is proportional to the number of folds). This is not a randomized cross-validation, but a blockwise cross-validation, which is usually the correct choice if the data stem from a time series. If there are few trials in the data, one can use a higher number here (e.g., 10) to ensure that more data is available for training.

    • verbose name: Number Of Cross-Validation Folds
    • default value: 5
    • port type: IntPort
    • value type: int (can be None)
  • num_jobs
    Number of parallel compute jobs. This value only affects the running time and not the results. Values between 1 and twice the number of CPU cores make sense to expedite computation, but may temporarily reduce the responsiveness of the machine. The value of -1 stands for all available CPU cores.

    • verbose name: Number Of Parallel Jobs
    • default value: 1
    • port type: IntPort
    • value type: int (can be None)
  • probabilistic
    Use probabilistic outputs. If enabled, the node will output for each class the probability that a given trial is of that class; otherwise it will output the most likely class label.

    • verbose name: Output Probabilities
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • verbosity
    Verbosity level. Higher numbers will produce more extensive diagnostic output.

    • verbose name: Verbosity Level
    • default value: 0
    • port type: IntPort
    • value type: int (can be None)
  • class_weights
    Per-class weights. Optionally this is a mapping from class label to weight. This is formatted as in, e.g., {0: 0.1, 1: 0.2, 2: 0.1}. Class weights are used fairly infrequently, except when the cost of false detections of different classes needs to be traded off.

    • verbose name: Per-Class Weight
    • default value: None
    • port type: Port
    • value type: object (can be None)
  • solver
    Solver to use. Different algorithms have different running times depending on the number of features, and the number of trials. It can make sense to identify the fastest method if this node is used a lot on similarly-sized data. When l1 regularization is used, the liblinear solver will be automatically picked.

    • verbose name: Solver To Use
    • default value: lbfgs
    • port type: EnumPort
    • value type: object (can be None)
  • dual_formulation
    Use dual formulation. This is an alternative way to solve the problem. If enabled, it can be faster when the number of trials is larger than the number of features, but it is not supported with l1 regularization.

    • verbose name: Use Alternative Dual Formulation
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • multiclass
    Technique to use when classifying more than two classes of data. These formulations will usually have little impact on the results, but multinomial can be solved in one step, whereas ovr (which stands for one-vs-rest) requires one run per class, which can be slower. Multinomial is only supported when using the lbfgs solver.

    • verbose name: Type Of Multiclass Approach
    • default value: ovr
    • port type: EnumPort
    • value type: object (can be None)
  • max_iter
    Maximum number of iterations. This is one of the stopping criteria to limit the compute time. The default is usually fine, and gains from increasing the number of iterations will be minimal (it can be worth experimenting with lower iteration numbers if the algorithm must finish in a fixed time budget, at a cost of potentially less accurate solutions).

    • verbose name: Maximum Number Of Iterations
    • default value: 100
    • port type: IntPort
    • value type: int (can be None)
  • include_bias
    Include a bias term. If false, your features need to be centered, or include a dummy feature set to 1.

    • verbose name: Include Bias Term
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • bias_scaling
    Scale for bias term. Since this logistic regression implementation applies the regularization to the bias term too (which is usually not ideal, although rarely a significant issue), you can use this scale to counter the effect.

    • verbose name: Bias Scaling
    • default value: 1.0
    • port type: FloatPort
    • value type: float (can be None)
  • tolerance
    Convergence tolerance. This is the desired errors tolerance or acceptable inaccuracy in the solution. Using larger values gives less accurate results, but will lead to faster compute times. Note that, for biosignal-driven machine learning systems, one often does not need very small tolerances.

    • verbose name: Tolerance
    • default value: 0.0001
    • port type: FloatPort
    • value type: float (can be None)
  • initialize_once
    Calibrate the model only once. If set to False, then this node will recalibrate itself whenever a non-streaming data chunk is received that has both training labels and associated training instances.

    • verbose name: Calibrate Only Once
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • dont_reset_model
    Do not reset the model when the preceding graph is changed. Normally, when certain parameters of preceding nodes are being changed, the model will be reset. If this is enabled, the model will persist, but there is a chance that the model is incompatible when input data format to this node has changed.

    • verbose name: Do Not Reset Model
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • cond_field
    The name of the instance data field that contains the conditions to be discriminated. This parameter will be ignored if the packet has previously been processed by a BakeDesignMatrix node.

    • verbose name: Cond Field
    • default value: TargetValue
    • port type: StringPort
    • value type: str (can be None)

MeasureLoss

Measure discrepancy between some predicted labels and true labels.

This node accepts as input some data that has both estimates or predictions (e.g., from a machine learning node), as well as ground-truth labels (desired outputs). Using these inputs, the node computes a loss function that measures the disagreement between the predictions and the labels. The node supports a large variety of losses that can be used in different circumstances (based on different assumptions about the type of estimates or the severity of different kinds of errors), such as misclassification rate, mean squared error, or area under receiver-operator curve. Be sure to inform yourself about what is the most appropriate loss metric for your use case before trusting the numbers blindly, or e.g., accidentally relying on misleading results (e.g., inapplicable metrics) to make decisions. The node is capable of accumulating results across multiple trials or data chunks, for instance during real-time processing. It is also possible to do the same across multiple datasets in offline or batch processing, by setting the appropriate parameters in the node. If 'output for stats table' is selected, the output of this node contains two Feature axes: a singleton axis containing the name of the selected measure, labeled 'explanatory-values', and a second one containing the value ('Value') and the number of trials ('N Trials'), labeled 'statistic-types'. This is so that it can be followed with a ParseStatsTable node. Otherwise, this node outputs a single Feature axis containing the name of the selected measure, and the number of trials.

More Info...

Version 1.0.2

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • loss
    Loss estimates.

    • verbose name: Loss
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: OUT
  • loss_metric
    Loss metric. The following measures can be calculated both offline and online: MCR is misclassification rate (aka error rate), MSE is mean-squared error, MAE is the mean absolute error, Sign is the sign error, Bias is the prediction bias. The following measures are currently only available during offline computations: SMAE is the standardized mean absolute error, SMSE is the standardized mean-squared error, max is the maximum error, RMSE is the root mean squared error, MedSE is the median squared error. MedAE is the median absolute error, SMedAE is the standardized median absolute error, AUC is negative area under ROC curve, R2 is the R-squared loss, CrossEnt is the cross-entropy loss, ExpVar is the negative explained variance, CM is the confusion matrix.

    • verbose name: Loss Metric
    • default value: MCR
    • port type: EnumPort
    • value type: object (can be None)
  • accumulate_offline
    Accumulate statistics across offline data packets, as well. Normally this node will only accumulate statistics on real-time (streaming) data. By enabling this, it will also accumulate across multiple non-streaming data sets, e.g., in batch processing.

    • verbose name: Accumulate Offline
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • ignore_resets
    Keep accumulating throughout state resets of the pipeline. For instance, after rewiring or changing files. This is most relevant in batch processing, where a pipeline iterates over multiple files. If you intend to aggregate statistics across multiple files, you need to enable this option.

    • verbose name: Ignore Resets
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • cond_field
    The name of the instance data field that contains the true target values. This parameter will be ignored if the packet has previously been processed by a BakeDesignMatrix node.

    • verbose name: Cond Field
    • default value: TargetValue
    • port type: StringPort
    • value type: str (can be None)
  • output_for_stats_table
    Output the data in such a way that it can be parsed by the ParseStatsTable node, for creating stats tables. This creates two feature axes instead of one.

    • verbose name: Output For Stats Table
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

NaiveBayes

Use naive Bayes to classify data instances.

This is a very basic classifier that can work well in situations where one would also use LDA or QDA. This method can handle large numbers of features without problems since it does not account for correlations between those features. Like all machine learning methods, this method needs to be calibrated ("trained") before it can make any predictions on data. For this, the method requires training instances and associated training labels. The typical way to get such labels associated with time-series data is to make sure that a marker stream is included in the data, which is usually imported together with the data using one of the Import nodes, or received over the network alongside with the data, e.g., using the LSL Input node (with a non-empty marker query). These markers are then annotated with target labels using the Assign Targets node. To generate instances of training data for each of the training markers, one usually uses the Segmentation node to extract segments from the continuous time series around each marker. Since this machine learning method is not capable of being trained incrementally on streaming data, the method requires a data packet that contains the entire training data; this training data packet can either be accumulated online and then released in one shot using the Accumulate Calibration Data node, or it can be imported from a separate calibration recording and then spliced into the processing pipeline using the Inject Calibration Data, where it passes through the same nodes as the regular data until it reaches the machine learning node, where it is used for calibration. Once this node is calibrated, the trainable state of this node can be saved to a model file and later loaded for continued use.

More Info...

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • probabilistic
    Use probabilistic outputs. If enabled, the node will output for each class the probability that a given trial is of that class; otherwise it will output the most likely class label.

    • verbose name: Output Probabilities
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • initialize_once
    Calibrate the model only once. If set to False, then this node will recalibrate itself whenever a non-streaming data chunk is received that has both training labels and associated training instances.

    • verbose name: Calibrate Only Once
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • dont_reset_model
    Do not reset the model when the preceding graph is changed. Normally, when certain parameters of preceding nodes are being changed, the model will be reset. If this is enabled, the model will persist, but there is a chance that the model is incompatible when input data format to this node has changed.

    • verbose name: Do Not Reset Model
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

QuadraticDiscriminantAnalysis

Classify data instances using Quadratic Discriminant Analysis (QDA).

The QDA method is a relatively simple statistical method that learns a quadratic mapping (i.e., a basic non-linear mapping) from input data to discrete category labels. QDA assumes that the data are Gaussian-distributed, that is, have no or very few major statistical outliers. To the extent that these assumptions are violated, this method will likely underperform. To ameliorate the outlier issue, the raw data can be cleaned of artifacts with various artifact removal methods. This implementation uses shrinkage regularization by default, which allows it to handle large numbers of features quite gracefully, compared to the unregularized variant, which will overfit or fail on more than a few dozen features. This method relies on a regularization parameter that is tuned using an internal cross-validation on the data. If there are very few trials, or some extensive stretches of the data exhibit only one class, this cross-validation can fail with an error that there were too few or no trials of a given class present. By default, this method will return not the most likely class label for each trial it is given, but instead the probabilies for each class (=category of labels), that the trial is actually of that class. Like all machine learning methods, this method needs to be calibrated ("trained") before it can make any predictions on data. For this, the method requires training instances and associated training labels. The typical way to get such labels associated with time-series data is to make sure that a marker stream is included in the data, which is usually imported together with the data using one of the Import nodes, or received over the network alongside with the data, e.g., using the LSL Input node (with a non-empty marker query). These markers are then annotated with target labels using the Assign Targets node. To generate instances of training data for each of the training markers, one usually uses the Segmentation node to extract segments from the continuous time series around each marker. Since this machine learning method is not capable of being trained incrementally on streaming data, the method requires a data packet that contains the entire training data; this training data packet can either be accumulated online and then released in one shot using the Accumulate Calibration Data node, or it can be imported from a separate calibration recording and then spliced into the processing pipeline using the Inject Calibration Data, where it passes through the same nodes as the regular data until it reaches the machine learning node, where it is used for calibration. Once this node is calibrated, the trainable state of this node can be saved to a model file and later loaded for continued use.

More Info...

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • shrinkage
    Regularization strength. Given as a list of numbers between 0 and 1 where 0 is no regularization and 1 is maximal regularization. The best parameter is searched, which can be slow. The details of the parameter search can be controlled via the search metric and number of folds parameters.

    • verbose name: Regularization Strength
    • default value: [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.8]
    • port type: ListPort
    • value type: list (can be None)
  • search_metric
    Parameter search metric. This method will run a cross-validation for each possible shrinkage parameter value and use this metric to score how well the method performs in each case, in order to select the best parameter. Therefore, the running time of the method is proportional to the number of parameter values times the number of folds in the cross-validation, which can be slow. While 'accuracy' is usually a good default, some other metrics can be useful under special circumstances, e.g., roc_auc for highly imbalanced ratios of trials from different classes.

    • verbose name: Scoring Metric For Parameter Search
    • default value: accuracy
    • port type: EnumPort
    • value type: object (can be None)
  • num_folds
    Number of cross-validation folds for parameter search. Cross-validation proceeds by splitting up the data into this many blocks of trials, and then tests the method on each block. For each fold, the method is re-trained on all the other blocks, excluding the test block. This is not a randomized cross-validation, but a blockwise cross-validation, which is usually the correct choice if the data stem from a time series. If there are few trials in the data, one can use a higher number here (e.g., 10) to ensure that more data is available for training

    • verbose name: Number Of Cross-Validation Folds
    • default value: 5
    • port type: IntPort
    • value type: int (can be None)
  • probabilistic
    Use probabilistic outputs. If enabled, the node will output for each class the probability that a given trial is of that class; otherwise it will output the most likely class label.

    • verbose name: Output Probabilities
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • class_weights
    Per-class weights. Optionally this is a mapping from class label to weight. This is formatted as in, e.g., {0: 0.1, 1: 0.2, 2: 0.1}. Class weights are used fairly infrequently, except when the cost of false detections of different classes needs to be traded off.

    • verbose name: Per-Class Weight
    • default value: None
    • port type: Port
    • value type: object (can be None)
  • initialize_once
    Calibrate the model only once. If set to False, then this node will recalibrate itself whenever a non-streaming data chunk is received that has both training labels and associated training instances.

    • verbose name: Calibrate Only Once
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • verbosity
    Verbosity level. Higher numbers will produce more extensive diagnostic output.

    • verbose name: Verbosity Level
    • default value: 0
    • port type: IntPort
    • value type: int (can be None)
  • dont_reset_model
    Do not reset the model when the preceding graph is changed. Normally, when certain parameters of preceding nodes are being changed, the model will be reset. If this is enabled, the model will persist, but there is a chance that the model is incompatible when input data format to this node has changed.

    • verbose name: Do Not Reset Model
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

RegularizedLogisticRegression

Classify data instances using regularized Logistic Regression with complex regularization terms.

The logistic regression method is a versatile and principled statistical method that learns a generalized linear mapping from input data to the probability of that data belonging to one of several possible classes. This method is highly competitive with other linear or generalized linear methods, such as LDA and SVM. In comparison to the simple logistic regression node, the emphasis of this implementation is on the support for complex regularization terms, in particular group sparsity and trace-norm regularization. When these features are not needed, it is recommended to consider using the simple node instead, as the training time may be shorter, and the parameter settings may allow for more timely or robust convergence on a wide range of data. Logistic regression makes relatively weak assumptions on the distribution of the data, so that it is moderately tolerant to outliers, especially when compared to relatively brittle methods like LDA. Logistic regression is not considered to be quite as robust as SVM (therefore it can make sense to remove artifacts beforehand using the appropriate nodes if the data are very noisy) -- however, the main benefit of logistic regression over SVM is that the outputs can be straightforwardly interpreted as probabilities without having to resort to 'tricks' such as probability calibration. This implementation uses regularization by default, which allows it to handle large numbers of features very well. This implementation supports several types of regularization, including l1, l2, l1/l2, and trace. The basic l2 regularization is a fine choice for most data (it is closely related to shrinkage in LDA or a Gaussian prior in Bayesian methods). The alternative l1 regularization is unique in that it can learn to identify a sparse subset of features that is relevant while pruning out all other features as irrelevant. This sparsity regularization is statistically very efficient and can deal with an extremely large number of irrelevant features. The l1/l2 regularization is a combination of the two, which can identify groups of features that are irrelevant (also known as group sparsity). In this implementation, sparsity acts on the features as if they formed a matrix, and will prune entire rows or columns of the matrix. In order to be able to do this, the input data tensor should have more than one axis that can be used as features, and these axes need to be identified via the group_axis parameter. The trace regularization mode also views the features as a matrix, but it will learn a weighting such that the corresponding matrix of weights per feature is low rank. As such, it will learn a weighting that can be decomposed into a sum of a small number of pairs of weight profiles for the two matrix axes, e.g., spatial and temporal profiles if features were space by time, which are multiplied together to each form a rank-1 matrix. To determine the optimal regularization strength, the a list of candidate parameter settings can be given, which is then searched by the method using an internal cross-validation on the data to find the best value. If there are very few trials, or some extensive stretches of the data exhibit only one class, this cross-validation can fail with an error that there were too few or no trials of a given class present. Also, the default search grid for regularization (i.e., the list of candidate values) is deliberately rather coarse to keep the method fast. For higher-quality results, use a more fine grained list of values (which will be correspondingly slower). There are also several other implementations of logistic regression with different regularization terms or different performance characteristics in other nodes. This method can be implemented using a number of different numerical approaches which have different running times depending on the number of data points and features. If you are re-solving the problem a lot, it can make sense to try out the various solvers to find the fastest one. By default, this method will return not the most likely class label for each trial it is given, but instead the probabilies for each class (=category of labels), that the trial is actually of that class. Like all machine learning methods, this method needs to be calibrated ("trained") before it can make any predictions on data. For this, the method requires training instances and associated training labels. The typical way to get such labels associated with time-series data is to make sure that a marker stream is included in the data, which is usually imported together with the data using one of the Import nodes, or received over the network alongside with the data, e.g., using the LSL Input node (with a non-empty marker query). These markers are then annotated with target labels using the Assign Targets node. To generate instances of training data for each of the training markers, one usually uses the Segmentation node to extract segments from the continuous time series around each marker. Since this machine learning method is not capable of being trained incrementally on streaming data, the method requires a data packet that contains the entire training data; this training data packet can either be accumulated online and then released in one shot using the Accumulate Calibration Data node, or it can be imported from a separate calibration recording and then spliced into the processing pipeline using the Inject Calibration Data, where it passes through the same nodes as the regular data until it reaches the machine learning node, where it is used for calibration. Once this node is calibrated, the trainable state of this node can be saved to a model file and later loaded for continued use.

More Info...

Version 1.0.2

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • penalty
    Regularization type. The default trace regularization is a good choice for most data. The trace norm yields low-rank solutions, and l1/l2 yields group-wise sparse solutions. The others are mostly included for completeness; it is more efficient to use LogisticRegression to apply these.

    • verbose name: Regularization Type
    • default value: trace
    • port type: EnumPort
    • value type: object (can be None)
  • group_axes
    Axes over which the penalty shall group. For the trace norm, 2 axes must be given. WARNING: this parameter is not safe to use with untrusted strings from the network.

    • verbose name: Group Axes
    • default value: space, time
    • port type: StringPort
    • value type: str (can be None)
  • lambdas
    Regularization strength. Stronger regularization makes it possible to fit a more complex model (more parameters), or use less data to fit a model of same complexity.

    • verbose name: Regularization Trength
    • default value: [0.1, 1.0, 5.0, 10.0]
    • port type: Port
    • value type: list (can be None)
  • search_metric
    Parameter search metric. When the regularization parameter is given as a list of values, then the method will run a cross-validation for each possible parameter value and use this metric to score how well the method performs in each case, in order to select the best parameter. While 'accuracy' is usually a good default, some other metrics can be useful under special circumstances, e.g., roc_auc for highly imbalanced ratios of trials from different classes.

    • verbose name: Scoring Metric If Searching Parameters
    • default value: accuracy
    • port type: EnumPort
    • value type: object (can be None)
  • num_folds
    Number of cross-validation folds for parameter search. Cross-validation proceeds by splitting up the data into this many blocks of trials, and then tests the method on each block. For each fold, the method is re-trained on all the other blocks, excluding the test block (therefore, the total running time is proportional to the number of folds). This is not a randomized cross-validation, but a blockwise cross-validation, which is usually the correct choice if the data stem from a time series. If there are few trials in the data, one can use a higher number here (e.g., 10) to ensure that more data is available for training.

    • verbose name: Number Of Cross-Validation Folds
    • default value: 5
    • port type: IntPort
    • value type: int (can be None)
  • num_jobs
    Number of parallel compute jobs. This value only affects the running time and not the results. Values between 1 and twice the number of CPU cores make sense to expedite computation, but may temporarily reduce the responsiveness of the machine. The value of -1 stands for all available CPU cores.

    • verbose name: Number Of Parallel Jobs
    • default value: 1
    • port type: IntPort
    • value type: int (can be None)
  • probabilistic
    Use probabilistic outputs. If enabled, the node will output for each class the probability that a given trial is of that class; otherwise it will output the most likely class label.

    • verbose name: Output Probabilities
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • verbosity
    Verbosity level. Higher numbers will produce more extensive diagnostic output.

    • verbose name: Verbosity Level
    • default value: 0
    • port type: IntPort
    • value type: int (can be None)
  • max_iter
    Maximum number of iterations. This is one of the stopping criteria to limit the compute time. The default is usually fine, and gains from increasing the number of iterations will be minimal (it can be worth experimenting with lower iteration numbers if the algorithm must finish in a fixed time budget, at a cost of potentially less accurate solutions).

    • verbose name: Maximum Number Of Iterations
    • default value: 1000
    • port type: IntPort
    • value type: int (can be None)
  • inner_gtol
    "LBFGS convergence tolerance. Larger values give less accurate results but faster solution time.

    • verbose name: Inner Gtol
    • default value: 0.0001
    • port type: FloatPort
    • value type: float (can be None)
  • inner_max_iter
    LBFGS maximum number of iterations for inner solver. Additional stopping criterion to limit compute time.

    • verbose name: Inner Max Iter
    • default value: 10
    • port type: IntPort
    • value type: int (can be None)
  • abs_tol
    Absolute convergence tolerance. Smaller values will lead the solver run to a closer approximation of the optimal solution, at the cost of increased running time. See also relative tolerance.

    • verbose name: Absolute Tolerance
    • default value: 1e-05
    • port type: FloatPort
    • value type: float (can be None)
  • rel_tol
    Relative convergence tolerance. Smaller values will lead the solver run to a closer approximation of the optimal solution, at the cost of increased running time. In contrast to the absolute tolerance, this value is relative to the magnitude of the regression weights. Note that the used method works best when the desired accuracy is not excessive, and merely a good approximation is sought.

    • verbose name: Relative Tolerance
    • default value: 0.01
    • port type: FloatPort
    • value type: float (can be None)
  • lbfgs_memory
    LBFGS memory length. This is the maximum number of variable-metric corrections tracked to approximate the inverse Hessian matrix.

    • verbose name: Lbfgs Memory
    • default value: 10
    • port type: IntPort
    • value type: int (can be None)
  • init_rho
    Initial value of augmented Lagrangian parameter. This parameter, which is specific to the used solver, can be auto-tuned, which makes the method is relatively robust to the initial value. However, slight adjustments in the 1 to 30 range can reduce the number of iterations required for convergence and thereby the running time. However, choosing a grossly inappropriate value can cause the method to fail to converge, which is easily diagnosed by having a solution that is essentially non-sparse.

    • verbose name: Solver Rho
    • default value: 4
    • port type: FloatPort
    • value type: float (can be None)
  • update_rho
    Auto-tune rho parameter. Whether to update the solver's rho parameter dynamically. This can be used to achieve faster convergence times in highly time-sensitive setups, but there is a modest risk that on some data the solution can 'blow up', although this could potentially be overcome by tuning the other solver parameters related to the rho update logic.

    • verbose name: Auto-Tune Solver Rho
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • rho_threshold
    Rho update trigger threshold. This determines how frequently updates to the solver rho parameter can be triggered. A larger value will lead to rho changing less frequently. This parameter is essentially a tradeoff between the solver adapting more quickly to settings that are optimal for convergence (using a lower value) versus preventing settings from changing too erratically (using a higher value) and thus prompting stalls or convergence failures on difficult data.

    • verbose name: Solver Rho Update Threshold
    • default value: 10
    • port type: FloatPort
    • value type: float (can be None)
  • rho_incr
    Rho update increment factor. When rho is being increased, this is the factor by which it is changed. Larger values can lead to quicker adaptation if the initial value is off or solutions change rapidly, at an increased risk of overshooting.

    • verbose name: Solver Rho Increment Factor
    • default value: 2
    • port type: FloatPort
    • value type: float (can be None)
  • rho_decr
    Rho update decrement factor. When rho is being decreased, this is the factor by which it is divided. Larger values can lead to quicker adaptation if the initial value is off or solutions change rapidly, at an increased risk of overshooting.

    • verbose name: Solver Rho Decrement Factor
    • default value: 2
    • port type: FloatPort
    • value type: float (can be None)
  • over_relaxation
    ADMM over-relaxation parameter. 1.0 is no over-relaxation.

    • verbose name: Over Relaxation
    • default value: 1.0
    • port type: FloatPort
    • value type: float (can be None)
  • initialize_once
    Calibrate the model only once. If set to False, then this node will recalibrate itself whenever a non-streaming data chunk is received that has both training labels and associated training instances.

    • verbose name: Calibrate Only Once
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • dont_reset_model
    Do not reset the model when the preceding graph is changed. Normally, when certain parameters of preceding nodes are being changed, the model will be reset. If this is enabled, the model will persist, but there is a chance that the model is incompatible when input data format to this node has changed.

    • verbose name: Do Not Reset Model
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

RidgeRegression

Estimate a continuous output value from features using Ridge Regression.

Ridge regression is a straightfoward and principled statistical technique to learn a linear mapping between input data and desired output values from training data. Ridge regression assumes that both inputs and outputs are Gaussian distributed, that is, have no or very few major statistical outliers. If the output follows a radically different distribution, for instance between 0 and 1, or nonnegative, or discrete values, then different methods may be more appropriate (for instance, classification methods for disrete values). To ameliorate the issue of outliers in the data, the raw data can be cleaned of artifacts with various artifact removal methods. To the extent that the assumptions hold true, this method is highly competitive with other linear methods, such as support vector regression (SVR). This method uses shrinkage regularization by default, which allows it to handle large numbers of features quite well. The regularization depends on a parameter that is automatically tuned using an internal cross-validation procedure (see tooltips for the controlling parameters for more details). Like all machine learning methods, this method needs to be calibrated ("trained") before it can make any predictions on data. For this, the method requires training instances and associated training labels. The typical way to get such labels associated with time-series data is to make sure that a marker stream is included in the data, which is usually imported together with the data using one of the Import nodes, or received over the network alongside with the data, e.g., using the LSL Input node (with a non-empty marker query). These markers are then annotated with target labels using the Assign Targets node. To generate instances of training data for each of the training markers, one usually uses the Segmentation node to extract segments from the continuous time series around each marker. Since this machine learning method is not capable of being trained incrementally on streaming data, the method requires a data packet that contains the entire training data; this training data packet can either be accumulated online and then released in one shot using the Accumulate Calibration Data node, or it can be imported from a separate calibration recording and then spliced into the processing pipeline using the Inject Calibration Data, where it passes through the same nodes as the regular data until it reaches the machine learning node, where it is used for calibration. Once this node is calibrated, the trainable state of this node can be saved to a model file and later loaded for continued use.

More Info...

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • alphas
    Regularization strength. This is a list of candidate values, the best of which is found via an exhaustive search (i.e., each value is tested one by one, and therefore the total running time is proportional to the number of values listed here). Larger values cause stronger regularization, that is, less risk of the method over-fitting to random details of the data, and thus better generalization to new data. A value of 0 means no regularization, and there is no upper limit to how large the values that can be given here may be -- however, depending on the scale of the data and the number of trials, there is a cutoff beyond which all features are weighted by zero, and are thus unused. Often one covers a range between 0.1 and 10, and at times 0.01 to 100. Typically the values here are not linearly spaced, but follow an exponential progression (e.g., 0.25, 0.5, 1, 2, 4, 8, ... etc). The default search range is intentionally coarse for quick running times; refine it to smaller steps to obtain potentially better solutions, but do not expect massive gains from refining.

    • verbose name: Regularization Strength
    • default value: [0.1, 0.5, 1.0, 5, 10.0]
    • port type: ListPort
    • value type: list (can be None)
  • search_metric
    Parameter search metric. When the regularization parameter is given as a list of values, then the method will run a cross-validation for each possible parameter value and use this metric to score how well the method performs in each case, in order to select the best parameter. While 'neg_mean_squared_error' is usually a good default, some other metrics can be useful under some circumstances, for instance 'neg_mean_absolute_error', which penalizes large deviations less strongly than mse.

    • verbose name: Scoring Metric For Parameter Search
    • default value: neg_mean_squared_error
    • port type: EnumPort
    • value type: object (can be None)
  • normalize_features
    Normalize features. Should only be disabled if the data comes with a predictable scale (e.g., normalized in some other way).

    • verbose name: Normalize Features
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • include_bias
    Include a bias term. If false, your features need to be centered, or include a dummy feature set to 1.

    • verbose name: Include Bias Term
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • initialize_once
    Calibrate the model only once. If set to False, then this node will recalibrate itself whenever a non-streaming data chunk is received that has both training labels and associated training instances.

    • verbose name: Calibrate Only Once
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • dont_reset_model
    Do not reset the model when the preceding graph is changed. Normally, when certain parameters of preceding nodes are being changed, the model will be reset. If this is enabled, the model will persist, but there is a chance that the model is incompatible when input data format to this node has changed.

    • verbose name: Do Not Reset Model
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

SparseBayesianRegression

Estimate a continuous output value from features using Sparse Bayesian Regression.

Sparse Bayesian regression is a an elegant Bayesian method to learn a linear mapping between input data and desired output values from training data, which is closely related to LASSO regression. The main difference to LASSO regression is that in LASSO regression, there is a tunable parameter that controls how strongly the method should be regularized. This parameter controls how "sparse" the solution is, that is, how many features shall be relied on by the model versus being pruned as irrelevant. This controls the complexity of the solution to prevent over-fitting to random details of the data and therefore to improve the generalization to new data. In LASSO, this parameter is tuned using cross-validation, that is, by empirical testing on held-out data. In the Bayesian variant, there is such a parameter as well, however, the optimal degree of regularization is estimated from the data, as well, in a theoretically clean and principled fashion. This method assumes that both inputs and outputs are Gaussian distributed, that is, have no or very few major statistical outliers. If the output follows a radically different distribution, for instance between 0 and 1, or nonnegative, or discrete values, then different methods may be more appropriate (for instance, classification methods for disrete values). To ameliorate the issue of outliers in the data, the raw data can be cleaned of artifacts with various artifact removal methods. To the extent that the assumptions hold true, this method is highly competitive with other (sparse) linear methods. Like all machine learning methods, this method needs to be calibrated ("trained") before it can make any predictions on data. For this, the method requires training instances and associated training labels. The typical way to get such labels associated with time-series data is to make sure that a marker stream is included in the data, which is usually imported together with the data using one of the Import nodes, or received over the network alongside with the data, e.g., using the LSL Input node (with a non-empty marker query). These markers are then annotated with target labels using the Assign Targets node. To generate instances of training data for each of the training markers, one usually uses the Segmentation node to extract segments from the continuous time series around each marker. Since this machine learning method is not capable of being trained incrementally on streaming data, the method requires a data packet that contains the entire training data; this training data packet can either be accumulated online and then released in one shot using the Accumulate Calibration Data node, or it can be imported from a separate calibration recording and then spliced into the processing pipeline using the Inject Calibration Data, where it passes through the same nodes as the regular data until it reaches the machine learning node, where it is used for calibration. Once this node is calibrated, the trainable state of this node can be saved to a model file and later loaded for continued use.

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • max_iter
    Maximum number of iterations. This is one of the stopping criteria to limit the compute time. The default is usually fine, and gains from increasing the number of iterations will be minimal (it can be worth experimenting with lower iteration numbers if the algorithm must finish in a fixed time budget, at a cost of potentially less accurate solutions).

    • verbose name: Maximum Number Of Iterations
    • default value: 300
    • port type: IntPort
    • value type: int (can be None)
  • tolerance
    Convergence tolerance. This is the desired errors tolerance or acceptable inaccuracy in the solution. Using larger values gives less accurate results, but will lead to faster compute times. Note that, for biosignal-driven machine learning systems, one often does not need very small tolerances.

    • verbose name: Tolerance
    • default value: 0.001
    • port type: FloatPort
    • value type: float (can be None)
  • verbosity
    Verbosity level. Higher numbers will produce more extensive diagnostic output.

    • verbose name: Verbosity Level
    • default value: 0
    • port type: IntPort
    • value type: int (can be None)
  • include_bias
    Include a bias term. If false, your features need to be centered, or include a dummy feature set to 1.

    • verbose name: Include Bias Term
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • normalize_features
    Normalize features. Should only be disabled if the data comes with a predictable scale (e.g., normalized in some other way).

    • verbose name: Normalize Features
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • alpha_shape
    Alpha shape parameter. This is only included for completeness and usually does not have to be adjusted. This is the shape parameter for the Gamma distribution prior over the alpha parameter. By default, this is an uninformative prior.

    • verbose name: Alpha Shape Parameter
    • default value: 1e-06
    • port type: FloatPort
    • value type: float (can be None)
  • alpha_rate
    Alpha rate parameter. This is only included for completeness and usually does not have to be adjusted. This is the rate parameter for the Gamma distribution prior over the alpha parameter. By default, this is an uninformative prior.

    • verbose name: Alpha Rate Parameter
    • default value: 1e-06
    • port type: FloatPort
    • value type: float (can be None)
  • lambda_shape
    Lambda shape parameter. This is only included for completeness and usually does not have to be adjusted. This is the shape parameter for the Gamma distribution prior over the lambda parameter. By default, this is an uninformative prior.

    • verbose name: Lambda Shape Parameter
    • default value: 1e-06
    • port type: FloatPort
    • value type: float (can be None)
  • lambda_rate
    Lambda rate parameter. This is only included for completeness and usually does not have to be adjusted. This is the rate parameter for the Gamma distribution prior over the lambda parameter. By default, this is an uninformative prior.

    • verbose name: Lambda Rate Parameter
    • default value: 1e-06
    • port type: FloatPort
    • value type: float (can be None)
  • initialize_once
    Calibrate the model only once. If set to False, then this node will recalibrate itself whenever a non-streaming data chunk is received that has both training labels and associated training instances.

    • verbose name: Calibrate Only Once
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • dont_reset_model
    Do not reset the model when the preceding graph is changed. Normally, when certain parameters of preceding nodes are being changed, the model will be reset. If this is enabled, the model will persist, but there is a chance that the model is incompatible when input data format to this node has changed.

    • verbose name: Do Not Reset Model
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

StochasticGradientDescentClassification

Classify data instances using models trained via stochastic gradient descent.

Stochastic gradient descent is an optimization technique that is very fast for very large amounts of training data, since it does not have to touch all the data in each update iteration. This node is rather flexible in that it can implement a variety of data terms, including logistic regression, support vector machines, certain robust classifiers, and some rarely-used variants. These can be combined with a variety of regularization terms to control the complexity of the solution, including shrinkage/ridge regularization (l2), sparsity ( l1), and elastic net (l1+l2). In this sense, this node can mimic the features of several other nodes in NeuroPype, with different tradeoffs in running time that depend on the number of data points and the number of features, where this node is mainly of interest for very large numbers of data points (i.e., trials). See also tooltips of the parameters for toggling these features for more information on the involved tradeoffs, as well as the documentation of the other nodes ( Logistic Regression, and Support Vector Classification). One unique feature of this node is the ability to implement a robust classifier that theoretically can perform better than most other methods if the data do indeed contain outliers. For this reason, removing artifacts from the raw data will make less of a difference when this method is used. Like the other methods, this node includes tuning parameters that are automatically tuned using cross-validation. If there are very few trials, or some extensive stretches of the data exhibit only one class, the procedure used to find the regularization parameters (cross-validation) can fail with an error that there were too few or no trials of a given class present. By default, this method will return not the most likely class label for each trial it is given, but instead the probabilies for each class (=category of labels), that the trial is actually of that class. Like all machine learning methods, this method needs to be calibrated ("trained") before it can make any predictions on data. For this, the method requires training instances and associated training labels. The typical way to get such labels associated with time-series data is to make sure that a marker stream is included in the data, which is usually imported together with the data using one of the Import nodes, or received over the network alongside with the data, e.g., using the LSL Input node (with a non-empty marker query). These markers are then annotated with target labels using the Assign Targets node. To generate instances of training data for each of the training markers, one usually uses the Segmentation node to extract segments from the continuous time series around each marker. Since this machine learning method is not capable of being trained incrementally on streaming data, the method requires a data packet that contains the entire training data; this training data packet can either be accumulated online and then released in one shot using the Accumulate Calibration Data node, or it can be imported from a separate calibration recording and then spliced into the processing pipeline using the Inject Calibration Data, where it passes through the same nodes as the regular data until it reaches the machine learning node, where it is used for calibration. Once this node is calibrated, the trainable state of this node can be saved to a model file and later loaded for continued use.

More Info...

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • loss
    Loss function to use. This selects the data term, i.e., what assumptions are being imposed on the data. Hinge yields an SVM, log yields logistic regression, modified_huber is another smooth loss that enables probability outputs and tolerance to outliers, squared_hinge is like hinge but is quadratically penalized. Perceptron is the loss used by the perceptron algorithm. Epsilon_insensitive is equivalent to support vector regression, and squared_epsilon_insensitive is a rarely used hybrid between linear and support vector regression. These regression techniques are here used to classify data instances, though.

    • verbose name: Data Term
    • default value: log
    • port type: EnumPort
    • value type: object (can be None)
  • regularizer
    Regularization term. Selecting l2 (default) is the most commonly used regularization, and is very effective in preventing over-fitting to random details in the training data, thus helping generalization to new data. The l1 method is applicable when only a small number of features in the data are relevant and most features are irrelevant, and this regularization is capable of identifying and selecting the relevant features (although if the optimal model is not sparse, then this will work less well than l2). The elasticnet version is a weighted combination of the l1 and l2, which addresses an issue with l1 where highly correlated features would receive rather arbitrary relative weights.

    • verbose name: Regularization Term
    • default value: l2
    • port type: EnumPort
    • value type: object (can be None)
  • alphas
    Regularization strength. This is a list of candidate values, the best of which is found via an exhaustive search (i.e., each value is tested one by one, and therefore the total running time is proportional to the number of values listed here). The details of the parameter search can be controlled via the search metric and number of folds parameters. Larger values cause stronger regularization, that is, less risk of over-fitting. A value of 0 means no regularization, and there is no upper limit to how large the values that can be given here may be -- however, depending on the scale of the data and the number of trials, there is a cutoff beyond which all features are weighted by zero, and are thus unused. Often one covers a range between 0.1 and 10, and at times 0.01 to 100. Typically the values here are not linearly spaced, but follow an exponential progression ( e.g., 0.25, 0.5, 1, 2, 4, 8, ... etc). The default search range is intentionally coarse for quick running times; refine it to smaller steps to obtain potentially better solutions, but do not expect massive gains from refining.

    • verbose name: Regularization Strength
    • default value: [0.1, 0.5, 1.0, 5, 10.0]
    • port type: ListPort
    • value type: list (can be None)
  • l1_ratio
    Tradeoff parameter between between l1 and l2 penalties when using the elasticnet regularization term. This parameter controls the balance between the l1 regularization term (as in LASSO) and the l2 regularization term (as in ridge regression). A value of 0 leads to exclusive use of l2, and a value of 1 leads to exlusively use of l1. This is a list of candidate values, the best of which is found via an exhaustive search (i.e., each value is tested one by one, and therefore the total running time is proportional to the number of values listed here). In fact, for each setting of this parameter, the entire list of possible values for the regularization strength is tested, so the running is also proportional to the number of those values. The details of the parameter search can be controlled via the search metric and number of folds parameters. By default this is not seached, but a reasonable choice would be [0.1, 0.5, 0.7, 0.9, 0.95, 0.99, 1].

    • verbose name: Regularization Type Tradeoff Parameter
    • default value: [0.5]
    • port type: ListPort
    • value type: list (can be None)
  • num_iter
    Number of passes through the data. This is proportional to the number of iterations of the method. [DEPRECATED. Will be removed from future version.]

    • verbose name: Number Of Iterations
    • default value: 5
    • port type: IntPort
    • value type: int (can be None)
  • num_jobs
    Number of parallel compute jobs. This value only affects the running time and not the results. Values between 1 and twice the number of CPU cores make sense to expedite computation, but may temporarily reduce the responsiveness of the machine. The value of -1 stands for all available CPU cores.

    • verbose name: Number Of Parallel Jobs
    • default value: 1
    • port type: IntPort
    • value type: int (can be None)
  • search_metric
    Parameter search metric. When the regularization parameter is given as a list of values, then the method will run a cross-validation for each possible parameter value and use this metric to score how well the method performs in each case, in order to select the best parameter. While 'accuracy' is usually a good default, some other metrics can be useful under special circumstances, e.g., roc_auc for highly imbalanced ratios of trials from different classes.

    • verbose name: Scoring Metric If Searching Parameters
    • default value: accuracy
    • port type: EnumPort
    • value type: object (can be None)
  • num_folds
    Number of cross-validation folds for parameter search. Cross-validation proceeds by splitting up the data into this many blocks of trials, and then tests the method on each block. For each fold, the method is re-trained on all the other blocks, excluding the test block (therefore, the total running time is proportional to the number of folds). This is not a randomized cross-validation, but a blockwise cross-validation, which is usually the correct choice if the data stem from a time series. If there are few trials in the data, one can use a higher number here (e.g., 10) to ensure that more data is available for training.

    • verbose name: Number Of Cross-Validation Folds
    • default value: 5
    • port type: IntPort
    • value type: int (can be None)
  • shuffle
    Shuffle data after each epoch. This is useful if the data comes from a time series with highly correlated successive trials, and is worth experimenting with to reach faster convergence. The main reason why it is disabled by default is that traditional SGD does not enable this by default.

    • verbose name: Shuffle Data
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • probabilistic
    Use probabilistic outputs. If enabled, the node will output for each class the probability that a given trial is of that class; otherwise it will output the most likely class label.

    • verbose name: Output Probabilities
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • warm_start
    Start from previous solution. This allows for online updating of the model.

    • verbose name: Warm Start
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • verbosity
    Verbosity level. Higher numbers will produce more extensive diagnostic output.

    • verbose name: Verbosity Level
    • default value: 0
    • port type: IntPort
    • value type: int (can be None)
  • averaging
    Use averaging. This is an alternative way of performing stochastic gradient descent, which can lead to faster convergence. Can also be an integer greater than 1; in this case, this is the number of samples seen after which averaging kicks in.

    • verbose name: Averaging
    • default value: False
    • port type: Port
    • value type: object (can be None)
  • include_bias
    Include a bias term. If false, your features need to be centered, or include a dummy feature set to 1.

    • verbose name: Include Bias Term
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • epsilon
    Epsilon for epsilon-insensitive and Huber data terms. Note that this depends strongly on the scale of the data. Can be interpreted as the cutoff in data units beyond which data values are treated more robustly (i.e., as potential outliers).

    • verbose name: Epsilon
    • default value: 0.1
    • port type: FloatPort
    • value type: float (can be None)
  • learning_rate_schedule
    Schedule for adapting the learning rate. This is a highly technical setting that allows to implement different forms of stochastic gradient descent. The options correspond to the following rules for computing the learning rate (step size) eta of the method dependent on the current iteration number t, where 'constant' means eta=eta0, 'optimal' yields the formula eta=1.0/(t+t0), and 'invscaling' corresponds to the formula eta=eta0/pow(t,power_t). The main reason to change this would be to replicate some other published implementation.

    • verbose name: Learning Rate Schedule
    • default value: invscaling
    • port type: EnumPort
    • value type: object (can be None)
  • eta0
    Initial learning rate. This is the parameter eta0 in the learning rate schedule formulate.

    • verbose name: Initial Learning Rate (Step Size)
    • default value: 0.01
    • port type: FloatPort
    • value type: float (can be None)
  • power_t
    Exponent in invscaling schedule. This is the exponent power_t in the invscaling learning rate schedule.

    • verbose name: Exponential Falloff (Invscaling Only)
    • default value: 0.25
    • port type: FloatPort
    • value type: float (can be None)
  • random_seed
    Random seed. Different values may give slightly different outcomes.

    • verbose name: Random Seed
    • default value: 12345
    • port type: IntPort
    • value type: int (can be None)
  • initialize_once
    Calibrate the model only once. If set to False, then this node will recalibrate itself whenever a non-streaming data chunk is received that has both training labels and associated training instances.

    • verbose name: Calibrate Only Once
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • class_weights
    Per-class weights. Optionally this is a mapping from class label to weight. This is formatted as in, e.g., {0: 0.1, 1: 0.2, 2: 0.1}. Alternatively, 'balanced' may be used which uses the values of y to automatically adjust weights inversely proportional to class frequencies in the input data as n_samples / (n_classes * np.bincount(y)). Class weights are used fairly infrequently, except when the cost of false detections of different classes needs to be traded off.

    • verbose name: Per-Class Weight
    • default value: None
    • port type: Port
    • value type: object (can be None)
  • dont_reset_model
    Do not reset the model when the preceding graph is changed. Normally, when certain parameters of preceding nodes are being changed, the model will be reset. If this is enabled, the model will persist, but there is a chance that the model is incompatible when input data format to this node has changed.

    • verbose name: Do Not Reset Model
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

StochasticGradientDescentRegression

Estimate a continuous output value from features using linear regression with using stochastic gradient descent.

Stochastic gradient descent is an optimization technique that is very fast for very large amounts of training data, since it does not have to touch all the data in each update iteration. This node is rather flexible in that it can implement a variety of data terms, including traditional linear regression, robust linear regression, support vector regression, and hybrid variants, which can be combied with a variety of regularization terms to control the complexity of the solution, including shrinkage/ridge regularization (l2), sparsity (l1), and elastic net (l1+l2). In this sense, this node can mimic the features of several other nodes in NeuroPype, with different tradeoffs in running time that depend on the number of data points and the number of features, where this node is mainly of interest for very large numbers of data points (i.e., trials). See also tooltips of the parameters for toggling these features for more information on the involved tradeoffs, as well as the documentation of the other nodes (Ridge Regression, LASSO Regression, Elastic Net Regression, Support Vector Regression), specifically regarding the regularization terms. One unique feature of this node is the ability to implement robust linear regression, which is robust to outliers in the data, and therefore will perform better than most other methods if the data do indeed contain outliers. For this reason, removing artifacts from the raw data will make less of a difference for this method. Like the other methods, this node includes tuning parameters that are automatically tuned using cross-validation. If there are very few trials, or some extensive stretches of the data exhibit only one class, the procedure used to find the regularization parameters (cross-validation) can fail with an error that there were too few or no trials of a given class present. Like all machine learning methods, this method needs to be calibrated ("trained") before it can make any predictions on data. For this, the method requires training instances and associated training labels. The typical way to get such labels associated with time-series data is to make sure that a marker stream is included in the data, which is usually imported together with the data using one of the Import nodes, or received over the network alongside with the data, e.g., using the LSL Input node (with a non-empty marker query). These markers are then annotated with target labels using the Assign Targets node. To generate instances of training data for each of the training markers, one usually uses the Segmentation node to extract segments from the continuous time series around each marker. Since this machine learning method is not capable of being trained incrementally on streaming data, the method requires a data packet that contains the entire training data; this training data packet can either be accumulated online and then released in one shot using the Accumulate Calibration Data node, or it can be imported from a separate calibration recording and then spliced into the processing pipeline using the Inject Calibration Data, where it passes through the same nodes as the regular data until it reaches the machine learning node, where it is used for calibration. Once this node is calibrated, the trainable state of this node can be saved to a model file and later loaded for continued use.

More Info...

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • loss
    Loss function to use. This selects the data term, i.e., what assumptions are being imposed on the data. Squared loss corresponds to traditional linear regression, huber fitting is robust regression (data may contain outliers), epsilon_insensitive is equivalent to support vector regression, and squared_epsilon_insensitive is a rarely used hybrid between linear and support vector regression.

    • verbose name: Data Term
    • default value: squared_loss
    • port type: EnumPort
    • value type: object (can be None)
  • regularizer
    Regularization term. Selecting l2 (default) is the most commonly used regularization, and is very effective in preventing over-fitting to random details in the training data, thus helping generalization to new data. The l1 method is applicable when only a small number of features in the data are relevant and most features are irrelevant, and this regularization is capable of identifying and selecting the relevant features (although if the optimal model is not sparse, then this will work less well than l2). The elasticnet version is a weighted combination of the l1 and l2, which addresses an issue with l1 where highly correlated features would receive rather arbirary relative weights.

    • verbose name: Regularization Term
    • default value: l2
    • port type: EnumPort
    • value type: object (can be None)
  • alphas
    Regularization strength. This is a list of candidate values, the best of which is found via an exhaustive search (i.e., each value is tested one by one, and therefore the total running time is proportional to the number of values listed here). The details of the parameter search can be controlled via the search metric and number of folds parameters. Larger values cause stronger regularization, that is, less risk of over-fitting. A value of 0 means no regularization, and there is no upper limit to how large the values that can be given here may be -- however, depending on the scale of the data and the number of trials, there is a cutoff beyond which all features are weighted by zero, and are thus unused. Often one covers a range between 0.1 and 10, and at times 0.01 to 100. Typically the values here are not linearly spaced, but follow an exponential progression ( e.g., 0.25, 0.5, 1, 2, 4, 8, ... etc). The default search range is intentionally coarse for quick running times; refine it to smaller steps to obtain potentially better solutions, but do not expect massive gains from refining.

    • verbose name: Regularization Strength
    • default value: [0.1, 0.5, 1.0, 5, 10.0]
    • port type: ListPort
    • value type: list (can be None)
  • l1_ratio
    Tradeoff parameter between between l1 and l2 penalties when using the elasticnet regularization term. This parameter controls the balance between the l1 regularization term (as in LASSO) and the l2 regularization term (as in ridge regression). A value of 0 leads to exclusive use of l2, and a value of 1 leads to exlusively use of l1. This is a list of candidate values, the best of which is found via an exhaustive search (i.e., each value is tested one by one, and therefore the total running time is proportional to the number of values listed here). In fact, for each setting of this parameter, the entire list of possible values for the regularization strength is tested, so the running is also proportional to the number of those values. The details of the parameter search can be controlled via the search metric and number of folds parameters. By default this is not seached, but a reasonable choice would be [0.1, 0.5, 0.7, 0.9, 0.95, 0.99, 1].

    • verbose name: Regularization Type Tradeoff Parameter
    • default value: [0.5]
    • port type: ListPort
    • value type: list (can be None)
  • num_iter
    Number of passes through the data. This is proportional to the number of iterations of the method. [DEPRECATED. Will be removed from future version.]

    • verbose name: Number Of Iterations
    • default value: 5
    • port type: IntPort
    • value type: int (can be None)
  • shuffle
    Shuffle data after each epoch. This is useful if the data comes from a time series with highly correlated successive trials, and is worth experimenting with to reach faster convergence. The main reason why it is disabled by default is that traditional SGD does not enable this by default.

    • verbose name: Shuffle Data
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • warm_start
    Start from previous solution. This allows for online updating of the model.

    • verbose name: Warm Start
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • search_metric
    Parameter search metric. When the regularization parameter is given as a list of values, then the method will run a cross-validation for each possible parameter value and use this metric to score how well the method performs in each case, in order to select the best parameter. While 'neg_mean_squared_error' is usually a good default, some other metrics can be useful under some circumstances, for instance 'neg_mean_absolute_error', which penalizes large deviations less strongly than mse.

    • verbose name: Scoring Metric For Parameter Search
    • default value: neg_mean_squared_error
    • port type: EnumPort
    • value type: object (can be None)
  • num_folds
    Number of cross-validation folds for parameter search. Cross-validation proceeds by splitting up the data into this many blocks of trials, and then tests the method on each block. For each fold, the method is re-trained on all the other blocks, excluding the test block (therefore, the total running time is proportional to the number of folds). This is not a randomized cross-validation, but a blockwise cross-validation, which is usually the correct choice if the data stem from a time series. If there are few trials in the data, one can use a higher number here (e.g., 10) to ensure that more data is available for training.

    • verbose name: Number Of Cross-Validation Folds
    • default value: 5
    • port type: IntPort
    • value type: int (can be None)
  • verbosity
    Verbosity level. Higher numbers will produce more extensive diagnostic output.

    • verbose name: Verbosity Level
    • default value: 0
    • port type: IntPort
    • value type: int (can be None)
  • averaging
    Use averaging. This is an alternative way of performing stochastic gradient descent, which can lead to faster convergence. Can also be an integer greater than 1; in this case, this is the number of samples seen after which averaging kicks in.

    • verbose name: Averaging
    • default value: False
    • port type: Port
    • value type: object (can be None)
  • include_bias
    Include a bias term. If false, your features need to be centered, or include a dummy feature set to 1.

    • verbose name: Include Bias Term
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • epsilon
    Epsilon for epsilon-insensitive and Huber data terms. Note that this depends strongly on the scale of the data. Can be interpreted as the cutoff in data units beyond which data values are treated more robustly (i.e., as potential outliers).

    • verbose name: Epsilon
    • default value: 0.1
    • port type: FloatPort
    • value type: float (can be None)
  • learning_rate_schedule
    Schedule for adapting the learning rate. This is a highly technical setting that allows to implement different forms of stochastic gradient descent. The options correspond to the following rules for computing the learning rate (step size) eta of the method dependent on the current iteration number t, where 'constant' means eta=eta0, 'optimal' yields the formula eta=1.0/(t+t0), and 'invscaling' corresponds to the formula eta=eta0/pow(t,power_t). The main reason to change this would be to replicate some other published implementation.

    • verbose name: Learning Rate Schedule
    • default value: invscaling
    • port type: EnumPort
    • value type: object (can be None)
  • eta0
    Initial learning rate. This is the parameter eta0 in the learning rate schedule formulate.

    • verbose name: Initial Learning Rate (Step Size)
    • default value: 0.01
    • port type: FloatPort
    • value type: float (can be None)
  • power_t
    Exponent in invscaling schedule. This is the exponent power_t in the invscaling learning rate schedule.

    • verbose name: Exponential Falloff (Invscaling Only)
    • default value: 0.25
    • port type: FloatPort
    • value type: float (can be None)
  • random_seed
    Random seed. Different values may give slightly different outcomes.

    • verbose name: Random Seed
    • default value: 12345
    • port type: IntPort
    • value type: int (can be None)
  • initialize_once
    Calibrate the model only once. If set to False, then this node will recalibrate itself whenever a non-streaming data chunk is received that has both training labels and associated training instances.

    • verbose name: Calibrate Only Once
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • dont_reset_model
    Do not reset the model when the preceding graph is changed. Normally, when certain parameters of preceding nodes are being changed, the model will be reset. If this is enabled, the model will persist, but there is a chance that the model is incompatible when input data format to this node has changed.

    • verbose name: Do Not Reset Model
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

SupportVectorClassification

Classification using support vector machines.

This is the traditional kernel support vector machine, which is very efficient at finding non-linear structure in the data. This method has two important regularization parameters (cost and gamma), which are by default searched over a relatively coarse interval. For best results you may have to use a finer range (at some performance cost). Note that it is typically not very reasonable to use a non-linear classifier like this on the channel signal without any sort of spatial filtering (such as CSP or ICA). Like all machine learning methods, this method needs to be calibrated ("trained") before it can make any predictions on data. For this, the method requires training instances and associated training labels. The typical way to get such labels associated with time-series data is to make sure that a marker stream is included in the data, which is usually imported together with the data using one of the Import nodes, or received over the network alongside with the data, e.g., using the LSL Input node (with a non-empty marker query). These markers are then annotated with target labels using the Assign Targets node. To generate instances of training data for each of the training markers, one usually uses the Segmentation node to extract segments from the continuous time series around each marker. Since this machine learning method is not capable of being trained incrementally on streaming data, the method requires a data packet that contains the entire training data; this training data packet can either be accumulated online and then released in one shot using the Accumulate Calibration Data node, or it can be imported from a separate calibration recording and then spliced into the processing pipeline using the Inject Calibration Data, where it passes through the same nodes as the regular data until it reaches the machine learning node, where it is used for calibration. Once this node is calibrated, the trainable state of this node can be saved to a model file and later loaded for continued use.

More Info...

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • kernel
    Kernel type to use. This is a non-linear transform of the feature space, allowing for non-linear decision boundaries between classes. The different kernels are Linear kernel (a trivial do-nothing kernel), Polynomial, which yields components that are all possible polynomial combinations of input features up to the desired degree, Radial-Basis Functions, which is one of the most commonly-used non-linear kernels and the Sigmoid kernel.

    • verbose name: Kernel Type
    • default value: rbf
    • port type: EnumPort
    • value type: object (can be None)
  • probabilistic
    Use probabilistic outputs. If enabled, the node will output for each class the probability that a given trial is of that class; otherwise it will output the most likely class label.

    • verbose name: Output Probabilities
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • tolerance
    Convergence tolerance. This is the desired errors tolerance or acceptable inaccuracy in the solution. Using larger values gives less accurate results, but will lead to faster compute times. Note that, for biosignal-driven machine learning systems, one often does not need very small tolerances.

    • verbose name: Tolerance
    • default value: 0.001
    • port type: FloatPort
    • value type: float (can be None)
  • max_iter
    Maximum number of iterations. This is one of the stopping criteria to limit the compute time. The default is usually fine, and gains from increasing the number of iterations will be minimal (it can be worth experimenting with lower iteration numbers if the algorithm must finish in a fixed time budget, at a cost of potentially less accurate solutions). If set to -1, no limit is in effect.

    • verbose name: Maximum Number Of Iterations
    • default value: -1
    • port type: IntPort
    • value type: int (can be None)
  • cost
    SVM cost parameter. This value determines the degree to which solutions are penalized that would mis-classify data points. Higher values result in models that are less likely to mis-classify the training data, but at the expense of potentially worse generalization to new data (less margin for error when slightly different trials are encountered in future test data). This is a list of candidate values, the best of which is found via an exhaustive search (i.e., each value is tested one by one, and therefore the total running time is proportional to the number of values listed here). The details of the parameter search can be controlled via the search metric and number of folds parameters. Larger values cause stronger regularization, that is, less risk of the method over-fitting to random details of the data, and thus better generalization to new data. A very small value means effectively no penalty, and there is no upper limit to how large the values that can be given here may be -- however, depending on the scale of the data and the number of trials. Often one covers a range between 0.1 and 10, and at times 0.01 to 100. Typically the values here are not linearly spaced, but follow an exponential progression (e.g., 0.25, 0.5, 1, 2, 4, 8, ... etc). The default search range is intentionally coarse for quick running times; refine it to smaller steps to obtain potentially better solutions, but do not expect massive gains from refining.

    • verbose name: Cost
    • default value: [0.01, 0.1, 1.0, 10.0, 100]
    • port type: Port
    • value type: list (can be None)
  • poly_degree
    Degree of the polynomial kernel. Ignored by other kernel types. This is the maximum degree of polynomial combinations of feature that are generated. This is also a list of possible values, and is searched in a grid search just like the cost parameter (see cost parameter for details on this procedure).

    • verbose name: Degree (Polynomial Kernel Only)
    • default value: [1, 2, 3]
    • port type: Port
    • value type: list (can be None)
  • gamma
    Gamma parameter of the RBF kernel. This parameter controls the scale of the kernel mapping, where lower scales can capture smaller-scale structure in the data. When left at the default, it resolves to 1 divided by the number of features. This is a list of possible values, and is searched in a grid search just like the cost parameter (see cost parameter for details on this procedure).

    • verbose name: Scale (Rbf Kernel Only)
    • default value: [0.0001, 0.001, 0.01, 0.1, 1.0, 10.0]
    • port type: Port
    • value type: list (can be None)
  • coef0
    Constant term in kernel function. Only used in polynomial and sigmoid kernels.

    • verbose name: Constant (Poly Or Sigmoid Kernels Only)
    • default value: [0.0, 1.0]
    • port type: Port
    • value type: list (can be None)
  • search_metric
    Parameter search metric. This method will run a cross-validation for each possible shrinkage parameter value and use this metric to score how well the method performs in each case, in order to select the best parameter. Therefore, the running time of the method is proportional to the number of parameter values times the number of folds in the cross-validation, which can be slow. While 'accuracy' is usually a good default, some other metrics can be useful under special circumstances, e.g., roc_auc for highly imbalanced ratios of trials from different classes.

    • verbose name: Scoring Metric For Parameter Search
    • default value: accuracy
    • port type: EnumPort
    • value type: object (can be None)
  • num_folds
    Number of cross-validation folds for parameter search. Cross-validation proceeds by splitting up the data into this many blocks of trials, and then tests the method on each block. For each fold, the method is re-trained on all the other blocks, excluding the test block. This is not a randomized cross-validation, but a blockwise cross-validation, which is usually the correct choice if the data stem from a time series. If there are few trials in the data, one can use a higher number here (e.g., 10) to ensure that more data is available for training

    • verbose name: Number Of Cross-Validation Folds
    • default value: 5
    • port type: IntPort
    • value type: int (can be None)
  • verbosity
    Verbosity level. Higher numbers will produce more extensive diagnostic output.

    • verbose name: Verbosity Level
    • default value: 0
    • port type: IntPort
    • value type: int (can be None)
  • class_weights
    Per-class weights. Optionally this is a mapping from class label to weight. This is formatted as in, e.g., {0: 0.1, 1: 0.2, 2: 0.1}. Alternatively, 'balanced' may be used which uses the values of y to automatically adjust weights inversely proportional to class frequencies in the input data as n_samples / (n_classes * np.bincount(y)). Class weights are used fairly infrequently, except when the cost of false detections of different classes needs to be traded off.

    • verbose name: Per-Class Weight
    • default value: None
    • port type: Port
    • value type: object (can be None)
  • shrinking
    Use shrinking heuristic.

    • verbose name: Shrinking
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • cache_size
    "Cache size in MB.

    • verbose name: Cache Size
    • default value: 200
    • port type: IntPort
    • value type: int (can be None)
  • random_seed
    Random seed. Different values may give slightly different outcomes.

    • verbose name: Random Seed
    • default value: 12345
    • port type: IntPort
    • value type: int (can be None)
  • initialize_once
    Calibrate the model only once. If set to False, then this node will recalibrate itself whenever a non-streaming data chunk is received that has both training labels and associated training instances.

    • verbose name: Calibrate Only Once
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • dont_reset_model
    Do not reset the model when the preceding graph is changed. Normally, when certain parameters of preceding nodes are being changed, the model will be reset. If this is enabled, the model will persist, but there is a chance that the model is incompatible when input data format to this node has changed.

    • verbose name: Do Not Reset Model
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)

SupportVectorRegression

Regression using support vector machines.

This method is very similar to Support Vector Classification, the difference being that the desired target output can be a continuous value (as opposed to a class label or probability) as with all regression methods. Like all machine learning methods, this method needs to be calibrated ("trained") before it can make any predictions on data. For this, the method requires training instances and associated training labels. The typical way to get such labels associated with time-series data is to make sure that a marker stream is included in the data, which is usually imported together with the data using one of the Import nodes, or received over the network alongside with the data, e.g., using the LSL Input node (with a non-empty marker query). These markers are then annotated with target labels using the Assign Targets node. To generate instances of training data for each of the training markers, one usually uses the Segmentation node to extract segments from the continuous time series around each marker. Since this machine learning method is not capable of being trained incrementally on streaming data, the method requires a data packet that contains the entire training data; this training data packet can either be accumulated online and then released in one shot using the Accumulate Calibration Data node, or it can be imported from a separate calibration recording and then spliced into the processing pipeline using the Inject Calibration Data, where it passes through the same nodes as the regular data until it reaches the machine learning node, where it is used for calibration. Once this node is calibrated, the trainable state of this node can be saved to a model file and later loaded for continued use.

More Info...

Version 1.0.0

Ports/Properties

  • set_breakpoint
    Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.

    • verbose name: Set Breakpoint (Debug Only)
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • data
    Data to process.

    • verbose name: Data
    • default value: None
    • port type: DataPort
    • value type: Packet (can be None)
    • data direction: INOUT
  • kernel
    Kernel type to use. This is a non-linear transform of the feature space, allowing for non-linear decision boundaries between classes. The different kernels are Linear kernel (a trivial do-nothing kernel), Polynomial, which yields components that are all possible polynomial combinations of input features up to the desired degree, Radial-Basis Functions, which is one of the most commonly-used non-linear kernels and the Sigmoid kernel.

    • verbose name: Kernel Type
    • default value: rbf
    • port type: EnumPort
    • value type: object (can be None)
  • cost
    SVM cost parameter. This value determines the degree to which solutions are penalized the outlier points. Higher values result in models that are less likely to be affected by the outliers in the training data, but at the expense of potentially worse generalization to new data (less margin for error when slightly different trials are encountered in future test data). This is a list of candidate values, the best of which is found via an exhaustive search (i.e., each value is tested one by one, and therefore the total running time is proportional to the number of values listed here). The details of the parameter search can be controlled via the search metric and number of folds parameters. Larger values cause stronger regularization, that is, less risk of the method over-fitting to random details of the data, and thus better generalization to new data. A very small value means effectively no penalty, and there is no upper limit to how large the values that can be given here may be -- however, depending on the scale of the data and the number of trials. Often one covers a range between 0.1 and 10, and at times 0.01 to 100. Typically the values here are not linearly spaced, but follow an exponential progression (e.g., 0.25, 0.5, 1, 2, 4, 8, ... etc). The default search range is intentionally coarse for quick running times; refine it to smaller steps to obtain potentially better solutions, but do not expect massive gains from refining.

    • verbose name: Cost
    • default value: [0.01, 0.1, 1.0, 10.0, 100]
    • port type: Port
    • value type: list (can be None)
  • poly_degree
    Degree of the polynomial kernel. Ignored by other kernel types. This is the maximum degree of polynomial combinations of feature that are generated. This is also a list of possible values, and is searched in a grid search just like the cost parameter (see cost parameter for details on this procedure).

    • verbose name: Degree (Polynomial Kernel Only)
    • default value: [1, 2, 3]
    • port type: Port
    • value type: list (can be None)
  • gamma
    Gamma parameter of the RBF kernel. This parameter controls the scale of the kernel mapping, where lower scales can capture smaller-scale structure in the data. When left at the default, it resolves to 1 divided by the number of features. This is a list of possible values, and is searched in a grid search just like the cost parameter (see cost parameter for details on this procedure).

    • verbose name: Scale (Rbf Kernel Only)
    • default value: [0.0001, 0.001, 0.01, 0.1, 1.0, 10.0]
    • port type: Port
    • value type: list (can be None)
  • coef0
    Constant term in kernel function. Only used in polynomial and sigmoid kernels.

    • verbose name: Constant (Poly Or Sigmoid Kernels Only)
    • default value: [0.0, 1.0]
    • port type: Port
    • value type: list (can be None)
  • epsilon
    Epsilon for epsilon-insensitive and Huber data terms. Note that this depends strongly on the scale of the data. Can be interpreted as the cutoff in data units beyond which data values are treated more robustly (i.e., as potential outliers).

    • verbose name: Epsilon
    • default value: 0.1
    • port type: FloatPort
    • value type: float (can be None)
  • tolerance
    Convergence tolerance. This is the desired errors tolerance or acceptable inaccuracy in the solution. Using larger values gives less accurate results, but will lead to faster compute times. Note that, for biosignal-driven machine learning systems, one often does not need very small tolerances.

    • verbose name: Tolerance
    • default value: 0.001
    • port type: FloatPort
    • value type: float (can be None)
  • max_iter
    Maximum number of iterations. This is one of the stopping criteria to limit the compute time. The default is usually fine, and gains from increasing the number of iterations will be minimal (it can be worth experimenting with lower iteration numbers if the algorithm must finish in a fixed time budget, at a cost of potentially less accurate solutions). If set to -1, no limit is in effect.

    • verbose name: Maximum Number Of Iterations
    • default value: -1
    • port type: IntPort
    • value type: int (can be None)
  • search_metric
    Parameter search metric. When the regularization parameter is given as a list of values, then the method will run a cross-validation for each possible parameter value and use this metric to score how well the method performs in each case, in order to select the best parameter. While 'neg_mean_squared_error' is usually a good default, some other metrics can be useful under some circumstances, for instance 'neg_mean_absolute_error', which penalizes large deviations less strongly than mse.

    • verbose name: Scoring Metric For Parameter Search
    • default value: neg_mean_squared_error
    • port type: EnumPort
    • value type: object (can be None)
  • num_folds
    Number of cross-validation folds for parameter search. Cross-validation proceeds by splitting up the data into this many blocks of trials, and then tests the method on each block. For each fold, the method is re-trained on all the other blocks, excluding the test block (therefore, the total running time is proportional to the number of folds). This is not a randomized cross-validation, but a blockwise cross-validation, which is usually the correct choice if the data stem from a time series. If there are few trials in the data, one can use a higher number here (e.g., 10) to ensure that more data is available for training.

    • verbose name: Number Of Cross-Validation Folds
    • default value: 5
    • port type: IntPort
    • value type: int (can be None)
  • verbosity
    Verbosity level. Higher numbers will produce more extensive diagnostic output.

    • verbose name: Verbosity Level
    • default value: 0
    • port type: IntPort
    • value type: int (can be None)
  • shrinking
    Use shrinking heuristic.

    • verbose name: Shrinking
    • default value: True
    • port type: BoolPort
    • value type: bool (can be None)
  • cache_size
    Cache size in MB.

    • verbose name: Cache Size
    • default value: 200
    • port type: IntPort
    • value type: int (can be None)
  • initialize_once
    Calibrate the model only once. If set to False, then this node will recalibrate itself whenever a non-streaming data chunk is received that has both training labels and associated training instances.

    • verbose name: Calibrate Only Once
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)
  • dont_reset_model
    Do not reset the model when the preceding graph is changed. Normally, when certain parameters of preceding nodes are being changed, the model will be reset. If this is enabled, the model will persist, but there is a chance that the model is incompatible when input data format to this node has changed.

    • verbose name: Do Not Reset Model
    • default value: False
    • port type: BoolPort
    • value type: bool (can be None)