LinearDiscriminantAnalysis¶
Classify data instances using Linear Discriminant Analysis (LDA).
The LDA method is a fast statistical method that learns a linear mapping from input data to discrete category labels. LDA assumes that the data are Gaussian-distributed, that is, have no or very few major statistical outliers. To the extent that these assumptions hold true, this method is highly competitive with other linear (or generalized linear) methods. To ameliorate the outlier issue, the raw data can be cleaned of artifacts with various artifact removal methods. This implementation uses shrinkage regularization by default, which allows it to handle large numbers of features quite gracefully, compared to the unregularized variant, which will overfit or fail on more than a few dozen features. This method can be implemented using a number of different numerical approaches which have different tradeoffs -- it is worth experimenting with different choices of the solver parameter if you cannot get the result that you seek. By default, this method will return not the most likely class label for each trial it is given, but instead the probabilies for each class (=category of labels), that the trial is actually of that class. This method can also optionally use a regularization parameter that is tuned using an internal cross-validation on the data. If there are very few trials, or some extensive stretches of the data exhibit only one class, this cross-validation can fail with an error that there were too few or no trials of a given class present. Like all machine learning methods, this method needs to be calibrated ("trained") before it can make any predictions on data. For this, the method requires training instances and associated training labels. The typical way to get such labels associated with time-series data is to make sure that a marker stream is included in the data, which is usually imported together with the data using one of the Import nodes, or received over the network alongside with the data, e.g., using the LSL Input node (with a non-empty marker query). These markers are then annotated with target labels using the Assign Targets node. To generate instances of training data for each of the training markers, one usually uses the Segmentation node to extract segments from the continuous time series around each marker. Since this machine learning method is not capable of being trained incrementally on streaming data, the method requires a data packet that contains the entire training data; this training data packet can either be accumulated online and then released in one shot using the Accumulate Calibration Data node, or it can be imported from a separate calibration recording and then spliced into the processing pipeline using the Inject Calibration Data, where it passes through the same nodes as the regular data until it reaches the machine learning node, where it is used for calibration. Once this node is calibrated, the trainable state of this node can be saved to a model file and later loaded for continued use. More Info... Version 1.1.1
Ports/Properties¶
data¶
Data to process.
class_weights¶
Per-class weights. Optionally this is a mapping from class label to weight. The weights represent the a priori ("prior") probability of encountering a specific class that the model shall assume. The weights will be renormalized so that they add up to one. Example syntax: {'0': 0.5, '1': 0.5} (note the quotes before the colons).
cond_field¶
The name of the instance data field that contains the conditions to be discriminated. This parameter will be ignored if the packet has previously been processed by a DescribeStatisticalDesign node.
shrinkage¶
Shrinkage regularization strength. If using 'auto', then a fast automatic method is used to determine the regularization parameter (using the Ledoit-Wolf method). However, if given as a list of numbers between 0 and 1 (formatted as in, e.g., [0.1, 0.2, 0.3]) where 0 is no regularization and 1 is maximal regularization, then the best parameter is searched, which can be slow. The details of the parameter search can be controlled via the search metric and number of folds parameters.
feature_selection¶
Feature selection criterion to use if sparse solutions are desired. If set to None, no feature selection is enabled. If set to rfe, recursive feature elimination is used. If set to anova, features with the highest ANOVA f-score are selected. In either case, the number of features is automatically tuned using a cross-validation, and can be capped using max_feature_select.
max_feature_select¶
Maximum number of features to retain, if feature selection is used. Using a value that is less than the number of features can be much faster than searching over all possible counts.
feature_sel_group_size¶
Number of successive features that form a group, for grouped feature selection.
feature_sel_group_op¶
Operation to use to calculate groupwise feature importance.
search_metric¶
Parameter search metric to optimize certain hyper-parameters of the node. When certain parameters (e.g., shrinkage or robust_gamma) are used and given as a list of values, then the method will run a cross-validation for each possible parameter value and use this metric to score how well the method performs in each case, in order to select the best parameter. Therefore, when a search is done the running time of the method is multiplied by the number of parameter values and the number of folds in the cross-validation, which can be slow. While 'accuracy' is usually a good default, some other metrics can be useful under special circumstances, e.g., roc_auc for highly imbalanced ratios of trials from different classes.
num_folds¶
Number of cross-validation folds for parameter search. Cross-validation proceeds by splitting up the data into this many blocks of trials, and then tests the method on each block. For each fold, the method is re-trained on all the other blocks, excluding the test block. This is not a randomized cross-validation, but a blockwise cross-validation, which is usually the correct choice if the data stem from a time series. If there are few trials in the data, one can use a higher number here (e.g., 10) to ensure that more data is available for training
cv_group_field¶
Optionally a field indicating the group from which each trial is sourced. If given, then data will be split such that test sets contain unseen groups. Examples groups are Subject_id, Session_id, etc.
cv_stratified¶
Optionally perform stratified cross-validation. This means that all the folds have the same relative percentage of trials with each label. Note that this requires labels to be quantized or binned to be meaningful.
num_jobs¶
Number of parallel compute jobs. This is only in effect when a parameter search is applied (e.g., in robust mode, or when passing shrinkage as a list). This value only affects the running time and not the results. Values between 1 and twice the number of CPU cores make sense to expedite computation, but may temporarily reduce the responsiveness of the machine. The value of -1 stands for all available CPU cores.
solver¶
Solver to use. This node supports formulations based on a least-squares solution (lsqr), eigenvalue decomposition (eigen), and singular-value decomposition (svd). Some of these methods are known to have numerical issues under various circumstances -- consider trying different settings if you cannot achieve the desired results. Note: the svd method can handle many features, but does not support shrinkage-type regularization.
tolerance¶
Threshold for rank estimation in SVD. Using a larger value will more aggressively prune features, but can make the difference between it working or not.
robust_method¶
Type of robust estimator to use, if any. This applies to the covariance matrix only at this point. The MCD method is a slow but accurate estimator using the minimum covariance determinant. The SGD method is a fast estimator using stochastic gradient descent, but requires some parameter tuning (automatic tuning is slow, but a single value can usually be determined for a given study and then used going forward). The SAG method uses stochastic average gradient and tends to be more resilient to bad parameter settings, but is otherwise similar to SGD. Both SGD and SAG are configured via the huber_threshold and robust_gamma parameters. Thus one may use MCD first to determine if robustness is helpful on the given data, and if fast runtime is needed, one can switch to SAG or SGD and tune the parameters until a similar performance is achieved. Note that, if shrinkage is set to 'auto', the shrinkage amount will generally be estimated in a non-robust fashion, which may cause some amount of over- or under-shrinkage.
huber_threshold¶
Huber threshold, applies only when using the SGD robust estimator. If set to 0, the geometric (or l1) median is used (which is maximally robust but has low statistical efficiency). If set to 'auto' or the empty list, a value will be searched. An if given as a list of values, the list will be used as the search range.
robust_gamma¶
Search range for gamma parameter when using the SGD robust estimation. Note that typically a much smaller range, or even a single value (which requires no built-in grid search), will do, but the value needs to be empirically determined.
robust_max_contamination¶
Maximum contaminated data fraction for MCD robust estimator. This is a tradeoff between resistance to larger proportions of bad data (the maximum is around 0.5, and can be used by setting the parameter to its default None value), and higher statistical efficiency since more data is used for the estimate.
initialize_once¶
Calibrate the model only once. If set to False, then this node will recalibrate itself whenever a non-streaming data chunk is received that has both training labels and associated training instances.
dont_reset_model¶
Do not reset the model when the preceding graph is changed. Normally, when certain parameters of preceding nodes are being changed, the model will be reset. If this is enabled, the model will persist, but there is a chance that the model is incompatible when input data format to this node has changed.
dimensionality_reduction¶
Reduce dimensionality of data to the given number of dimensions. If given, this overrides the regular outputs of LDA (governed by the probabilistic flag), and causes it to produce data of the desired number of dimensions.
probabilistic¶
Use probabilistic outputs. If enabled, the node will output for each class the probability that a given trial is of that class; otherwise it will output the most likely class label.
verbosity¶
Verbosity level. Higher numbers will produce more extensive diagnostic output.
set_breakpoint¶
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.
metadata¶
User-definable meta-data associated with the node. Usually reserved for technical purposes.