LogisticRegression¶
Classify data instances using regularized Logistic Regression.
The logistic regression method is a versatile and principled statistical method that learns a generalized linear mapping from input data to the probability of that data belonging to one of several possible classes. This method is highly competitive with other linear or generalized linear methods, such as LDA and SVM. Logistic regression makes relatively weak assumptions on the distribution of the data, so that it is moderately tolerant to outliers, especially when compared to relatively brittle methods like LDA. Logistic regression is not considered to be quite as robust as SVM (therefore it can make sense to remove artifacts beforehand using the appropriate nodes if the data are very noisy) -- however, the main benefit of logistic regression over SVM is that the outputs can be straightforwardly interpreted as probabilities without having to resort to 'tricks' such as probability calibration. This implementation uses regularization by default, which allows it to handle large numbers of features very well. Importantly, there are two types of regularization that one can choose from: the default l2 regularization is a fine choice for most data (it is closely related to shrinkage in LDA or a Gaussian prior in Bayesian methods). The alternative l1 regularization is unique in that it can learn to identify a sparse subset of features that is relevant while pruning out all other features as irrelevant. This sparsity regularization is statistically very efficient and can deal with an extremely large number of irrelevant features. To determine the optimal regularization strength, the a list of candidate parameter settings can be given, which is then searched by the method using an internal cross-validation on the data to find the best value. If there are very few trials, or some extensive stretches of the data exhibit only one class, this cross-validation can fail with an error that there were too few or no trials of a given class present. Also, the default search grid for regularization (i.e., the list of candidate values) is deliberately rather coarse to keep the method fast. For higher-quality results, use a more fine grained list of values (which will be correspondingly slower). There are also several other implementations of logistic regression with different regularization terms or different performance characteristics in other nodes. This method can be implemented using a number of different numerical approaches which have different running times depending on the number of data points and features. If you are re-solving the problem a lot, it can make sense to try out the various solvers to find the fastest one. By default, this method will return not the most likely class label for each trial it is given, but instead the probabilies for each class (=category of labels), that the trial is actually of that class. Like all machine learning methods, this method needs to be calibrated ("trained") before it can make any predictions on data. For this, the method requires training instances and associated training labels. The typical way to get such labels associated with time-series data is to make sure that a marker stream is included in the data, which is usually imported together with the data using one of the Import nodes, or received over the network alongside with the data, e.g., using the LSL Input node (with a non-empty marker query). These markers are then annotated with target labels using the Assign Targets node. To generate instances of training data for each of the training markers, one usually uses the Segmentation node to extract segments from the continuous time series around each marker. Since this machine learning method is not capable of being trained incrementally on streaming data, the method requires a data packet that contains the entire training data; this training data packet can either be accumulated online and then released in one shot using the Accumulate Calibration Data node, or it can be imported from a separate calibration recording and then spliced into the processing pipeline using the Inject Calibration Data, where it passes through the same nodes as the regular data until it reaches the machine learning node, where it is used for calibration. Once this node is calibrated, the trainable state of this node can be saved to a model file and later loaded for continued use. More Info... Version 1.1.1
Ports/Properties¶
data¶
Data to process.
feature_scaling¶
Feature scaling to use. If set to auto, then scaling will default to robust if a sag or saga solver are used, and otherwise std. If features are not already standardized beforehand, enabling this tends to ensure that the explored range of the alphas parameter is well matched to the data and will also ensure that features are treated equally by the regularization instead of it being scale-dependent. The -scale variants do not also shift the data.
class_weights¶
Per-class weights. Optionally this is a mapping from class label to weight. The weights represent the a priori ("prior") probability of encountering a specific class that the model shall assume. The weights will be renormalized so that they add up to one. Example syntax: {'0': 0.5, '1': 0.5} (note the quotes before the colons).
cond_field¶
The name of the instance data field that contains the conditions to be discriminated. This parameter will be ignored if the packet has previously been processed by a DescribeStatisticalDesign node.
regularizer¶
Regularization type. The default l2 regularization is a good choice for most data. The alternative l1 regularization results in a type of "feature selection", that is, only a sparse set of features will have a non-zero weight, and all other features will remain effectively unused. This sparsity regularization is very useful if only a small number of features are relevant. It is nevertheless a good idea to compare to the performance of using l2 regularization, since excessive sparsity can sometimes degrade performance.
alphas¶
Regularization strength. This is a list of candidate values, the best of which is found via an exhaustive search (i.e., each value is tested one by one, and therefore the total running time is proportional to the number of values listed here). The details of the parameter search can be controlled via the search metric and number of folds parameters. Larger values cause stronger regularization, that is, less risk of the method over-fitting to random details of the data, and thus better generalization to new data. A value of 0 means no regularization, and there is no upper limit to how large the values that can be given here may be -- however, depending on the scale of the data and the number of trials, there is a cutoff beyond which all features are weighted by zero, and are thus unused. Often one covers a range between 0.1 and 10, and at times 0.01 to 100. Typically the values here are not linearly spaced, but follow an exponential progression (e.g., 0.25, 0.5, 1, 2, 4, 8, ... etc). The default search range is intentionally coarse for quick running times; refine it to smaller steps to obtain potentially better solutions, but do not expect massive gains from refining.
l1_ratios¶
Elastic-net mixing ratio. 0 is equiv. to l2, and 1 is eqiv. to l1. In-between values yield combinations. Defaults to [1/4, 2/4, 3/4].
search_metric¶
Parameter search metric. When the regularization parameter is given as a list of values, then the method will run a cross-validation for each possible parameter value and use this metric to score how well the method performs in each case, in order to select the best parameter. While 'accuracy' is usually a good default, some other metrics can be useful under special circumstances, e.g., roc_auc for highly imbalanced ratios of trials from different classes.
num_folds¶
Number of cross-validation folds for parameter search. Cross-validation proceeds by splitting up the data into this many blocks of trials, and then tests the method on each block. For each fold, the method is re-trained on all the other blocks, excluding the test block (therefore, the total running time is proportional to the number of folds). This is not a randomized cross-validation, but a blockwise cross-validation, which is usually the correct choice if the data stem from a time series. If there are few trials in the data, one can use a higher number here (e.g., 10) to ensure that more data is available for training.
num_jobs¶
Number of parallel compute jobs. This value only affects the running time and not the results. Values between 1 and twice the number of CPU cores make sense to expedite computation, but may temporarily reduce the responsiveness of the machine. The value of -1 stands for all available CPU cores.
cv_group_field¶
Optionally a field indicating the group from which each trial is sourced. If given, then data will be split such that test sets contain unseen groups. Examples groups are SubjectID, SessionID, etc.
cv_stratified¶
Optionally perform stratified cross-validation. This means that all the folds have the same relative percentage of trials in each class.
solver¶
Solver to use. Not all solvers support all regularizers -- specifically, newton-cg, sag, and lbfgs only support l2. Elasticnet is only supported by saga. Also, different algorithms have different running times depending on the number of features, and the number of trials. It can make sense to identify the fastest method if this node is used a lot on similarly-sized data.
dual_formulation¶
Use dual formulation. This is an alternative way to solve the problem. If enabled, it can be faster when the number of trials is larger than the number of features, but it is not supported with l1 regularization.
max_iter¶
Maximum number of iterations. This is one of the stopping criteria to limit the compute time. The default is usually fine, and gains from increasing the number of iterations will be minimal (it can be worth experimenting with lower iteration numbers if the algorithm must finish in a fixed time budget, at a cost of potentially less accurate solutions).
tolerance¶
Convergence tolerance. This is the desired errors tolerance or acceptable inaccuracy in the solution. Using larger values gives less accurate results, but will lead to faster compute times. Note that, for biosignal-driven machine learning systems, one often does not need very small tolerances.
multiclass¶
Technique to use when classifying more than two classes of data. These formulations will usually have little impact on the results, but multinomial can be solved in one step, whereas ovr (which stands for one-vs-rest) requires one run per class, which can be slower. Multinomial is only supported when using the lbfgs solver. Auto will auto-select based on the chosen solver.
include_bias¶
Include a bias term. If false, your features need to be centered, or include a dummy feature set to 1.
bias_scaling¶
Scale for bias term. Since this logistic regression implementation applies the regularization to the bias term too (which is usually not ideal, although rarely a significant issue), you can use this scale to counter the effect.
random_seed¶
Random seed. Different values may give slightly different outcomes.
initialize_once¶
Calibrate the model only once. If set to False, then this node will recalibrate itself whenever a non-streaming data chunk is received that has both training labels and associated training instances.
dont_reset_model¶
Do not reset the model when the preceding graph is changed. Normally, when certain parameters of preceding nodes are being changed, the model will be reset. If this is enabled, the model will persist, but there is a chance that the model is incompatible when input data format to this node has changed.
probabilistic¶
Use probabilistic outputs. If enabled, the node will output for each class the probability that a given trial is of that class; otherwise it will output the most likely class label.
verbosity¶
Verbosity level. Higher numbers will produce more extensive diagnostic output.
set_breakpoint¶
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.
metadata¶
User-definable meta-data associated with the node. Usually reserved for technical purposes.