TangMartinClassification¶
Binary sparse logistic regression using the model- space variational empirical-Bayes approximation of Tang and Martin (2025).
The variational distribution is an independent Bernoulli approximation directly on the model configuration space, not on the regression coefficients themselves.
The implementation follows the paper's coordinate-ascent variational inference algorithm based on the Jaakkola-Jordan lower bound and a fixed sparse pilot estimator beta_tilde. By default the pilot is a lasso model followed by an unpenalized refit on its selected support to reduce shrinkage bias. An SCAD pilot is also available as an expert option when skglm is installed, but it can be substantially slower and is not guaranteed to improve the resulting model. The pilot estimator's internal cross-validation can optionally be grouped or stratified to better match the structure of the training data. Posterior inclusion probabilities are approximated by the variational parameters phi, and the final predictive model is obtained by refitting logistic regression on the coordinates with phi >= 0.5.
Ports/Properties¶
data¶
Data to process.
cond_field¶
Instance data field that contains the class labels to discriminate. The field must be binary.
feature_scaling¶
Feature scaling applied before fitting. Standardization is recommended because the pilot sparse estimator and the variational updates are both scale-sensitive.
complexity_penalty¶
Complexity penalty applied to each additional feature included in the model. Larger values impose a stronger cost for model complexity, preferring simpler models with fewer active features; smaller values are more permissive and allow more features to be retained. The penalty scales with the total number of features, so in very high-dimensional settings (many more features than observations) the penalty can become so dominant that no feature is selected. The implementation detects this situation and automatically falls back to evaluating only the features identified by the pilot estimator. The default of 0.01 is conservative and works well across a wide range of problem sizes.
likelihood_power¶
Controls how strongly the model follows the training data versus relying on prior assumptions. A value of 1.0 gives the data full weight; values below 1.0 discount the data contribution, which acts as a form of regularization that can help prevent overfitting, especially when the number of features is much larger than the number of observations. The default of 0.99 applies a minimal discount that is sufficient for theoretical benefits while leaving the data signal nearly intact. Values around 0.9-0.95 can be tried if the model appears to overfit.
posterior_spread¶
Controls how tightly the final model coefficients are constrained around the initial pilot estimate. Smaller values force the model to stay close to the pilot, which is appropriate when the pilot is well calibrated. Larger values give the model more freedom to deviate from the pilot, which can help on datasets where the pilot estimate is unreliable or poorly calibrated (e.g., due to high noise or very few observations). The default of 0.1 is a moderately tight setting that works well in most cases.
include_bias¶
Include a bias (intercept) term. The intercept captures the overall baseline log-odds of the positive class and is not subject to feature selection. If disabled, the features should be centered or include a constant feature set to 1.
support_threshold¶
Threshold on the estimated probability that a feature is relevant. Features whose inclusion probability meets or exceeds this threshold are considered active and included in the final predictive model; all other features are discarded. A value of 0.5 (the default) corresponds to the median probability model, which selects features that are more likely relevant than not. Lower values (e.g., 0.3) will retain more features at the risk of including some noise; higher values (e.g., 0.7) are more conservative.
sample_size_factor¶
Sample size correction factor or effective sample size. When observations are not independent (e.g., multiple trials per participant), the effective information content may be considerably lower than the nominal sample count, causing the variational inference to be overconfident in feature selection. This parameter tempers the likelihood by the factor n_eff/n, reducing the data terms in the CAVI updates relative to the complexity penalty. The correction is applied by internally multiplying the likelihood power parameter by n_eff/n. Values between 0 and 1 (inclusive) are interpreted as a multiplicative factor applied to the actual number of observations; for instance, if you have 2000 trials from 30 participants, set this to 30/2000 = 0.015. Values > 1 must be integers and are interpreted as an absolute effective sample size; for instance, 30 means that the effective sample size is 30 regardless of the actual number of observations.
pilot_method¶
Method used to obtain an initial sparse estimate of the coefficients before the main variational inference step. The lasso (default) uses L1-regularized logistic regression with cross-validated regularization strength, and is fast and reliable for most problems. SCAD (smoothly clipped absolute deviation) is an alternative non-convex penalty that can reduce the shrinkage bias of the lasso on large coefficients, but requires the optional 'skglm' package and can be substantially slower. In practice, lasso is recommended unless there is a specific reason to prefer SCAD.
pilot_refit¶
Refit an unpenalized logistic regression on only the features selected by the pilot before running the main variational inference step. This is enabled by default because the pilot's regularization tends to shrink the selected coefficients toward zero, which can cause the subsequent variational step to underestimate how many features are truly relevant. The refit removes this shrinkage bias and generally improves results, especially on high-dimensional datasets. Disabling this is not recommended unless the pilot already produces well-calibrated coefficient magnitudes.
scad_gamma¶
Shape parameter for the SCAD penalty when the SCAD pilot method is selected. Larger values make the penalty behave more like the lasso; the default of 3.7 is the value recommended in the SCAD literature and works well in most settings. Only used when the pilot method is set to SCAD.
pilot_noise_scale¶
Scale of the small random noise injected into pilot coefficients that are exactly zero. This ensures that features not selected by the pilot still have a chance of being included by the variational inference step if the data supports it. The default of 1e-6 is negligibly small but sufficient to keep all features updatable. There is rarely a reason to change this value.
max_iterations¶
Maximum number of iterations for the variational inference optimization. Convergence typically occurs within 50-150 iterations for most problems. Higher-dimensional datasets may need up to 200 iterations. The default of 300 provides a generous safety margin. A warning is logged if the method does not converge within the given number of iterations.
convergence_tolerance¶
Convergence tolerance for the variational optimization. The method stops when the largest change in any feature's inclusion probability between consecutive iterations falls below this threshold. The default of 1e-4 is well calibrated for this method; tighter values rarely improve results but increase the number of iterations needed.
num_folds¶
Number of cross-validation folds used by the pilot estimator to select its regularization strength. More folds give a more reliable selection at the cost of longer fitting time. The default of 5 is a good balance for most problems.
cv_group_field¶
Optionally a field indicating the group from which each trial is sourced. If given, then pilot-model cross-validation is split such that validation folds contain unseen groups. Examples groups are SubjectID, SessionID, etc.
cv_stratified¶
Optionally perform stratified cross-validation for the pilot estimator. This means that all the folds have the same relative percentage of trials in each class.
pilot_max_iterations¶
Maximum number of iterations for the pilot estimator and the final logistic refit. The default of 3000 is generous and ensures convergence even on difficult datasets. Reducing this value can speed up fitting at the risk of the pilot not fully converging.
random_seed¶
Random seed used for pilot-noise injection and deterministic pilot fitting.
initialize_once¶
Calibrate the model only once on the first labeled data that arrives.
dont_reset_model¶
Do not reset the model when the preceding graph changes.
probabilistic¶
Output class probabilities rather than hard class labels.
verbosity¶
Verbosity level. Higher values emit more diagnostics during fitting.
set_breakpoint¶
Set a breakpoint on this node. If this is enabled, your debugger (if one is attached) will trigger a breakpoint.
metadata¶
User-definable meta-data associated with the node. Usually reserved for technical purposes.