Adaptive estimators¶

Two-stage adaptive penalty estimators (Zou 2006 for adaptive lasso, extended here to MCP and SCAD). The recipe:

Pilot fit. Run a plain lasso path (MCP at γ → ∞) on (X, y) and read β at a chosen position along the path (default: middle of the auto-generated λ-grid).
Adaptive weights. Compute per-feature w_j = 1 / max(|β_pilot[j]|, ε)^η. Larger pilot magnitudes mean smaller penalty weights — truly active features are shrunk less, inactive features (with β_pilot ≈ 0) get huge weights and stay at zero.
Final fit. Re-fit the chosen final penalty (Lasso / MCP / SCAD) with these adaptive weights. The path is the final estimator’s path, λ-decreasing.

The motivating result is the oracle property: under regularity conditions, adaptive lasso recovers the true sparse support and yields asymptotically unbiased estimates on the active features — neither of which plain lasso provides. Adaptive MCP / SCAD inherit this story but typically need fewer outer iterations because the underlying penalty already has the “near-unbiased on active features” property.

This family is the headline use of skein’s per-feature weights= parameter — the underlying solvers all accept it directly, so adaptive estimators are pure composition with no Rust changes.

Pilot strategy¶

The pilot is a MCPPathRegressor(gamma=1e9) fit (i.e. plain lasso) of length n_pilot_lambdas (default 10). The β at pilot_position (default 'mid' — index n_pilot_lambdas // 2) is the pilot estimate. Other positions:

'last' — smallest λ (closest to OLS for n > p, but unstable for n < p).
An integer index into the pilot path — full control.

The pilot runs on the full data even inside the CV variants — pilot weights are a data-derived hyperparameter, not a model parameter, and re-fitting the pilot per-fold would be a different procedure.

Adaptive lasso — path + CV¶

class skein_glm.adaptive.AdaptiveLassoPathRegressor(*, eta=1.0, eps_pilot=1e-06, n_pilot_lambdas=10, pilot_position='mid', lambdas=None, n_lambdas=100, lambda_min_ratio=0.001, max_iter=100, tol=1e-06, fit_intercept=True, standardize=False, screening='strong', acceleration=5)[source]¶

Bases: _AdaptivePathBase

Adaptive lasso (Zou 2006) along a λ-path with warm starts.

Pilot is a plain lasso fit (MCP at γ = 1e9); final is also lasso with the per-feature inverse-magnitude weights. With pilot magnitudes η-rescaled, the final solve produces the asymptotically unbiased “oracle” sparse solution under the right regularity conditions.

Parameters:

eta (float, default 1.0) – Adaptive-weight exponent. w_j = 1 / max(|β_pilot[j]|, eps)^η.
eps_pilot (float, default 1e-6) – Floor on |β_pilot| to keep weights finite.
n_pilot_lambdas (int, default 10) – Length of the pilot’s auto λ-grid.
pilot_position ({'mid', 'last'} or int, default 'mid') – Which λ along the pilot path to read β from. 'mid' is a good default — the path’s middle is typically a reasonable bias/variance compromise. 'last' is closest to OLS.
lambdas (array-like or None, default None) – Forwarded to the final estimator (MCPPathRegressor at γ = 1e9). See its docstring for the rest of the standard kwargs (n_lambdas, lambda_min_ratio, max_iter, tol, fit_intercept, standardize, screening, acceleration).
n_lambdas (int)
lambda_min_ratio (float)
max_iter (int)
tol (float)
fit_intercept (bool)
standardize (bool)
screening (str)
acceleration (int | None)

coefs_¶

Type:: (n_lambdas, n_features)

intercepts_¶

Type:: (n_lambdas,)

lambdas_¶

Type:: (n_lambdas,)

coef_pilot_¶

Type:: (n_features,) — the β read off the pilot path.

weights_¶

Type:: (n_features,) — the adaptive weights computed from the pilot.

info_¶

Type:: dict

n_features_in_¶

Type:: int

set_fit_request(*, x='$UNCHANGED$')¶

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to fit.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

x (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for x parameter in fit.
self (AdaptiveLassoPathRegressor)

Returns:

self – The updated object.

Return type:

Adaptive estimators¶

Pilot strategy¶

Adaptive lasso — path + CV¶

Adaptive MCP — path + CV¶

Adaptive SCAD — path + CV¶

Adaptive group estimators¶

Adaptive GLMs (Logistic, Poisson, Cox)¶

Adaptive logistic — Path + CV¶

Adaptive Poisson — Path + CV¶

Adaptive Cox PH — Path + CV¶