Nested Cross-Validation for LPS with Explicit, Recorded Folds

Runs outer-fold nested cross-validation around fit.lps(): for each outer fold, candidate selection is one ordinary fit.lps call on the inner-training rows only, with an explicit inner fold id and X.eval set to the held-out outer-test rows, so the held-out fold never participates in inner selection. The pooled outer-test error is the nested generalization estimate. The same call also computes the selected-min arm — an ordinary fit.lps on all rows using the same outer.foldid — so optimism comparisons are paired on one fold assignment by construction.

Usage

lps.nested.cv(
  X,
  y,
  outer.foldid,
  fit.args = list(),
  inner.folds = 5L,
  cluster.id = NULL,
  inner.foldid.method = c("round.robin", "grouped"),
  inner.shuffle.seed = NULL
)

Arguments

X: Numeric matrix of observations (rows) used for training.
y: Numeric response vector with length(y) == nrow(X).
outer.foldid: Positive integer vector of length nrow(X) assigning rows to outer folds (at least two distinct folds).
fit.args: Named list of additional arguments passed to every fit.lps call (grids, backend, design settings, ...). Must not contain X, y, foldid, or X.eval: the fold plumbing is owned by this function so both arms see the same folds.
inner.folds: Number of inner selection folds (integer >= 2).
cluster.id: Optional cluster labels (length nrow(X)), required for inner.foldid.method = "grouped".
inner.foldid.method: Inner fold construction: "round.robin" (default) or "grouped" (cluster-respecting).
inner.shuffle.seed: Optional integer; per-fold offset seed for the randomized variants described above. Recorded in the return value.

Value

A list of class "lps_nested_cv":

nested.rmse: pooled RMSE of the outer-test predictions over all rows with finite predictions.
n.missing.predictions: count of non-finite outer-test predictions (e.g. unstable.action = "na" fallbacks).
folds: one row per outer fold: fold label, test size, selected support.size / degree / kernel / bandwidth.multiplier, the inner selected-min CV score, and the fold's outer-test RMSE.
predictions: length-nrow(X) vector of outer-test predictions (each row predicted by the fold that held it out).
selected.min: the paired selected-min arm on the same outer.foldid: selected (the fit.lps selected row), cv.score (its observed CV RMSE), foldid, and fit (the full-data fit.lps object, usable for deployment on an external test set).
outer.foldid, test.index, train.index, inner.foldid, inner.foldid.used, inner.cv.table: complete fold/index telemetry: the realized inner fold ids as constructed and as recorded by each inner fit.lps ($foldid), per-fold index sets, and each fold's full inner CV candidate table.
outer.cluster.whole: TRUE/FALSE whether every cluster lies wholly inside one outer fold (NA when cluster.id is absent).
inner.folds, inner.foldid.method, inner.shuffle.seed, fit.args, call: the recorded configuration.

Details

The existing fit.lps behavior is untouched: this utility consumes the public API with explicit foldid only. It currently supports outcome.family = "gaussian" (the default) and scores with RMSE.

Inner folds are built per outer fold and fully recorded in the return value: "round.robin" assigns rep_len(1:inner.folds, n) over the inner-training rows in row order (deterministic; if inner.shuffle.seed is supplied, fold k — by position — uses set.seed(inner.shuffle.seed + k) and permutes the balanced assignment, giving a reproducible randomized variant); "grouped" builds cluster-respecting inner folds with lps.grouped.foldid() on the inner-training rows of cluster.id (passing the same per-fold offset seed when inner.shuffle.seed is supplied).

Examples