Test Constancy and Assess Gaussian Trend-Filtering Prediction

Assess a continuous outcome using order-2 trend filtering, retuned outcome permutations and a separate nested-CV comparison with a constant mean.

Usage

assess.gaussian.trend.filtering(
  object,
  permutations = 999L,
  inner.folds = 5L,
  outer.folds = 5L,
  seed = 1L,
  predictive = TRUE,
  warn = TRUE,
  additional.tests = character()
)

# S3 method for class 'gaussian.trend.filtering.assessment'
print(x, ...)

Arguments

object: A gaussian.trend.filtering.fit with independent, unit-weight observations, no covariates and no offset. Predictor choice, penalty path, ridge penalties and numerical settings must be specified independently of outcomes. The assessment uses order 2 and reuses these explicit settings.
permutations: Number of random outcome permutations, at least one. The default 999 gives a minimum possible p-value of 0.001. Small values in examples illustrate usage only.
inner.folds: Number of balanced, outcome-independent tuning folds. Used for the global test and within each outer training set.
outer.folds: Number of outer folds for predictive assessment.
seed: Nonnegative integer random seed. The caller's RNG state is restored. Realized folds, permutation seeds and RNG kind are returned.
predictive: Whether to run nested CV in addition to the global test.
warn: Warn if the observed fit, any permutation, or an outer fit fails.
additional.tests: Unique optional names "absolute.deviation" and/or "total.variation". Default character() tests squared-error improvement only. Both observed curve summaries are always returned.
x: A gaussian.trend.filtering.assessment to print.
...: Reserved for generic compatibility.

Value

A gaussian.trend.filtering.assessment with the same evidence structure as assess.logistic.trend.filtering, replacing null.probability with null.mean and the primary likelihood test with squared.error. Scalar statistic, p.value, p.value.bounds, status and the permutations table refer to squared-error improvement. fit is the newly tuned observed fit. curve.summaries, tests, and permutation.statistics report curve summaries and requested tests. Fold assignments, seeds, candidate diagnostics and failure messages are retained. Optional prediction includes row-level mean and constant.mean predictions, losses, fold diagnostics, model scores and baseline-minus-smoother improvements. No confidence intervals are supplied.

Details

The randomization null is an outcome distribution that is unchanged across the fixed predictor locations. Independent outcomes must be exchangeable under the null; a constant mean plus identically distributed independent errors is a sufficient model. Normality is unnecessary. This is not a test of constant mean under arbitrary heteroskedasticity. Repeated participants, serial dependence, covariate adjustment and unequal weights require other null procedures and are rejected where detectable. Independence and exchangeability cannot be checked from a fit.

Fresh balanced folds are generated independently of outcomes; stored fit selections and fold labels are ignored. The explicit lambda path is tuned by CV on the full data and again on every shuffled outcome vector, always using the same tuning folds. The primary statistic is max(0, sum((y-mean(y))^2) - sum((y-f.hat(x))^2)), the nonnegative reduction in residual sum of squares, in squared response units. It is not a penalized-objective difference or a chi-squared statistic. Constant outcomes have zero statistics and permutation p-values of one.

With B requested permutations the inclusive plus-one p-value is (1 + sum(permuted.statistic >= observed.statistic))/(B + 1), allowing a conservative numerical tie tolerance of 1e-10 * max(1, abs(observed.statistic)). Retuning is part of each randomization. Failed candidates are ineligible, and a replicate fails if none is eligible. Failed replicates are retained without retries or denominator changes. Any failure suppresses the point p-value; lower and upper bounds count missing statistics as nonexceedances or exceedances. Observed-fit failure leaves both bounds unavailable. These are failure bounds, not confidence intervals.

Absolute deviation is mean(abs(f.hat(x)-mean(y))), averaging over observations with repeated predictors counted at their observed frequency. Total variation sums absolute changes along the continuous fitted curve over [min(x),max(x)], including every knot and interior quadratic turning point. Both are in response units. Total variation is not the third- difference penalty. These describe the newly tuned observed fit; they are not bias-corrected population effects or uncertainty intervals. Optional tests reuse the same fitted permutations, with per-statistic failure bounds and Holm adjustment across squared error and the requested tests. An incomplete family has no Holm p-values. This adjustment does not cover additional predictors or outcomes. Choose tests and penalties in advance.

Nested CV inherits the input fit's cv.penalty.scale. Under "training_mass", outer training fits scale the reference lambda grid and ridge penalties by their sample-size fraction; inner folds apply their own training fraction. Full-data observed and permutation refits retain the reference grid. Older saved objects without this field use "none" to preserve their earlier convention. Nested CV tunes using only each outer training set. Its constant baseline is that set's sample mean. Pooled mean squared error (mse) and mean absolute error (mae) use the same held-out rows; improvements are constant minus smoother loss. Positive values favor trend filtering. The baseline remains the training mean for both scores. Boundary predictions use the fitter's constant-boundary policy and are flagged. Incomplete predictions leave overall smoother scores unavailable. No standard errors are computed from overlapping CV folds. Prediction scores assess training at the outer-training sample size, and need not improve when the global association test rejects.

Runs are sequential, with the same fitting cost and failure policy as assess.logistic.trend.filtering. The test does not establish nonlinearity, causality or a useful predictive effect.

References

Phipson and Smyth (2010). Permutation p-values should never be zero. doi:10.2202/1544-6115.1585

Examples

x <- seq(0, 1, length.out = 40)
y <- sin(2*pi*x) + 0.2*cos(seq_along(x))
fit <- fit.gaussian.trend.filtering(x, y, c(Inf, 0.01), backend = "banded")
# Nine permutations illustrate the API; use more for inference.
assessment <- assess.gaussian.trend.filtering(fit, permutations = 9,
    inner.folds = 3, outer.folds = 3, seed = 42,
    additional.tests = c("absolute.deviation", "total.variation"))
assessment$tests
#>                 test  statistic   status requested completed failed exceedances
#> 1      squared.error 19.3392875 complete         9         9      0           0
#> 2 absolute.deviation  0.6093144 complete         9         9      0           0
#> 3    total.variation  3.6858303 complete         9         9      0           0
#>   comparison.tolerance resolution p.value p.value.lower p.value.upper
#> 1         1.933929e-09        0.1     0.1           0.1           0.1
#> 2         1.000000e-10        0.1     0.1           0.1           0.1
#> 3         3.685830e-10        0.1     0.1           0.1           0.1
#>   p.value.holm
#> 1          0.3
#> 2          0.3
#> 3          0.3
assessment$prediction$improvement
#>       mse       mae 
#> 0.5265733 0.4868482