Baltor Get started

Library / Code module

Split conformal prediction intervals

split_conformal_intervals

Turn any point predictor into prediction intervals with a finite-sample coverage guarantee, using held-out calibration residuals and the exact ceil((n + 1)(1 - alpha)) quantile index, optionally per group (Mondrian), with a seeded coverage check on held-out data.

Files, names only

  • ATTRIBUTION.md 3.9 KB
  • LICENSE 1.0 KB
  • README.md 2.8 KB
  • atom_cli.py 3.0 KB
  • component.json 9.3 KB
  • numerics.py 28.0 KB
  • schema_lite.py 8.9 KB
  • split_conformal_intervals.py 6.6 KB
  • test_package.py 12.0 KB
  • test_split_conformal_intervals.py 3.8 KB

Full files in your harness.

What it takes and gives

Runs with
Python 3.10 or later
Entry points
conformal_quantile, split_conformal, coverage, coverage_simulation
Known answers
8 cases its tests reproduce

Cases it covers

  • no uncorrected quantile: The plain empirical quantile uses index ceil(n (1 - alpha)) = 9 and under-covers; the finite-sample index is ceil((n + 1)(1 - alpha)) = 10.

Limits and scope

Limits
Marginal coverage only: it does not guarantee coverage for each input value, and under distribution shift the guarantee is lost. Absolute residuals give symmetric intervals of constant width per group. With few calibration points the interval can be unbounded. The simulation is a seeded consistency check on one synthetic model.
Guarantees
P(y in interval) >= 1 - alpha marginally over calibration and test draws; below 1 - alpha + 1/(n + 1) when residuals have no ties. The index k = ceil((n + 1)(1 - alpha)) is computed exactly from the decimal value of alpha; k > n gives an unbounded interval. Mondrian intervals give the same guarantee within each group.
Assumes
Calibration and test points are exchangeable (for Mondrian intervals: exchangeable within each group). The predictor was fitted on data disjoint from the calibration set.

Details

Kind
Code module
Licence
MIT
Declared effects
runs commands
Source
github.com/alisonjieli-png/loop-engine/tools/creative_originals/evaluation_atoms/items/split_conformal_intervals
Size
10 files, 79.5 KB
Identity
library.supply.creative_originals.d14ec4f066a67babbfed2113.a12faaa2ff0abd67
Version
a12faaa2ff0abd670c73e4b22097fb40239805ed4b9956abdec6a968d4b00d01

The version is the SHA-256 of the item your harness downloads; every file in it names its own checksum.

Where it is shown

Related components

Something wrong with this item, its rights or its safety? Report it on the support page, or from your harness after a download.