Baltor Get started

Library / Code module

Silent concept-shift twin datasets

silent_concept_shift_twins

Generate two datasets whose inputs and targets have exactly the same marginal distributions while the input-target relationship is reversed, and compare them with input-only, target-only and joint statistics. Use it to show that a drift monitor reading inputs alone cannot detect a change of concept.

Files, names only

  • ATTRIBUTION.md 3.9 KB
  • LICENSE 1.0 KB
  • README.md 2.7 KB
  • atom_cli.py 3.0 KB
  • component.json 8.7 KB
  • numerics.py 28.0 KB
  • schema_lite.py 8.9 KB
  • silent_concept_shift_twins.py 6.9 KB
  • test_package.py 12.0 KB
  • test_silent_concept_shift_twins.py 4.4 KB

Full files in your harness.

What it takes and gives

Runs with
Python 3.10 or later
Entry points
generate_twins, input_only_statistics, target_only_statistics, joint_statistics, silent_shift_report
Known answers
8 cases its tests reproduce

Cases it covers

  • relationship is not unchanged: An input-only monitor reports no drift on these twins, yet every input-target covariance has reversed sign.

Limits and scope

Limits
A linear relationship with Gaussian inputs and noise; real concept shifts are rarely this clean. The construction needs inputs symmetric under negation; it does not cover shifts that also move the input distribution. Statistics are basic (mean, variance, Kolmogorov-Smirnov, covariance); nothing here estimates the power of a particular drift detector.
Guarantees
The two datasets hold identical multisets of input rows and identical multisets of targets, so every input-only or target-only statistic is equal on both. Every input-target covariance of the shifted twin is exactly the negative of the reference one. With labels the label multisets also match and the covariances still flip sign.
Assumes
The twins are a constructed fixture, not a model of any particular real shift. Seeded draws use random.Random.random() only, so a seed gives the same twins on every Python version.

Details

Kind
Code module
Licence
MIT
Declared effects
runs commands
Source
github.com/alisonjieli-png/loop-engine/tools/creative_originals/evaluation_atoms/items/silent_concept_shift_twins
Size
10 files, 79.5 KB
Identity
library.supply.creative_originals.74a90e1d8357d3e7fc9991b4.7daecd2fa08fee3b
Version
7daecd2fa08fee3b0f3322310944ac46c4ccaead3bb94bdfdd7bc144b0f73fac

The version is the SHA-256 of the item your harness downloads; every file in it names its own checksum.

Where it is shown

Related components

Something wrong with this item, its rights or its safety? Report it on the support page, or from your harness after a download.