Library / Code module
Effective rank, participation ratio and stable rank
effective_rank_spectral_entropy
Measure how many directions a representation matrix actually uses: the effective rank (exponential of the entropy of normalized singular values), the participation ratio of the eigenvalues and the stable rank. Use it to detect collapsed embeddings or compare the dimensionality of learned representations.
ATTRIBUTION.md3.9 KBLICENSE1.0 KBREADME.md2.2 KBatom_cli.py3.0 KBcomponent.json8.9 KBeffective_rank_spectral_entropy.py3.0 KBnumerics.py28.0 KBschema_lite.py8.9 KBtest_effective_rank_spectral_entropy.py3.3 KBtest_package.py12.0 KB
Full files in your harness.
What it takes and gives
- Runs with
- Python 3.10 or later
- Entry points
- spectral_dimensions, singular_values
- Known answers
- 8 cases its tests reproduce
Cases it covers
- not the eigenvalue entropy: Taking the entropy of the eigenvalues sigma^2 instead of the singular values gives 2.38, a different measure than the effective rank.
Limits and scope
- Limits
- Dense Gram matrices: the smaller dimension should stay below a few hundred. The measures summarize a spectrum; they do not say which directions matter for a task, and the cutoff for zero singular values affects nearly rank-deficient matrices.
- Guarantees
- With k equal non-zero singular values, the effective rank, participation ratio and stable rank all equal k; a rank-one matrix gives 1. All three are invariant to orthogonal transformations of rows or columns and to scaling the whole matrix. 1 <= effective rank <= numerical rank.
- Assumes
- The scale of each feature is meaningful; the measures change if features are rescaled unequally. Singular values at or below 1e-12 times the largest are treated as zero.
Details
- Kind
- Code module
- Licence
- MIT
- Declared effects
- runs commands
- Source
- github.com/alisonjieli-png/loop-engine/tools/creative_originals/evaluation_atoms/items/effective_rank_spectral_entropy
- Size
- 10 files, 74.4 KB
- Identity
library.supply.creative_originals.1bf44a7f3f379f065daff8ba.83a7496600633927- Version
83a749660063392759e9a844f83c283be68177b614a05b64a5bf2bb104974ec8
The version is the SHA-256 of the item your harness downloads; every file in it names its own checksum.
Where it is shown
- Statistics for comparing resultsDeterministic skills
Related components
Something wrong with this item, its rights or its safety? Report it on the support page, or from your harness after a download.