Skip to content

Reproducing the measurements

The figures quoted in the guide, in ../why.md and in ../reference.md come from these. They are kept out of the test suite because they take minutes rather than seconds.

script what it measures
quadrature_accuracy.py Gauss-Legendre against Gauss-Hermite and Monte Carlo on relu³, whose moments are known exactly
gaussian_breakdown.py How far a high moment departs from the Gaussian prediction with width, and whether the Gamma correction tracks it
stability_table.py χ and σ* for every builtin activation and for relu^p
gain_table.py E[a²] through a 20-layer MLP under the usual gain table and under both AnyInit modes
draw_drift.py How far one draw of a deep ReLU MLP lands from the analytic prediction, with and without center=True, against the empirical mode
depth_sweep.py Measured rescaling across depth and width for relu^p, showing which degrees survive

Run any of them directly:

python docs/experiments/stability_table.py

depth_sweep.py, draw_drift.py and gain_table.py need PyTorch; the others need only NumPy.

Tied embeddings keeps its script inline instead: it needs a GPU and a dataset, and it justifies one decision rather than a figure.