The paper asks whether a hyperdimensional classifier whose learned parameters are low-bit angles (a 'phase memory') can be trained while storing nothing but the model, instead of also keeping an optimizer's records of past gradients. It proposes Phase-HDC, which turns each stored angle by at most one notch per update, against the sign of the current minibatch gradient and only when the gradient magnitude exceeds a threshold, and proves this rule is the exact minimizer of a first-order loss model with a fixed cost per changed parameter. In a controlled experiment on scikit-learn handwritten dig

Interactive companion · Nebli, arXiv 2026

Training a model usually needs a second, hidden memory: the optimizer’s running record of past gradients. For a classifier built from 6-bit dials, that record is several times larger than the model. Phase-HDC throws it away. Each dial turns one notch against its gradient — and only when the push is strong enough. Here, 5,824 such dials are learning to read handwritten digits, live in your browser.

Dial angles (teal) · 1st moment (coral) · 2nd moment (blue). Logical array state as counted in the paper, not measured device memory.

this run paper: mean of 5 replayed runs

—

—

—

Tap or click any dial in the bank to pin it.

Filled dots: runs in this page (own random seeds; ◇ = settings other than the paper’s). Ticks: the paper’s five seeds in its controlled test (Table 15).

01

Why one notch is exactly right

At every update each dial has three options: turn one notch back, stay, or turn one notch forward. Phase-HDC prices them with two ingredients — the gradient g, which predicts how the loss would change (to first order, g × the move), and a fixed write cost τ charged to any dial that changes — and picks the cheapest. The paper proves (Proposition 1) that the winner is always the same simple rule: move against the sign of g if |g| > τ, otherwise stay.

Drawn with 16 notches so one notch is visible; the model uses 64.

q+ = ( q − sign(g) · 1[ |g| > τ ] ) mod K

τ = 0 gives plain sign descent: every dial with a non-zero gradient moves.

A threshold is not a budget. How many dials turn is decided by the gradients, not fixed in advance.

Nothing persists. The rule reads only this update’s gradient, so the dials are the entire training state: Γ = 1.

The proposition justifies the form of one step on one minibatch; on its own it does not guarantee convergence.

02

How 5,824 angles read a digit

For every pixel, the classifier takes two angles — one for where the pixel is (table P) and one for how dark it is (table V) — and adds them. That is binding . Then it lays the 64 resulting arrows head to tail. That is bundling : arrows that agree build a long sum, scattered ones cancel. It does this in 64 dimensions at once and scores each digit by how strongly the 64 sums line up with that digit’s own row of angles (table ψ).

Draw on the grid to make your own digit.

—

Score sc = |Σd hd · w̄c,d| / D, then a softmax with temperature T = 0.3. Model: the author’s released Phase-HDC checkpoint (seed 1), 92.78% on the 360 test digits.

03

What Adam’s memory was really doing

Adam keeps two running statistics per parameter: a running mean of gradients (the first moment, like the velocity of a rolling ball) and a running mean of squared gradients (the second moment, a per-parameter volume control). Held to the same dials without a high-precision copy, projected Adam must round its step η·r to whole notches — so it, too, becomes a gate: a dial moves only if |η r| ≥ ½. In a steady state r ≈ sign × 1/√(1 + S−2), where S is the signal-to-noise ratio of that dial’s gradient. The two moment arrays decide whether to move, from history; Phase-HDC makes the same decision from the size of the current gradient.

Eq. 12: the gate opens when S ≳ 1/√(4η² − 1). All Adam arms in the controlled test selected η = 3, the top of their grid — on a grid, a small learning rate does not make steps smaller; it switches them off.

Instrumented replays of the 20 final controlled runs (Figure 7a), mean of five seeds. Phase-HDC writes to about 72% of dials per update until ≈ update 70, then falls below 5% as its training loss drops from 0.028 to 0.009. Adam’s gate closes earlier, while its loss is still higher: final training loss 0.036 (float32 moments) vs 0.0081 for Phase-HDC — yet test accuracy ends at parity. Different routes, same generalisation.

0 of 5,824

With η = 0.3, projected Adam (float32 moments) changed none of its stored indices in 180 updates. Validation accuracy: 9.7% — chance.

÷ scale

What the moments do buy is scale invariance : r ignores the gradient’s overall size, while τ is measured in gradient units. Phase-HDC therefore needs its own τ per table, per dataset and per bit width — the main practical cost of keeping no state.

04

Where 6-bit history breaks: the rare values

Squeezing Adam’s moments into 6-bit codes with one range per table (per-tensor affine quantization) has a specific failure. In the digits, intensity 0 fills 49% of pixel slots and intensity 16 fills 9%; each of the other fifteen intensities fills only 2–4%. Their second moments are tiny next to the table’s largest entry, fall below the first quantization step, are stored at the lowest code and read back as almost zero. Dividing by that near-zero makes r explode, and the dial jumps many notches — landing anywhere on the circle.

Labels: share of pixel slots · share of that row’s dials jumping > 3 notches per update. Animated at these per-update rates, measured in the paper’s replays (projected Adam, 6-bit moments). Dial size ∝ how often the intensity occurs.

Position rows are immune: every position occurs exactly once in every example, so their second moments are balanced by construction. On digits the damage stays partial (91.22% test accuracy), but the same mechanism is consistent with per-tensor Adam’s collapses elsewhere: 8-bit moments fail on byte-level enwik8 (256 symbols, many rare; 0.78%), yet stay within about a point of float32 Adam on text8 (27 common symbols). The paper did not measure symbol frequencies on those runs, and names a remedy it has not tested: block-wise ranges, or one range per value row.

05

The price list

Removing history costs different amounts against different competitors. Pick a dataset and set a memory budget: systems that keep more state than the budget fade out, and the best one that still fits is named.

16–23×

less state than float32 Adam across eleven datasets, at an average cost of 5.3 accuracy points (7.0 on image and tabular data, 3.3 on text).

6 of 11

datasets where Phase-HDC beats per-tensor 8-bit Adam while keeping 4–6× less state — including enwik8, where 8-bit Adam collapses to 0.78%.

encoder

On digits, most of the gap to float32 Adam comes from its continuous parameters , which accumulate many small updates between notches — not from its moments.

06

How many notches?

The bit width k sets both the alphabet of the model and the geometry of its update. Adding notches makes every turn finer and the dial harder to turn: two more bits shrink each step to a quarter of the angle, and a fixed threshold becomes four times stricter in angle (Eq. 6). Coarse notches are blunt in a different way: one notch on a shared value row rotates every pixel holding that value at once.

The same threshold grid was reused at every width, so this sweep changes representation, step and gate together. On digits, extending the grid at k = 8 raised accuracy from 86.22% to 93.81%. A dense 2-bit table reached 96% on digits in another study, yet one-notch training at k = 2 stayed near chance: an optimisation limit, not a capacity limit.

4-bit math

Rounding the softmax arithmetic to 4 bits flips about a third of gradient signs at a frozen state, yet training still ends at 94.81 ± 0.26% vs 94.96 ± 1.17% in float (simulated; Table 11).

dense wins

Generating the item memory from one learned frequency per attribute reached 38–72%; a dense table of independent angles reached 94–96% (Table 9). Every value needs its own free code.

07

What the numbers measure

Logical state

Memory figures count ideal packed parameter and moment arrays. The experiments store codes in floating-point arrays and compute in floating point; peak memory, latency and energy were not measured.

Controlled test

One small dataset, one stratified split, five seeds, validation-only tuning. The paired interval [−0.44, 2.89] points includes zero: it supports parity, not superiority.

Eleven datasets

Exploratory: tuning reused the reported evaluation data, and the Adam arms there use a free complex class head, so they compare complete training systems.

Next steps named

A packed integer implementation on a microcontroller; a threshold set each step from a quantile of |g| (still no persistent state); comparisons with block-wise 8-bit Adam, per-row ranges and Adafactor.

08

Sources and credits

Paper. Ahmed Nebli (2026). “Phase-HDC: Replacing Optimizer History with Gradient Thresholds in Discrete Phase Learning.” arXiv preprint arXiv:2610.10630 [cs.LG]. arxiv.org/abs/2610.10630 — licensed CC BY 4.0. Charts on this page are redrawn and restyled from its Tables 3–8 and 14 and Figures 1, 2 and 7.

Code and released records. github.com/ahmednebli/Phase-HDC (MIT License). The in-browser trainer is a JavaScript port of its NumPy reference (src/phase_hdc_numpy.py); given the same initial tables and minibatch order it reproduces the released seed-1 checkpoints exactly for Phase-HDC and for float32-moment projected Adam. Replay curves and value-row statistics come from results/diagnostics; the digit reader uses results/validation/sign_seed1.npz.

Data. scikit-learn “digits”, a copy of the UCI Optical Recognition of Handwritten Digits test set (E. Alpaydin and C. Kaynak, 1998; CC BY 4.0), with the paper’s train / validation / test split (1,077 / 360 / 360).

Live runs. Runs in the bank use their own random seeds, so their accuracies vary around — not exactly at — the paper’s reported runs. Adam’s per-tensor quantizer follows the paper’s Eqs. 15–16.