Pushing experts below four bits
Low-bit expert sidecars reduced storage pressure, but the quality study made clear that this is a trade to qualify rather than a free saving.
Routed experts dominate the cold weight store, so they are the natural target for lower-bit representations. The experiment separated storage success from numerical and behavioural acceptance.
Method
- Generate low-bit sidecars atomically and bind them to their source artifacts.
- Check loader and execution equivalence at the representation boundary.
- Measure sensitivity by layer and prompt rather than relying on one aggregate.
- Freeze the acceptance rule before reading the result.
Findings
The representation and execution path worked, and the storage reduction was real. Quality loss was uneven: some prompts and layers tolerated the change better than others, so an aggregate could hide the weakest cases.
Decision
Cite this page: Dingal AI Research. “Pushing experts below four bits”. 28 July 2026.
Related
A negative result worth publishing
A larger target missed the threshold written down before the run, so the release decision stayed negative even though parts of the result looked promising.
ShiftWing: an inference engine built from scratch
Why ShiftWing manages graphics memory, system memory and fast storage as one runtime instead of treating local inference as a smaller copy of server inference.
Treating VRAM, RAM and NVMe as one memory hierarchy
Experiments with direct storage, learned expert locality and predictive prefetch turned three separate resources into one managed path.