ShiftWing: an inference engine built from scratch
Why ShiftWing manages graphics memory, system memory and fast storage as one runtime instead of treating local inference as a smaller copy of server inference.
ShiftWing is a from-scratch inference engine for large mixture-of-experts systems on consumer hardware. It is built around the fact that the complete weights do not belong in one kind of memory.
The runtime
- Frequently useful experts are learned and retained.
- Cold experts stream from fast storage when routing selects them.
- Processor and accelerator backends share the same correctness contract.
- The server supports familiar chat, tool, streaming and cancellation behaviour.
The research instrument
The engine and the research log grew together. Optimisations are retained only after controlled comparisons and output checks; failures remain in the record so later work does not inherit a rewritten history.
Cite this page: Dingal AI Research. “ShiftWing: an inference engine built from scratch”. 24 August 2026.
Related
Why frontier AI has to run on the hardware people own
Frontier capability matters most when regular people can use it without surrendering their work, context or control to a remote service.
Pushing experts below four bits
Low-bit expert sidecars reduced storage pressure, but the quality study made clear that this is a trade to qualify rather than a free saving.
Speculative decoding that changes no output
A speculative path was admitted by confidence, verified against the ordinary path and rejected whenever it could not preserve the exact result.