Escha Labs ships 2-bit Qwen3.6-35B-A3B build that runs on a 24GB GPU

Running a 35-billion-parameter language model on a single consumer GPU might sound like a distant dream, but a new 2-bit quantization from Escha Labs is making it a practical reality. The company has released Escha-W2, a 2-bit build of Qwen3.6-35B-A3B, a Mixture-of-Experts model with 256 experts, that weighs just 12.3 GB on disk. The quantized model runs on a single 24 GB GPU, and even on 16 GB cards like the RTX 5060 Ti, the runtime can trade context length or concurrency to fit in memory.

The repo on Hugging Face ships only the model weights; the actual runtimes live in a separate repository, EschaLabs/escha-runtime-qwen3moe. Two engines are provided. The recommended sglang engine is a Python-based server that supports concurrency, tool calling, structured output, and a reasoning parser. It is the engine used for all the performance and quality numbers on the model page. The alternative zml engine is a single binary with no Python dependencies, ideal for single-user setups that want long prompt generation without a Python environment; however, it requires a full 24 GB of VRAM and can fail on 16 GB cards with long prompts.

The install process involves installing the Escha runtime, downloading the model, and running a serve script. For those integrating with existing tools, the server speaks the OpenAI-compatible API, so anything from LM Studio to the openai Python package can plug in. There is also a ready-made opencode.json for users of the opencode coding agent.

The key selling point of a 2-bit quant is how much quality it preserves. Escha Labs provides a KL divergence measurement of 0.0694 nats against the BF16 baseline over 524,032 held-out tokens, meaning the model picks the same next token 9 out of 10 times. In an apples-to-apples comparison against an FP8 baseline (since BF16 doesn't fit on 24 GB), the 2-bit build stays within noise on most benchmarks, with the only significant gap in long-horizon code generation on LiveCodeBench. Coding parity is corroborated by near-ties on HumanEval+ and CRUXEval-O. However, there are important caveats: the MMLU-Pro score is conservative because a 13.9% of thinking-on questions hit the 4096-token reasoning budget, and both the test and baseline suffer equally. Also, the model is text-only despite the config declaring a vision tower.