#distributed-inference

Lumabri Lets You Run Huge Mixture-of-Experts Models Across a Swarm of Peer Machines

A new pure-C, dependency-free system pools CPU and disk from any machines on a network to serve massive MoE models, with byte-identical output and no upfront downloads.