Describe mixture-of-experts and how it grows model capacity without growing per-token compute.

A sparse MoE router sends each token to a small subset of expert MLPs. Total parameters can grow while active FLOPs per token stay bounded, but routing balance, communication, expert capacity, and dropped tokens become central concerns.

Use equations or tensor shapes where they clarify the claim, then name an experiment or ablation that would distinguish competing explanations.