Design a 70B+ model serving stack for hundreds of millions of assistant users and explain where the money goes.

Define latency, throughput, context, quality, and cost SLOs; route requests by model capability, batch continuously, manage KV memory, and stream output. Add admission control, fallbacks, versioned prompts, safety filters, canaries, and token-level telemetry.

Make interfaces and ownership explicit; add versioning, access control, monitoring, canary rollout, rollback, and a plan for delayed labels or human review.