Explain SwiGLU and why gated activations displaced ReLU/GELU in current LLM feed-forward blocks.

SwiGLU multiplies a SiLU-transformed branch by a learned linear gate before projection. The gate improves expressiveness and optimization, usually at a parameter or width trade-off versus a plain ReLU/GELU MLP.

Use equations or tensor shapes where they clarify the claim, then name an experiment or ablation that would distinguish competing explanations.