DeepOffer

Implement scaled dot-product self-attention in NumPy or PyTorch with an optional causal mask.

ML CodingReported interview question
Reported in a public interview report — OpenAI, Meta, AI-lab MLE screens

Project Q, K, and V; reshape to heads; compute QKᵀ/√d; apply causal or padding masks before softmax; multiply by V; then concatenate heads and project. Test tensor shapes and fully masked rows.

Test empty input, one-element input, duplicates, boundary indices, invalid states, and the largest allowed size; state time and space complexity.

Common follow-up questions

Practice this question with an AI interviewer

Get asked follow-ups live, then receive a scored report — like a real MLE interview loop.

Start AI mock interview