NVIDIA MLE Interview Questions (2026)

Questions reported from NVIDIA machine learning engineer interviews. This bank tracks 21 questions asked at NVIDIA (15 with free model answers). Practice them live with DeepOffer's AI voice interviewer.

← All 465 questions

Practice this question with an AI interviewer

Get asked follow-ups live, then receive a scored report — like a real MLE interview loop.

Start AI mock interview

#5 Compare Plain SGD, Momentum, and Adam—Where Can Adam Fall Short? ★
#43 Describe the KV cache, derive its memory formula, and discuss what it costs at production scale. ★
#126 Explain strategies for combating overfitting in tree-based classification models.
#128 A network confuses pugs with pit bulls and some training labels are wrong — how do you change the model and the data?
#129 Explain how to evaluate a clustering model when no pre-labelled groups exist.
#165 Contrast the prefill and decode phases and explain why prefill is compute-bound while decode is memory-bandwidth-bound. ★
#166 Explain continuous (in-flight) batching and why it replaced static batching. ★
#167 Explain PagedAttention and the KV-cache fragmentation problem it solves. ★
#168 Explain speculative decoding: why output quality is preserved and when it does not help. ★
#170 Compare FP16, BF16, FP8, INT8, INT4, and FP4 for serving and name what breaks at each step down. ★
#171 Compare tensor, pipeline, data, sequence, and expert parallelism and explain when to combine them. ★
#172 Estimate the GPU memory needed to serve a 70B model, accounting for weights, KV cache, activations, and fragmentation. ★
#174 Do the roofline math: how many tokens/sec can one H100 produce for a 70B model at batch size 1? ★
#175 Decide when to choose vLLM, SGLang, TensorRT-LLM, or a custom stack. ★
#176 Enumerate and rank every lever for cutting LLM serving cost by 10x. ★
#190 Design the parallelism strategy for serving a 405B dense model: where TP, PP, and EP go and why. ★
#193 Walk through serving a 70B model on a single 80GB GPU: whether it fits and the expected single-stream tokens/sec. ★
#194 Explain what TensorRT/TensorRT-LLM actually does to speed up a model and when it will not help. ★
#311 Implement the block manager for a paged KV cache: allocate, append, free, and copy-on-write prefix sharing.
#354 A model runs fine in FP32 but produces garbage after conversion to FP16 — debug it.
#355 Debug a CUDA kernel running 10x slower than expected, without running it: name the usual suspects and how to confirm each.