Clarify how cross-entropy, KL divergence, and perplexity relate, and why cross-entropy is the standard LM training loss.

ML TheoryReported interview question

Reported in public interview compilations — OpenAI

Premium question

This is a premium question from DeepOffer's 2026 ml theory interview bank. Members get the full model answer — what interviewers are really testing, the step-by-step reasoning, common mistakes to avoid — plus AI voice mock interviews with live follow-ups and a scored report.

Unlock the full model answer

Join DeepOffer to unlock this and 347 more premium questions.

Unlock premium bank

Practice this question with an AI interviewer

Get asked follow-ups live, then receive a scored report — like a real MLE interview loop.

Start AI mock interview

Clarify how cross-entropy, KL divergence, and perplexity relate, and why cross-entropy is the standard LM training loss.

Unlock the full model answer

Practice this question with an AI interviewer

Related interview questions