Skip to content
Tutorials Center
My learning

Topics

Lesson 62 of 63 ยท Practice and Review

NLP Interview Questions

28 common NLP interview questions with concise answers: tokenization, TF-IDF, embeddings, metrics, transformers, LLMs and RAG, plus runnable code questions.

Natural language processing interviews usually test two things: whether you understand the ideas well enough to choose between methods, and whether you can explain a trade-off in plain words. This page collects 28 questions that come up again and again, grouped by level and topic, with short answers you can expand in your own voice.

Try to answer each question out loud before you open it. If you cannot explain an idea in two or three sentences, follow the link and review the lesson. Four questions come with a small editor so you can check a claim by running it, which is exactly what you can do in a real interview when you are asked to reason about code.

Tip: a short correct answer plus a tiny example beats a long recital. Whenever you name a method, add when it fails ("TF-IDF ignores word order, so it cannot tell dog bites man from man bites dog"). That habit is what separates a mid-level answer from a junior one.

What interviewers look for

Concepts Why each step exists.
Trade-offs Speed, data, accuracy, cost.
Evaluation The right metric for the problem.
Practice Small, correct code.
1 / 2

Fundamentals

What is NLP, and why is it hard?

NLP is the field of building programs that process human language: classifying, extracting, translating, answering and generating text. It is hard because language is ambiguous (I saw her duck), depends on context and world knowledge (it in the trophy does not fit in the suitcase because it is too big), is full of variation (typos, slang, dialects) and keeps changing. See What is NLP.

What are the typical steps of a classic NLP pipeline?

Collect and clean the text, tokenize it, optionally normalize (lowercase, stem or lemmatize, remove stop words), turn tokens into numeric features (bag of words, TF-IDF, embeddings), train or apply a model, and evaluate it. Modern transformer pipelines shrink the middle: the model has its own subword tokenizer and learned representations, but cleaning, evaluation and error analysis remain. See text pipelines.

What is the difference between stemming and lemmatization?

A stemmer chops suffixes with rules, so it is fast and crude: studies becomes studi, and the result may not be a word. A lemmatizer uses a dictionary and, ideally, the part of speech to return a real base form: studies becomes study, and better can become good. Use stemming when speed and recall matter (search), lemmatization when you need readable, accurate forms. See stemming and lemmatization.

When should you not remove stop words?

When the small words carry meaning. Negations (not, no) flip sentiment, and prepositions matter in phrases such as to be or not to be. Transformer models also expect natural text and are hurt by stop-word removal. Removal helps most with bag-of-words models on topic-like tasks, where those words only add noise. See stop words.

What is tokenization, and why do modern models use subwords?

Tokenization splits text into units. Word-level vocabularies are huge and cannot represent unseen words (the out-of-vocabulary problem), while character-level sequences are very long. Subword methods such as BPE and WordPiece sit between the two: frequent words stay whole and rare words break into reusable pieces, so any string can be encoded with a fixed vocabulary. See subword tokenization.

Representations and classic models

Explain bag of words and its main weakness.

A bag of words represents a document as a vector of word counts over a fixed vocabulary. It is simple, fast and a strong baseline. It throws away word order and context, so dog bites man and man bites dog get identical vectors, and it creates very wide, sparse vectors. N-grams recover a little local order. See bag of words.

How does TF-IDF work, and why is the logarithm there?

TF-IDF multiplies how often a word occurs in a document (term frequency) by how rare it is across the collection (inverse document frequency, log(N / df)). Words in every document get weight near zero; words that are frequent here and rare elsewhere get the largest weights. The logarithm damps the scale: a word that is ten times rarer is more informative, not ten times more. See TF-IDF.

Why use cosine similarity instead of Euclidean distance for text?

Cosine looks at the angle between vectors, so it ignores their length. A long document that repeats the same topics as a short one still points in the same direction, while Euclidean distance would call them far apart. On length-normalized TF-IDF vectors the two are monotonically related, but cosine is the convention. See document similarity.

What is feature hashing, and what do you give up?

Instead of storing a vocabulary, a hash function maps each token to one of a fixed number of columns. Memory is constant and streaming is easy, with no vocabulary to build. You give up the ability to look up which word a column stands for, and different words can collide in one column. See feature hashing.

Why does Naive Bayes work well for text although its assumption is false?

It assumes words are independent given the class, which is clearly untrue. But for classification you only need the correct class to get the highest score, not accurate probabilities, and the independence assumption makes training a single pass over counts with very little data. Its probabilities are usually badly calibrated. See Naive Bayes.

Why is smoothing needed in n-gram models and Naive Bayes?

Without smoothing, any word or n-gram never seen in training has probability zero, which makes the whole product zero. Smoothing (add-one, backoff, interpolation) moves some probability mass to unseen events. See smoothing and perplexity.

What is perplexity?

Perplexity is the exponent of the average negative log probability a language model gives to a text. Intuitively it is the effective number of equally likely choices the model is torn between at each step: lower is better. A model that guesses uniformly among 4 words has perplexity 4.

Evaluation

Why can accuracy be misleading?

On imbalanced data a model that always predicts the majority class scores high while being useless. If 95 percent of messages are not spam, predicting "not spam" always gives 95 percent accuracy and finds no spam. Use precision, recall, F1 and a confusion matrix. See evaluating classifiers.

Define precision, recall and F1. When would you favor each?

Precision is the share of predicted positives that are correct; recall is the share of real positives that you found; F1 is their harmonic mean. Favor precision when false alarms are costly (blocking a legitimate email), recall when misses are costly (missing a safety report). The harmonic mean punishes a big gap between the two.

Macro, micro and weighted averages: what is the difference?

Macro averages the per-class scores, so every class counts equally and rare classes matter. Micro pools all decisions before computing the score, so large classes dominate. Weighted averages per-class scores by class size. On imbalanced data, report macro F1 next to accuracy.

What is data leakage in text projects?

Information from the test set influencing training. In NLP the classic case is fitting a vectorizer (vocabulary, IDF, stop-word frequencies) on all the data before splitting, or having near-duplicate documents on both sides of the split. Fit every preprocessing step inside a Pipeline on the training fold only. See scikit-learn pipelines.

Embeddings and neural models

What is a word embedding, and how is it different from one-hot encoding?

An embedding is a short dense vector (for example 300 numbers) learned so that words used in similar contexts get similar vectors. One-hot vectors are as long as the vocabulary and every pair of words is equally far apart, so they say nothing about meaning. See word embeddings.

What is the distributional hypothesis?

"You shall know a word by the company it keeps": words that appear in similar contexts tend to have similar meanings. Co-occurrence counts, SVD and Word2Vec are all ways of turning this idea into vectors. See distributional semantics.

What problem do static embeddings like Word2Vec have?

One vector per word, whatever the context, so bank (river) and bank (money) share a vector. They also copy biases present in the training text. Contextual models such as BERT produce a different vector for each occurrence.

Why did Transformers replace RNNs and LSTMs?

RNNs process tokens one by one, which cannot be parallelized and makes long-range dependencies hard to learn (vanishing gradients). Self-attention lets every token look at every other token directly, trains in parallel on GPUs and scales to huge data. The cost is attention that grows quadratically with sequence length. See attention and the Transformer.

What is the difference between encoder, decoder and encoder-decoder models?

Encoders (BERT) read the whole text in both directions and suit classification, tagging and retrieval embeddings. Decoders (GPT-style) predict the next token and suit generation and chat. Encoder-decoder models (T5, translation models) read an input and generate a different output. See pre-trained models.

What does temperature do when an LLM generates text?

The model's raw scores (logits) are divided by the temperature before the softmax. Low temperature sharpens the distribution toward the most likely token, high temperature flattens it and adds variety. Top-k and top-p (nucleus) sampling instead restrict the choice to the most likely tokens. See LLMs and prompting.

What is RAG, and why use it instead of fine-tuning?

Retrieval-augmented generation first retrieves relevant passages from your own documents and puts them in the prompt, so the model answers from fresh, checkable sources. It is cheaper to update (change the index, not the model), can cite sources and reduces hallucinations. Fine-tuning is better for changing the model's style or behavior, not for injecting facts. See retrieval and RAG.

What is a hallucination and how do you reduce it?

A fluent answer that is not supported by facts. The model is trained to produce plausible text, not true text. Mitigations: ground answers in retrieved sources, ask for citations, allow "I don't know", lower the temperature, and verify outputs with checks or a second source.

Practical and coding questions

Your classifier scores 95 percent accuracy. How do you check whether that means anything?

Compare against a majority-class baseline and look at the per-class report. The next editor shows how an empty model can reach that number.

Python Accuracy of a model that never predicts spam

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The accuracy is 0.95 while recall and F1 are 0.0: the model never finds a single spam message.

What happens to words in the test set that were not in the training vocabulary?

A fitted TfidfVectorizer or CountVectorizer silently ignores them, because it only has columns for training words. That is correct (no leakage) but means a very short test text may become an all-zero vector.

Python Unseen words are ignored by a fitted vectorizer

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

terrible has no column, so the first test text becomes a vector that only contains film. The second one keeps both great and film, with great weighted higher because it is rarer in the training set.

Write a function that computes the perplexity of a sequence from the probabilities a model assigned to each token.

Take the exponential of the mean negative log probability. Working in log space avoids multiplying many tiny numbers into zero.

Python Perplexity from token probabilities

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The first two results are exactly the number of equally likely choices, 4.0 and 2.0. The third, about 2.811, is worse than the confident-looking 0.9 suggests, because the 0.1 hurts a lot.

Show how temperature changes a next-token distribution.

Divide the logits by T, then apply softmax.

Python Softmax with temperature

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

At T = 0.5 the top token takes 86 percent of the probability; at T = 2.0 it takes only about half. Temperature never changes the ranking, only how sharply the model prefers its first choice.

Summary

  • Be able to explain each pipeline step and when it hurts (stop words, stemming, lowercasing).
  • Know the classic stack: bag of words, TF-IDF, Naive Bayes, logistic regression, and why smoothing exists.
  • Choose metrics for the problem: precision, recall, macro F1, never accuracy alone on imbalanced data.
  • Understand embeddings, attention, encoder versus decoder models, sampling and RAG at a one-paragraph level.
  • Fit all preprocessing on the training data only, and check any score against a trivial baseline.

For a one-page review of the formulas and snippets, open the NLP cheat sheet.