Skip to content
Tutorials Center
My learning

Topics

Lesson 110 of 111 ยท Practice and Review

ML Coding Interview Questions

Practice the machine learning coding interview: implement metrics, AUC, scalers, kNN, k-means, linear and logistic regression from scratch in NumPy and check them against scikit-learn.

Many machine learning interviews include a coding round where you implement a basic algorithm without a library: "write k-nearest neighbors", "implement k-means", "compute precision and recall by hand", "train logistic regression with gradient descent". The interviewer is not testing whether you remember a formula. They want to see that you understand what the algorithm does, can turn it into correct and reasonably efficient NumPy, and notice the edge cases (empty clusters, division by zero, overflow, data leakage).

This lesson works through the most common questions. Every solution is a short NumPy implementation that is verified against scikit-learn, so you can see it match. For conceptual questions (bias and variance, regularization, leakage) see Machine Learning Interview Questions. To practice the same skills with exercises, use Exercises: Ensembles, Pipelines and Unsupervised.

How to approach a coding question

Clarify Inputs, shapes, edge cases, constraints.
Outline Say the algorithm before you type.
Code simply Vectorize, but keep it readable.
Test Tiny example, then compare with a library.

Then state time and memory costs.

1 / 2

Question 1 to 3: Metrics, AUC, split and scaling

Metrics are the warm-up. Precision is TP / (TP + FP), recall is TP / (TP + FN) and F1 is their harmonic mean. Handle the case where there are no predicted positives (precision would divide by zero) and say what you return.

ROC AUC is a favorite follow-up. The slow way sweeps thresholds. The elegant way uses the fact that AUC is the probability that a randomly chosen positive example gets a higher score than a randomly chosen negative one. Rank all scores (averaging the ranks of ties), sum the ranks of the positives, and subtract the smallest possible sum: that is the Mann-Whitney U statistic divided by the number of positive-negative pairs.

A stratified split keeps the class proportions in both parts, which matters for rare classes. A scaler must learn its mean and standard deviation from the training rows only, then apply them to everything else, and must not divide by zero for a constant column.

Python Metrics, AUC from ranks, stratified split and a scaler

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

All four match the reference. The tie handling in the AUC deserves a sentence in an interview: we rounded the scores to create ties, and without averaging the ranks of ties the result would differ from scikit-learn's. The split keeps the 10.9% positive share in both parts, and the scaler turns the constant column into zeros instead of NaN.

Question 4 and 5: k-nearest neighbors and k-means

kNN needs no training: store the data, and at prediction time find the k closest training points and take a vote. Computing all distances with broadcasting is the key move: X[:, None, :] - X_train[None, :, :] has shape (queries, training points, features). It costs O(n_query ยท n_train ยท d) time and memory, and a good answer mentions that a k-d tree or approximate search reduces the cost, and that features must be scaled first (k-Nearest Neighbors). Use np.argpartition to find the k smallest without sorting everything.

k-means (Lloyd's algorithm) alternates two steps until nothing changes: assign each point to the nearest center, then move each center to the mean of its points. Things a strong answer adds: initialize with k-means++ (pick the first center at random and each next one with probability proportional to its squared distance from the nearest chosen center), restart several times and keep the lowest inertia, and handle an empty cluster (here the center simply stays where it is). See k-Means.

Python kNN and k-means from scratch, checked against scikit-learn

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The kNN gives the same predictions as scikit-learn on every test flower. K-means reaches the same inertia (503.3) as scikit-learn and recovers the four generated groups perfectly. The final loop answers a common follow-up, "can you choose k by minimizing the loss?": no. The inertia falls from 4,752 (k = 2) to 503 (k = 4) to 317 (k = 8) and would reach zero when k equals the number of points, so you need an elbow, the silhouette score or the gap statistic (Evaluating Clusters).

Question 6 and 7: Linear and logistic regression

Linear regression has a closed form, the normal equation w = (Aแต€A)โปยน Aแต€y, where A is the data with a column of ones for the intercept. In code, use np.linalg.solve rather than computing the inverse, say why (it is faster and more stable), and mention that Aแต€A is singular when features are collinear, which regularization (adding ฮป to the diagonal, but not for the intercept) repairs (Regularization). The same model by gradient descent repeats w โ† w โˆ’ lr ยท gradient, where the gradient of the mean squared error is (2/n) ยท Aแต€(Aw โˆ’ y) (Gradient Descent from Scratch).

Logistic regression has no closed form, so gradient descent is the answer. The loss is the log loss, and its gradient has a famously clean form: Aแต€(ฯƒ(Aw) โˆ’ y) / n, where ฯƒ is the sigmoid. Two numerical details matter: clip the input of exp to avoid overflow, and add a tiny epsilon inside log so a probability of exactly 0 or 1 does not produce infinity. And the best way to show you really understand the gradient is a gradient check: compare it with a finite-difference estimate. If they agree to 8 or more digits, the formula is right.

Python Linear regression two ways, logistic regression and a gradient check

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The normal equation, gradient descent and scikit-learn find the same line (intercept 3.989, weights 1.973, -1.029 and 0.516, close to the 4, 2, -1 and 0.5 we planted). The analytic and numeric gradients of the log loss agree to about 1e-10, so the formula is correct. The loss falls quickly (0.608 after one step, 0.138 after 100 epochs) and then slowly, and the training accuracy matches scikit-learn (0.972 against 0.970). The weights, however, differ (-5.36 against -6.48 for the second feature). That is not a bug. The two classes are almost linearly separable, so the likelihood keeps improving as the weights grow, and scikit-learn's solver ran much further than our 500 epochs. The predictions are almost the same, which is the point to make out loud: when classes are separable, weights are not well determined without regularization, so compare predictions and not coefficients.

More questions to practice

Once these feel easy, try the following on your own. Each one is a typical follow-up.

  • Decision tree split. Given a feature and labels, find the threshold with the lowest weighted Gini impurity (Decision Trees).
  • PCA with SVD. Center the data, take np.linalg.svd, project on the top components, and compute explained variance (PCA).
  • Naive Bayes. Class priors plus per-feature likelihoods, in log space, with Laplace smoothing (Naive Bayes).
  • K-fold cross-validation indices. Generate train and validation indices without a library, with shuffling and stratification (Cross-Validation).
  • Softmax and cross-entropy for several classes, with the max-subtraction trick for stability.
  • Precision at k, and mean average precision for ranking.
  • Reservoir sampling and shuffling for streaming data.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why use np.linalg.solve(A.T @ A, A.T @ y) instead of np.linalg.inv(A.T @ A) @ A.T @ y?

    Both solve the same equation, but solving the linear system directly avoids computing an explicit inverse, which is slower and loses precision when the matrix is nearly singular.
  2. 2. Your k-means implementation ends up with an empty cluster in one iteration. What is a reasonable handling?

    The mean of zero points is undefined. Common fixes are to keep the previous center or to move it to the point farthest from its current center. Say which one you chose.
  3. 3. How do you check that a hand-derived gradient is correct?

    A gradient check evaluates (loss(w + h) - loss(w - h)) / 2h for each weight and compares it with the analytic gradient. A tiny difference confirms the formula.
  4. 4. On nearly separable data, your logistic regression weights differ from scikit-learn's while predictions agree. Why?

    When classes can be separated, the likelihood improves as the weights grow, so the solution is not unique in practice. The decision boundary, and therefore the predictions, are almost identical.

Summary

  • Interviewers want correct, readable NumPy plus awareness of edge cases: empty classes, constant columns, ties, overflow, log of zero, empty clusters.
  • Metrics and AUC come from confusion-matrix counts and ranks; fit scalers and splits so no test information leaks.
  • kNN is one broadcasted distance matrix plus a vote; k-means is assign, update, repeat, with k-means++ seeding and restarts.
  • Linear regression has a closed form (use solve) or gradient descent; logistic regression needs gradient descent with a stable sigmoid and log, and a gradient check proves the gradient.
  • Always test against a library on a small example and state the time and memory cost.

Next: review everything in the Machine Learning Cheat Sheet.