Machine Learning Interview Questions and Answers

Answer

Supervised learning trains on labelled pairs (x, y) and learns a mapping from inputs to known targets: predicting loan default from application data, classifying a support ticket, forecasting demand. Almost everything that ships revenue in Indian product companies is supervised: fraud scoring at Razorpay, delivery-time prediction at Swiggy, ranking at Flipkart. Unsupervised learning gets only x, no labels, and looks for structure: clustering users into segments with k-means, reducing dimensions with PCA, detecting anomalies as points far from learned structure.

Reinforcement learning learns a policy through trial and error against an environment that returns rewards, used in ad bidding, recommendation exploration, and robotics, but rare in most Indian ML roles. Interviewers push past the definitions in two ways. First, hybrids: self-supervised learning creates labels from the data itself (predict the masked word, predict the next frame), and this is how modern LLMs and embedding models are pretrained, so the supervised/unsupervised boundary has blurred.

Second, framing: given a real problem, can you pick the paradigm? 'Group our sellers by behaviour' is unsupervised; 'flag risky sellers' is supervised if you have historical fraud labels and anomaly detection if you do not. A strong answer notes that label availability, not the algorithm, usually decides the framing, and that many teams start unsupervised, use the output to bootstrap labels, and graduate to supervised once labels accumulate.

Answer

Expected prediction error decomposes into three parts: bias squared, variance, and irreducible noise. Bias is error from wrong assumptions: a linear model fitting a curved relationship will be systematically wrong no matter how much data you give it. Variance is error from sensitivity to the training sample: a deep unpruned decision tree memorises one training set and produces a very different tree on a resample, so its predictions swing wildly.

Simple models are high-bias low-variance; flexible models are low-bias high-variance; total error is minimised somewhere in between. In practice you diagnose the tradeoff from the gap between training and validation error. High training error with a small gap means high bias: add features, add interactions, use a more flexible model, reduce regularization.

Low training error with a large gap means high variance: get more data, regularize harder, prune, or ensemble. Interviewers probe two follow-ups. First, which knobs move which term: in gradient boosting, more trees and depth lower bias but raise variance, while a lower learning rate with more trees usually improves both; in neural nets, width and depth lower bias, dropout and weight decay lower variance.

Second, the modern caveat: very large neural networks can violate the classical U-shaped picture (the double descent phenomenon), where test error falls again past the interpolation point. You are not expected to derive it, but knowing the classical tradeoff is a heuristic, not a law, reads well at senior loops. Anchor your answer in the diagnostic, not the textbook curve.

Answer

Overfitting is when a model captures noise specific to the training sample, so training performance is strong while held-out performance is weak. The diagnosis is always a comparison: plot training and validation metrics against training set size (learning curves) or against training epochs/iterations. A widening gap as capacity grows, or a validation curve that improves then reverses while training keeps improving, is the signature.

In gradient boosting, watch the validation metric per boosting round; in neural nets, per epoch. Fixes, in the order most teams try them: get more data or augment what you have; regularize (L1/L2, dropout, early stopping); reduce capacity (shallower trees, fewer features, smaller network); ensemble (bagging averages away variance); and check your features for leakage, because a feature that encodes the label produces a model that looks brilliantly overfit but is actually broken. Early stopping deserves emphasis because it is free: hold out a validation set, stop training when its metric stops improving for a patience window, and restore the best checkpoint.

Interviewers often add a trap: 'training accuracy is 99%, validation is 97%, is that overfitting?' The right answer is that a gap alone is not the problem; the question is whether validation performance is acceptable for the task and stable across folds and time periods. A small gap with terrible absolute performance is underfitting, and closing the gap by crippling the model helps nobody. Always report both curves, never just the final number.

import numpy as np
from sklearn.datasets import make_classification
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.model_selection import learning_curve

X, y = make_classification(n_samples=3000, n_features=20, random_state=42)

sizes, train_scores, val_scores = learning_curve(
    GradientBoostingClassifier(max_depth=6, random_state=42),
    X, y, cv=5, scoring="roc_auc",
    train_sizes=np.linspace(0.1, 1.0, 8), n_jobs=-1,
)

for n, tr, va in zip(sizes, train_scores.mean(1), val_scores.mean(1)):
    gap = tr - va
    print(f"n={n:5d}  train_auc={tr:.3f}  val_auc={va:.3f}  gap={gap:.3f}")
# A gap that stays wide as n grows -> variance problem: regularize.
# Both curves plateauing low -> bias problem: add capacity/features.

Answer

The training set fits parameters. The validation set selects between models and hyperparameters. The test set gives one final, untouched estimate of generalization.

The subtle failure with only train and test is that every time you look at the test score and change something in response, you leak information from the test set into your modelling decisions. After fifty rounds of 'tweak, check test, tweak', your test score is an optimistically biased estimate, because you effectively performed hyperparameter search on it. That is why Kaggle has a public and private leaderboard, and why competitors who overfit the public split fall hundreds of ranks on the private one.

The clean protocol: iterate freely on train/validation (or better, cross-validation on the training portion), and touch the test set once, at the end, to report the number that goes in the deck. In production settings the test set should also be out-of-time, not just out-of-sample: train on January-September, validate on October, test on November-December. Random splits overstate performance for any data with temporal structure because the model gets to see the future's distribution.

Interviewers commonly follow up with sizing: for large datasets a 98/1/1 split is fine because absolute validation size (tens of thousands of rows) matters more than the ratio; for small datasets, use k-fold cross-validation instead of a fixed validation split and keep a small untouched test set. Mentioning out-of-time evaluation unprompted is one of the cheapest ways to signal production experience.

from sklearn.model_selection import train_test_split

# Two-stage split: 70% train, 15% validation, 15% test
X_train, X_tmp, y_train, y_tmp = train_test_split(
    X, y, test_size=0.30, stratify=y, random_state=42
)
X_val, X_test, y_val, y_test = train_test_split(
    X_tmp, y_tmp, test_size=0.50, stratify=y_tmp, random_state=42
)

# For temporal data, split by time instead of randomly:
# df = df.sort_values("event_date")
# train = df[df.event_date < "2025-10-01"]
# val   = df[(df.event_date >= "2025-10-01") & (df.event_date < "2025-11-01")]
# test  = df[df.event_date >= "2025-11-01"]

Answer

K-fold cross-validation splits the data into k equal folds, trains on k-1 of them, validates on the held-out fold, and rotates until every fold has served as validation once. You report the mean and standard deviation of the metric across folds, which gives you both a less noisy performance estimate than a single split and a sense of its variance. Five or ten folds is standard; more folds means less bias in the estimate but more compute and higher variance per fold.

Stratified k-fold preserves the class ratio inside every fold, which matters whenever classes are imbalanced: with 2% positives and plain k-fold, an unlucky fold can end up with almost no positives, making the metric meaningless. Use StratifiedKFold for classification by default. Grouped k-fold ensures all rows belonging to the same entity (user, session, hospital, device) land in the same fold.

This is the one candidates miss: if the same user appears in both train and validation folds, the model partially memorises users rather than learning generalizable patterns, and your CV score will be far above production reality. Any dataset with repeated entities needs GroupKFold. For time series, none of these apply; you need forward-chaining splits where training always precedes validation chronologically (TimeSeriesSplit). The interview follow-up is usually 'your CV score is 0.85 but production is 0.70, why?': the leading suspects are group leakage, temporal leakage from random splits, and preprocessing fit on the full dataset before splitting.

from sklearn.model_selection import StratifiedKFold, GroupKFold, cross_val_score
from sklearn.ensemble import RandomForestClassifier

model = RandomForestClassifier(n_estimators=300, random_state=42)

# Stratified: preserves class ratio per fold (default for classification)
skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
scores = cross_val_score(model, X, y, cv=skf, scoring="roc_auc")
print(f"AUC {scores.mean():.3f} +/- {scores.std():.3f}")

# Grouped: all rows of one user stay in one fold (prevents identity leakage)
# groups = df["user_id"].values
# gkf = GroupKFold(n_splits=5)
# scores = cross_val_score(model, X, y, cv=gkf, groups=groups, scoring="roc_auc")

Answer

The classical assumptions: the relationship between features and target is linear in the parameters; errors are independent; errors have constant variance (homoscedasticity); errors are normally distributed; and features are not perfectly multicollinear. Which ones matter depends on what you are using the model for. If you only care about predictions, the load-bearing assumption is linearity itself: if the true relationship is curved, the model has irreducible bias, fixable with polynomial or spline features, or by switching to trees.

Independence of errors matters enormously for time series and grouped data: with autocorrelated errors your model can still predict, but every standard error and confidence interval it reports is wrong, usually overconfident. If you are doing inference, reading coefficients, reporting significance, making causal-ish claims to a business stakeholder, then heteroscedasticity and multicollinearity move from footnotes to real problems. Multicollinearity does not hurt predictions much, but it makes individual coefficients unstable and uninterpretable: with two nearly duplicate features the model can assign +100 to one and -98 to the other.

Diagnose it with variance inflation factors and fix it by dropping or combining features, or by using ridge regression, which handles collinearity gracefully by shrinking correlated coefficients together. Normality of errors is the least important in large samples thanks to the central limit theorem. The interview signal is exactly this ranking: candidates who recite five assumptions with equal weight sound like a textbook; candidates who say 'depends whether you want prediction or inference' sound like they have shipped models.

Answer

Fitting linear regression to 0/1 labels fails in three ways. Predictions are unbounded, so you get 'probabilities' like 1.4 or -0.2 that cannot feed any downstream decision. Squared-error loss penalises confident correct predictions: a point predicted at 1.3 when the label is 1 incurs loss, so extreme but correct points drag the decision boundary around.

And the constant-variance assumption is structurally violated because Bernoulli variance depends on p. Logistic regression fixes this by passing the linear score z = w.x + b through the sigmoid, mapping it to (0, 1), and training with log loss (binary cross-entropy), which is the negative log-likelihood of the Bernoulli model. Log loss punishes confident wrong predictions harshly and never penalises confidence in the right direction.

The optimisation is convex, so there is a single global optimum, which is one reason logistic regression remains the default first model and the standard baseline in fintech credit models where regulators require explainability. Two probes interviewers love: first, coefficient interpretation, each unit increase in a feature multiplies the odds by e^w, which is a log-odds statement, not a probability statement; second, perfect separation, when a feature splits classes perfectly the weights diverge to infinity, and regularization is what keeps the model finite, which is why scikit-learn applies L2 by default (C=1.0), a detail that surprises many candidates when coefficients do not match a statsmodels fit. Mention that at Indian NBFCs and credit bureaus, scorecards are still logistic regression on binned features (weight of evidence), because explainability to RBI-regulated stakeholders beats a half-point of AUC.

import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
from sklearn.metrics import log_loss, roc_auc_score

clf = make_pipeline(
    StandardScaler(),
    LogisticRegression(C=1.0, max_iter=1000),  # C is 1/lambda: L2 by default
)
clf.fit(X_train, y_train)

proba = clf.predict_proba(X_val)[:, 1]
print("log loss:", log_loss(y_val, proba))
print("roc auc :", roc_auc_score(y_val, proba))

# Odds-ratio interpretation of coefficients (on standardized features)
lr = clf.named_steps["logisticregression"]
odds = np.exp(lr.coef_[0])
print("odds ratios per 1-std increase:", np.round(odds, 2))

Answer

Both add a penalty on weight magnitude to the loss, trading a little bias for a reduction in variance. L2 (ridge) adds lambda * sum(w^2): it shrinks all weights smoothly toward zero but never exactly to zero, and it handles correlated features gracefully by splitting weight between them. L1 (lasso) adds lambda * sum(|w|): its penalty has corners at zero, so the optimum lands exactly on zero for weak features, giving you built-in feature selection and a sparse model.

The geometric intuition interviewers want: the L1 constraint region is a diamond whose corners sit on the axes, and the loss contours usually first touch a corner, zeroing coordinates; the L2 ball is round, so the touch point is rarely on an axis. Practical guidance: use L2 as the default, especially with correlated features, because lasso arbitrarily picks one feature from a correlated group and zeroes the rest, which makes the selection unstable across resamples. Use L1 when you genuinely believe few features matter and you want the model to say which, for example a first pass over thousands of engineered features.

Elastic net combines both penalties and is the pragmatic answer for wide, correlated data (it selects groups rather than lone members). Two follow-ups to be ready for: lambda is tuned by cross-validation, and in scikit-learn's LogisticRegression the knob is C = 1/lambda, so smaller C means stronger regularization; and regularization requires standardized features, because the penalty is scale-sensitive, a feature measured in paise would be penalised thousands of times harder than the same feature in lakhs.

from sklearn.linear_model import Lasso, Ridge, ElasticNet
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
import numpy as np

for name, reg in [
    ("ridge", Ridge(alpha=1.0)),
    ("lasso", Lasso(alpha=0.05)),
    ("enet ", ElasticNet(alpha=0.05, l1_ratio=0.5)),
]:
    pipe = make_pipeline(StandardScaler(), reg)
    pipe.fit(X_train, y_train)
    w = pipe[-1].coef_
    n_zero = int(np.sum(np.abs(w) < 1e-8))
    print(f"{name} zeroed {n_zero}/{len(w)} features, "
          f"val R2 = {pipe.score(X_val, y_val):.3f}")
# Lasso zeroes weak features; ridge shrinks but keeps all of them.

Answer

The confusion matrix for binary classification has four cells: true positives (predicted positive, actually positive), false positives (predicted positive, actually negative), false negatives (missed positives), and true negatives. Precision = TP / (TP + FP): of everything I flagged, how much was real. Recall = TP / (TP + FN): of everything real, how much did I catch.

They pull against each other through the decision threshold: lower the threshold and you flag more, recall rises, precision falls; raise it and the reverse. F1 is the harmonic mean of the two, which punishes imbalance between them, a model with precision 0.9 and recall 0.1 gets F1 0.18, not the arithmetic 0.5. The interview skill is mapping errors to business cost.

In fraud detection at a payments company, a false negative is money lost to fraud, but a false positive blocks a legitimate customer's transaction, which has its own cost in support tickets and churn; the threshold is a business decision, not a modelling one. In cancer screening, recall dominates because a miss is catastrophic and a false positive triggers a follow-up test. In a hiring-spam filter, precision dominates because deleting a genuine application is worse than letting spam through.

Be ready for F-beta: F2 weights recall higher, F0.5 weights precision higher, and stating which beta matches the stated business cost is a strong senior signal. Also know that with multi-class problems you must choose macro (average per class, treats rare classes equally) versus micro or weighted averaging, and the choice changes the number substantially when classes are imbalanced.

from sklearn.metrics import (confusion_matrix, precision_score,
                             recall_score, f1_score, classification_report)
import numpy as np

proba = model.predict_proba(X_val)[:, 1]

for threshold in (0.3, 0.5, 0.7):
    pred = (proba >= threshold).astype(int)
    tn, fp, fn, tp = confusion_matrix(y_val, pred).ravel()
    print(f"t={threshold}  TP={tp} FP={fp} FN={fn} TN={tn}  "
          f"precision={precision_score(y_val, pred):.2f}  "
          f"recall={recall_score(y_val, pred):.2f}  "
          f"f1={f1_score(y_val, pred):.2f}")

# Full per-class breakdown (macro vs weighted matters when imbalanced)
print(classification_report(y_val, (proba >= 0.5).astype(int)))

Answer

Accuracy lies whenever the classes are imbalanced, and most valuable problems are imbalanced. Concrete example: UPI transaction fraud runs well below 1% of transactions. A model that predicts 'not fraud' for every single transaction scores over 99% accuracy while catching zero fraud, providing zero value, and looking excellent in a slide.

The same failure appears in churn (5-10% monthly churners), loan default (2-8%), rare disease screening, and defect detection. The fix is metrics that expose performance on the minority class: precision and recall at the operating threshold, the full precision-recall curve and PR-AUC when you need a threshold-free summary, and business-native metrics like recall at a fixed alert budget ('of the top 1,000 transactions we can manually review per day, how many are actually fraud'). Balanced accuracy (mean of per-class recall) and Matthews correlation coefficient are also robust to imbalance and worth naming.

Two deeper points earn credit. First, the baseline framing: always state the majority-class baseline before quoting any accuracy; '92% accuracy' is meaningless until you say the baseline is 90%. Second, accuracy also lies when error costs are asymmetric even without imbalance: a 50/50 problem where false negatives cost 100x false positives should not be evaluated symmetrically. Interviewers at analytics services firms like Fractal and Tiger Analytics use this question as a filter for client-facing readiness: the candidate who instinctively asks 'what is the class ratio and what does each error cost?' before naming a metric passes; the one who says 'I would check the accuracy' fails.

Answer

ROC-AUC plots true positive rate against false positive rate across all thresholds and summarises to the probability that a randomly chosen positive scores higher than a randomly chosen negative. It is threshold-free, and 0.5 means random ranking. PR-AUC plots precision against recall across thresholds.

The critical difference is what happens under heavy imbalance. The false positive rate has all the negatives in its denominator: with a million legitimate transactions, 5,000 false positives is an FPR of just 0.5%, so ROC-AUC barely moves and can sit at a flattering 0.95 while your alert queue is 80% noise. Precision has your predicted positives in the denominator, so those same 5,000 false alarms crush it.

Rule: for balanced problems or when you care about ranking across the full population, ROC-AUC is fine and comparable across datasets; for rare-event detection where you act on the flagged set (fraud review queues, churn outreach lists, medical alerts), PR-AUC or precision-recall at your operating point tells the truth. A useful interview line: 'ROC-AUC answers how well the model separates the classes; PR-AUC answers how useful the alerts are.' Also know the baselines: random ROC-AUC is 0.5 regardless of imbalance, but random PR-AUC equals the positive prevalence, so PR-AUC of 0.30 on a 2% positive problem is a 15x lift over random, which sounds unimpressive until you state the baseline. Finally, neither metric captures calibration; a model can rank perfectly while its probabilities are systematically wrong, which matters as soon as you multiply probability by monetary value.

import numpy as np
from sklearn.metrics import roc_auc_score, average_precision_score

rng = np.random.default_rng(42)
n, prevalence = 100_000, 0.01           # 1% positives, fraud-like
y = (rng.random(n) < prevalence).astype(int)
# A mediocre scorer: positives shifted slightly right of negatives
scores = rng.normal(0, 1, n) + y * 1.2

print("ROC-AUC :", round(roc_auc_score(y, scores), 3))   # looks strong
print("PR-AUC  :", round(average_precision_score(y, scores), 3))
print("PR base :", prevalence)  # random-model PR-AUC = prevalence

# Same separation, balanced classes -> PR-AUC jumps, ROC-AUC ~unchanged.
# ROC-AUC is insensitive to prevalence; precision is not.

Answer

A decision tree grows greedily: at each node it scans candidate splits (each feature, each threshold) and picks the one that most reduces impurity, meaning the children are purer in class composition than the parent, weighted by how many samples each child receives. Gini impurity is 1 - sum(p_k^2): the probability of misclassifying a random sample if you labelled it by the node's class distribution. Entropy is -sum(p_k * log2 p_k), the information-theoretic uncertainty; splits are chosen to maximise information gain, the drop in entropy.

Both are zero for pure nodes and maximal for uniform mixes, and in practice they choose nearly identical trees; Gini is the scikit-learn default because it avoids computing logarithms. For regression trees, the impurity is variance (MSE) and splits minimise within-child variance. The parts interviewers actually probe: greediness means the tree makes locally optimal splits with no lookahead, so it can miss globally better structures (XOR-style interactions need two levels to express and a greedy tree may never find them); an unconstrained tree will grow until leaves are pure, memorising the training set, so max_depth, min_samples_leaf, and min_impurity_decrease are your variance controls; and trees are completely insensitive to monotonic feature transformations, so scaling and log transforms change nothing, which is why tree models are the low-preprocessing workhorse for messy tabular data. A good closing observation: single trees are rarely deployed because they are high-variance and unstable (a few changed rows can flip the root split), which is precisely the weakness bagging and boosting were invented to fix.

Answer

A single deep tree is a low-bias, high-variance model: it fits the training data closely but a slightly different sample produces a very different tree. Random forests attack the variance with two sources of decorrelation. First, bagging (bootstrap aggregating): each tree trains on a bootstrap sample, drawn with replacement, of the training data, so each tree sees a slightly different world.

Second, feature subsampling: at every split, only a random subset of features (typically sqrt(n_features) for classification) is considered, which prevents one dominant feature from making every tree open with the same root split. Averaging many decorrelated, individually overfit trees cancels their idiosyncratic errors while keeping the shared signal: variance drops roughly with the number of effectively independent trees, while bias stays close to that of a single deep tree. This is why forests barely overfit as you add trees; more estimators only stabilise the average, and the practical limit is compute, not overfitting.

Useful details to volunteer: out-of-bag (OOB) evaluation scores each tree on the roughly 37% of rows its bootstrap missed, giving a free validation estimate without a held-out set; forests train embarrassingly parallel because trees are independent, unlike boosting which is sequential; and the tuning surface is small (n_estimators, max_features, min_samples_leaf), which is why a forest is the standard 'strong baseline in ten minutes' before reaching for XGBoost. The classic follow-up is 'when does a forest lose to boosting?': when bias, not variance, is the binding constraint, because averaging cannot fix what every tree gets systematically wrong, while boosting explicitly targets residual errors.

from sklearn.ensemble import RandomForestClassifier
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import cross_val_score

tree = DecisionTreeClassifier(random_state=42)
forest = RandomForestClassifier(
    n_estimators=500,
    max_features="sqrt",      # feature subsampling per split
    min_samples_leaf=2,
    oob_score=True,            # free validation from out-of-bag rows
    n_jobs=-1,
    random_state=42,
)

print("tree  :", cross_val_score(tree, X, y, cv=5, scoring="roc_auc").mean())
print("forest:", cross_val_score(forest, X, y, cv=5, scoring="roc_auc").mean())

forest.fit(X_train, y_train)
print("OOB score:", forest.oob_score_)  # no held-out set needed

Answer

KNN is the simplest non-parametric method: to classify a point, find its k closest training points under some distance metric (usually Euclidean) and take a majority vote (or average, for regression). There is no training phase beyond storing the data; all cost is paid at prediction time, which makes naive KNN O(n) per query and motivates approximate nearest-neighbour indexes for anything large. Choosing k trades bias against variance: k=1 memorises the training set (high variance), very large k smooths toward the global majority (high bias); tune it with cross-validation and prefer odd k for binary votes.

Feature scaling is mandatory, since a feature measured in rupees will dominate one measured in years inside a Euclidean distance. The deeper part of the question is the curse of dimensionality. As dimensions grow, volume concentrates near the boundary of any region, and the ratio between the nearest and farthest neighbour distances approaches 1: everyone becomes roughly equidistant, so 'nearest' stops being meaningful and KNN's core assumption (local similarity implies label similarity) collapses.

Practically, KNN degrades noticeably beyond a few dozen informative dimensions, and irrelevant features actively poison the distance. Mitigations: dimensionality reduction (PCA, UMAP) before KNN, feature selection, or learned distance metrics. The modern relevance is worth stating: approximate KNN over learned embeddings is exactly what vector databases do for semantic search and recommendation retrieval; the embedding model solves the curse by compressing meaning into a few hundred dense dimensions, and HNSW-style indexes solve the O(n) query cost. That connection turns a textbook answer into a 2026 answer.

Answer

K-means partitions data into k clusters by alternating two steps until assignments stop changing: assign each point to its nearest centroid, then recompute each centroid as the mean of its assigned points. This is coordinate descent on within-cluster sum of squares (inertia), and it always converges, but only to a local optimum, so scikit-learn runs multiple restarts (n_init) and keeps the best. Initialisation matters: k-means++ seeds centroids far apart and is the default for good reason.

Choosing k: the elbow method plots inertia against k and looks for the bend, which is subjective and often absent; silhouette score (how much closer each point is to its own cluster than to the next best) gives a number in [-1, 1] you can compare across k; but the honest senior answer is that k is usually a product decision, if marketing can act on five segments, k is five, and the metrics only sanity-check that five is not badly wrong. Failure modes to volunteer: k-means assumes convex, isotropic, similar-sized clusters, so it butchers elongated shapes, nested rings, and clusters of very different densities (DBSCAN or Gaussian mixtures handle those); it is sensitive to outliers because means are; and unscaled features let one dimension dominate the distance. Categorical features do not belong in k-means at all; means of one-hot columns are not meaningful centroids, use k-modes or embed first. Practical Indian-market example: segmenting Swiggy-style users on recency, frequency, and monetary value (RFM) after log-transforming and scaling each axis, then profiling the clusters back in business terms.

import numpy as np
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score
from sklearn.preprocessing import StandardScaler

# RFM-style user segmentation: recency, frequency, monetary
Xs = StandardScaler().fit_transform(np.log1p(rfm_df.values))

for k in range(2, 9):
    km = KMeans(n_clusters=k, n_init=10, random_state=42).fit(Xs)
    sil = silhouette_score(Xs, km.labels_)
    print(f"k={k}  inertia={km.inertia_:10.1f}  silhouette={sil:.3f}")

best = KMeans(n_clusters=5, n_init=10, random_state=42).fit(Xs)
rfm_df["segment"] = best.labels_
print(rfm_df.groupby("segment").mean().round(1))  # profile the clusters

Answer

PCA finds an orthogonal rotation of the feature space whose axes (principal components) point in the directions of maximum variance, ordered from most to least. Mechanically, it is the eigendecomposition of the covariance matrix (or the SVD of the centred data matrix): each component is a linear combination of the original features, components are uncorrelated, and keeping the top d components gives the best possible d-dimensional linear reconstruction of the data in a squared-error sense. Standardise features first; PCA on unscaled data just finds whichever feature has the biggest units.

Legitimate uses: compressing hundreds of correlated features before a distance-based method (KNN, k-means) or a linear model with limited data; decorrelating features to stabilise coefficients; visualising structure in two components; and denoising, since low-variance components are often noise. When not to use it, which is what interviewers listen for: PCA is unsupervised, so it preserves variance, not predictive signal; a low-variance direction can carry the label, and PCA will throw it away. Tree ensembles neither need nor benefit from it, they handle correlated raw features fine, and you destroy interpretability because 'component 3' means nothing to a stakeholder or a regulator. The standard follow-ups: choose the number of components by cumulative explained variance (say 95%) or by downstream CV performance, never by eyeballing alone; and fit PCA inside the pipeline on training folds only, because fitting it on the full dataset before splitting leaks the test set's covariance structure into training, an easy-to-miss form of leakage.

import numpy as np
from sklearn.decomposition import PCA
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score

# PCA fit inside the pipeline -> no leakage across CV folds
pipe = make_pipeline(
    StandardScaler(),
    PCA(n_components=0.95),   # keep enough PCs for 95% variance
    LogisticRegression(max_iter=1000),
)
print("CV AUC:", cross_val_score(pipe, X, y, cv=5, scoring="roc_auc").mean())

pca = make_pipeline(StandardScaler(), PCA()).fit(X)[-1]
cum = np.cumsum(pca.explained_variance_ratio_)
print("components for 95% variance:", int(np.searchsorted(cum, 0.95)) + 1)

Answer

The dividing line is whether the algorithm computes distances, dot products, or shared penalties across features. Need scaling: KNN and k-means (Euclidean distance mixes units), SVMs (margins and kernels are geometric), PCA (maximises variance, which is unit-dependent), neural networks (unscaled inputs produce badly conditioned loss surfaces and slow, unstable gradient descent), and any regularized linear model (L1/L2 penalties are scale-sensitive, so an unscaled feature gets unfairly punished or favoured). Gradient descent itself converges faster on scaled features even without regularization, because the loss contours become more spherical and one learning rate suits all directions.

Do not need scaling: decision trees, random forests, and gradient boosting (XGBoost, LightGBM), because splits are rank-based thresholds on one feature at a time, and any monotonic transformation leaves the tree unchanged. Naive Bayes is also scale-indifferent since it models each feature's distribution separately. The choice of scaler is the second half of the answer: StandardScaler (zero mean, unit variance) is the default; MinMaxScaler maps to [0, 1] and suits bounded inputs like pixels; RobustScaler uses median and IQR and is the right pick when outliers are real, because a single extreme value drags the mean and variance of StandardScaler. Two production gotchas: fit the scaler on training data only and apply it to validation, test, and live traffic with the same fitted parameters (leakage otherwise, and training-serving skew if the serving path recomputes statistics); and log-transform heavy-tailed monetary features like transaction amounts before scaling, because standardising a power-law distribution still leaves extreme values dominating.

from sklearn.preprocessing import StandardScaler, RobustScaler
from sklearn.pipeline import Pipeline
from sklearn.svm import SVC
from sklearn.model_selection import cross_val_score
import numpy as np

# Scaling INSIDE the pipeline: fit on train folds only, no leakage
svm_scaled = Pipeline([
    ("scale", StandardScaler()),
    ("svm", SVC(kernel="rbf", C=1.0)),
])
svm_raw = SVC(kernel="rbf", C=1.0)

print("SVM raw   :", cross_val_score(svm_raw, X, y, cv=5).mean())
print("SVM scaled:", cross_val_score(svm_scaled, X, y, cv=5).mean())

# Heavy-tailed money features: log first, then scale
amounts = np.log1p(df[["txn_amount"]])
scaled = RobustScaler().fit_transform(amounts)  # robust to outliers

Answer

Start with why the values are missing, because the mechanism decides the fix. Missing completely at random (a sensor dropped packets) is benign; simple imputation works. Missing at random given other features (income missing more often for younger users) needs imputation that conditions on those features.

Missing not at random (income missing because high earners decline to state it) is the dangerous case: the missingness itself carries signal, and imputing it away destroys information. The practical toolkit: median imputation for skewed numeric features (mean is dragged by outliers), most-frequent or an explicit 'missing' category for categoricals, KNN or iterative (model-based) imputation when features are correlated enough to predict each other, and, critically, a binary missingness indicator column alongside any imputation so the model can learn from the pattern of absence. Tree ensembles change the calculus: XGBoost and LightGBM handle missing values natively by learning at every split which direction missing rows should go, so for gradient boosting the strongest move is often to leave NaNs in place and let the model use them.

Imputation is actively wrong in a few cases: dropping or filling target values (never impute the label); imputing before a train/test split (statistics leak); imputing time-series with future values (use forward fill, never backward fill, for anything causal); and imputing when missingness is the feature, in credit underwriting, 'declined to provide employer' predicts default better than most provided fields. Always fit the imputer on training data only, inside the pipeline, and log the missing rate per feature in production, because a feed silently going 90% null is a common way models rot.

import numpy as np
import pandas as pd
from sklearn.impute import SimpleImputer
from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer

num_cols = ["age", "monthly_income", "txn_count_30d"]
cat_cols = ["city_tier", "employment_type"]

# Missingness indicators BEFORE imputing: absence is often signal
for c in ["monthly_income", "employment_type"]:
    df[f"{c}_missing"] = df[c].isna().astype(int)

preprocess = ColumnTransformer([
    ("num", SimpleImputer(strategy="median"), num_cols),
    ("cat", SimpleImputer(strategy="constant", fill_value="missing"), cat_cols),
])
# Fit on train only; the same fitted imputer transforms val/test/live.
X_train_t = preprocess.fit_transform(df_train)
X_val_t = preprocess.transform(df_val)

# For XGBoost/LightGBM: often better to keep NaN and let the tree route it.

Answer

One-hot encoding creates a binary column per category. It is the safe default for low-cardinality features (under roughly 15-20 levels) with linear models and neural nets, because it imposes no false ordering. Its failure mode is cardinality: one-hot encoding 19,000 Indian pincodes gives you 19,000 sparse columns, slows training, and leaves each column with too few positives to learn from.

Label (ordinal) encoding maps categories to integers. For linear models this invents a fake ordering ('Chennai < Delhi < Mumbai') and is simply wrong, but for tree models it is often acceptable because trees can carve integer ranges with repeated splits, and LightGBM goes further with native categorical support that finds optimal category partitions. Target encoding replaces each category with a statistic of the label for that category, typically the smoothed mean: pincode 400001 becomes its historical default rate.

It is the high-cardinality workhorse, one dense, informative column instead of thousands, but it is also the most dangerous encoder in the toolkit, because computed naively on the full training set it leaks the label into the features and produces inflated CV scores that evaporate in production. The fixes are mandatory, not optional: compute encodings out-of-fold (each fold's encoding uses only the other folds' labels), and smooth toward the global mean so rare categories with three rows do not get extreme values. scikit-learn's TargetEncoder does both. Also mention frequency encoding (category replaced by its count) as a cheap, leak-free alternative that often captures most of the signal, and hashing for streaming settings where new categories appear constantly.

from sklearn.preprocessing import OneHotEncoder, TargetEncoder
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import make_pipeline
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score

low_card = ["city_tier", "device_os"]          # few levels -> one-hot
high_card = ["pincode", "employer_name"]        # thousands -> target encode

pre = ColumnTransformer([
    ("ohe", OneHotEncoder(handle_unknown="ignore", min_frequency=50), low_card),
    # TargetEncoder does out-of-fold encoding internally during fit_transform
    ("te", TargetEncoder(smooth="auto"), high_card),
], remainder="passthrough")

pipe = make_pipeline(pre, LogisticRegression(max_iter=1000))
print(cross_val_score(pipe, X, y, cv=5, scoring="roc_auc").mean())
# Naive target encoding fit on ALL rows would leak labels and inflate this.

Answer

Leakage is any information available at training time that will not be available at prediction time, and it is the most expensive bug in applied ML because everything looks great until the model meets reality. Three real classes. Target leakage: a feature that is a consequence of the label.

A churn model with 'number of retention calls received' as a feature scores brilliantly, because retention calls happen after someone is flagged as churning; the feature encodes the answer. In credit, 'days past due at month end' leaking into a default model is the same failure. Temporal leakage: training on data from after the prediction moment.

Random train/test splits on time-stamped data let the model learn November patterns and get tested on October; features built with full-history aggregates ('user's lifetime order count') silently include future orders unless computed as-of the prediction date. Preprocessing leakage: fitting any statistic on the full dataset before splitting, scaler means, imputation medians, PCA components, target encodings, vocabulary for TF-IDF. Each one lets test-set information shape the transformation applied to training data.

This third class is the one scikit-learn Pipelines eliminate structurally: when the preprocessing steps live inside the pipeline and the pipeline goes through cross_val_score, every fold fits its transformers on that fold's training portion only. The interview follow-up is detection: be suspicious of any feature with implausibly high importance, of AUCs above roughly 0.95 on human-behaviour problems, and of large gaps between CV and out-of-time evaluation; audit each top feature by asking 'would I know this value at the moment of prediction?' with a timestamp-level answer.

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score

# WRONG: scaler sees all rows, including future validation folds
# X_scaled = StandardScaler().fit_transform(X)
# cross_val_score(LogisticRegression(), X_scaled, y, cv=5)

# RIGHT: scaler is fit inside each CV fold on training rows only
pipe = Pipeline([
    ("scale", StandardScaler()),
    ("clf", LogisticRegression(max_iter=1000)),
])
scores = cross_val_score(pipe, X, y, cv=5, scoring="roc_auc")
print(scores.mean())

# Temporal leakage check for any feature: could I compute this value
# using only data with timestamp < prediction_time for this row?

Answer

Gradient descent minimises a loss by repeatedly stepping opposite the gradient: w := w - lr * dL/dw. The variants differ in how much data computes each gradient. Batch gradient descent uses the entire dataset per step: the gradient is exact, the descent path is smooth, and each step is brutally expensive; for a dataset that does not fit in memory it is simply impractical.

Stochastic gradient descent (SGD) uses one example per step: updates are cheap and frequent, and the gradient is an unbiased but very noisy estimate, so the loss jitters downward and the noise can actually help escape saddle points and poor local minima, but convergence near the optimum is erratic without learning-rate decay. Mini-batch gradient descent uses a batch of typically 32-1024 examples: the noise averages down with batch size while each step stays cheap, and, decisively, batched matrix operations saturate GPU parallelism, which pure SGD cannot. That hardware fit, plus gradient noise low enough to be stable and high enough to regularise, is why mini-batch is the universal default in deep learning; when papers or frameworks say 'SGD' they almost always mean mini-batch SGD.

Follow-ups worth anticipating: an epoch is one full pass over the data, and steps per epoch = n / batch_size; larger batches allow larger learning rates but very large batches tend to find sharper minima that generalise slightly worse, which is why linear learning-rate scaling with warmup accompanies big-batch training; and shuffling data each epoch matters because ordered data (all positives together) biases consecutive gradients. Classical ML is not exempt: LightGBM and XGBoost use all rows per boosting iteration (with optional subsampling), which is closer to batch descent in function space.

import numpy as np

# Mini-batch SGD for linear regression, from scratch
rng = np.random.default_rng(0)
X = rng.normal(size=(10_000, 5))
true_w = np.array([2.0, -1.0, 0.5, 3.0, 0.0])
y = X @ true_w + rng.normal(0, 0.1, 10_000)

w = np.zeros(5)
lr, batch_size = 0.05, 64
for epoch in range(20):
    idx = rng.permutation(len(X))          # reshuffle every epoch
    for start in range(0, len(X), batch_size):
        b = idx[start:start + batch_size]
        grad = 2 * X[b].T @ (X[b] @ w - y[b]) / len(b)
        w -= lr * grad
    loss = np.mean((X @ w - y) ** 2)
    if epoch % 5 == 0:
        print(f"epoch {epoch:2d}  mse={loss:.4f}")
print("recovered:", np.round(w, 2))

Answer

The learning rate is the single most important hyperparameter in gradient-based training, and its failure modes are asymmetric. Too low: training crawls, loss decreases painfully slowly, you burn compute, and optimisation can stall on plateaus or in shallow local structure before reaching anything good; the loss curve looks like a nearly flat line that never converges within budget. Too high: steps overshoot the minimum, loss oscillates or, past a threshold, diverges to NaN, often within the first few hundred steps; a moderately-too-high rate produces a loss that falls quickly then plateaus at a worse level than a smaller rate would reach, because the optimiser keeps bouncing across the valley floor instead of descending into it.

The classic diagnostic is the loss-versus-learning-rate sweep (the LR range test): train briefly while increasing lr exponentially, plot loss against lr, and pick a value about an order of magnitude below where loss starts rising. Schedules resolve the tension between wanting big early steps and small late ones. Step decay and cosine annealing shrink lr over training; cosine with a linear warmup is the default recipe for transformer training, warmup matters because early gradients are noisy while normalisation statistics and Adam's moment estimates stabilise, and a large initial rate at that stage can wreck training irrecoverably.

ReduceLROnPlateau cuts lr when validation loss stalls, a robust choice when you cannot pre-plan the schedule. Also connect to the optimiser: Adam adapts per-parameter step sizes, which makes it far more forgiving of the base lr than plain SGD, but it does not remove the need for one; and note that batch size and lr interact, doubling batch size typically supports a proportionally higher lr.

Answer

Naive Bayes applies Bayes' theorem to compute P(class | features) and makes one aggressive simplification: it assumes every feature is conditionally independent of every other feature given the class. That is the 'naive' part, and it is plainly false almost everywhere: in an email, the words 'lottery' and 'winner' are obviously correlated even within spam. The assumption collapses an intractable joint distribution into a product of per-feature likelihoods, P(x|c) = product of P(x_i|c), which makes training a single counting pass over the data and prediction a sum of log-probabilities: fast, memory-light, trivially parallel, and workable with tiny training sets because each parameter is estimated from marginal counts rather than joint ones.

Why it still works: classification only needs the argmax over classes to be right, not the probabilities to be right. Violated independence distorts the estimated probabilities, usually pushing them toward 0 or 1 overconfidently, but frequently leaves the ranking of classes intact. For text with bag-of-words features, thousands of weakly informative, partially redundant features, multinomial Naive Bayes remains a strong baseline that trains in seconds, and for spam filtering and short-text routing it is still deployed in 2026 where latency and simplicity beat a transformer's marginal accuracy. Details that earn credit: Laplace (add-one) smoothing prevents a single unseen word from zeroing an entire class posterior; work in log space to avoid floating-point underflow when multiplying thousands of small probabilities; use Gaussian NB for continuous features and Bernoulli NB for binary presence features; and never trust NB's probability outputs for downstream expected-value calculations without recalibration, precisely because of the overconfidence the independence violation causes.

Answer

A parametric model commits to a functional form with a fixed number of parameters, decided before seeing data: linear and logistic regression, Naive Bayes, and a neural network of fixed architecture are parametric, whether the parameter count is five or five billion. A non-parametric model lets its effective complexity grow with the data: KNN keeps the entire training set as its 'parameters', decision trees grow as many splits as the data supports, kernel density estimators and Gaussian processes likewise scale with n. The distinction is not about having no parameters, it is about whether capacity is fixed in advance or data-driven.

Why it matters practically: parametric models make strong assumptions, and when the assumptions are roughly right they are data-efficient, fast at inference (evaluate a fixed formula), compact to deploy, and their extrapolation behaviour is defined, a linear model will confidently extend a trend beyond the training range, for better or worse. When the assumptions are wrong, they hit a bias floor that more data cannot fix. Non-parametric models assume little and can approximate arbitrary shapes given enough data, but they need more data to do it, their inference cost or memory often grows with n (naive KNN carries the dataset to production), and they extrapolate poorly, a tree predicts a constant beyond the range of its training splits, which is exactly why tree ensembles fail on trending time series unless you model the trend separately. The interview framing that lands well: parametric versus non-parametric is a data-efficiency versus flexibility trade, and the practical 2026 answer for tabular problems, gradient-boosted trees, is non-parametric, while the practical answer for perception and language, deep networks, is parametric with the capacity dial turned very high.

Answer

Work through four layers, cheapest first. Layer one, do nothing to the data: use a model and metric that respect imbalance. Gradient boosting on the raw distribution with PR-AUC evaluation is a legitimate baseline; imbalance is not automatically a problem if the classes are separable.

Layer two, reweight: class_weight='balanced' in scikit-learn or scale_pos_weight in XGBoost multiplies the minority class's contribution to the loss, pushing the model to care about positives without touching the data. This is usually the best effort-to-value trade and should be your default answer. Layer three, resample: random undersampling of the majority class throws away data but can help when you have millions of negatives; random oversampling duplicates positives and overfits them; SMOTE synthesises new minority points by interpolating between neighbours.

Be sceptical of SMOTE in interviews, on high-dimensional or categorical-heavy data the interpolations are often meaningless (a synthetic 'half of pincode A, half of pincode B' user), and multiple large empirical studies show reweighting matches or beats it on gradient boosting. If you resample, do it inside the CV loop on training folds only; resampling before splitting leaks duplicated or synthetic positives into validation and inflates every metric. Layer four, change the framing: with 0.3% positives, anomaly detection, or a two-stage funnel (cheap high-recall filter, expensive precise second stage) may fit the operational reality of a fraud review queue better than one classifier. Close with the two things that matter regardless: evaluate with PR-AUC and recall at the review-queue budget, never accuracy or plain ROC-AUC, and remember that reweighting and resampling destroy probability calibration, so recalibrate before using scores as probabilities.

from xgboost import XGBClassifier
from sklearn.model_selection import StratifiedKFold, cross_val_score
import numpy as np

neg, pos = np.bincount(y)
print(f"imbalance {neg}:{pos}  ({pos / len(y):.3%} positive)")

model = XGBClassifier(
    n_estimators=500,
    learning_rate=0.05,
    max_depth=5,
    scale_pos_weight=neg / pos,   # reweight instead of resample
    eval_metric="aucpr",
    n_jobs=-1,
)
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
scores = cross_val_score(model, X, y, cv=cv, scoring="average_precision")
print(f"PR-AUC {scores.mean():.3f} (random baseline = {pos / len(y):.3f})")

# If you must SMOTE: use imblearn.pipeline.Pipeline so resampling
# happens inside each training fold, never on validation rows.

Answer

The 0.5 default threshold is an artefact, not a decision; the right threshold comes from costs. If a false negative costs C_fn and a false positive costs C_fp, the expected-cost-minimising threshold on calibrated probabilities is C_fp / (C_fp + C_fn): a fraud problem where a miss costs 50x a false alarm implies flagging at roughly 2% probability, nowhere near 0.5. When costs are hard to pin down, sweep the threshold on the validation set and pick the operating point from the precision-recall curve that satisfies the business constraint, typically phrased as 'maximise recall subject to precision >= 80%' or 'best F1' or 'fill the 500-case daily review queue with the highest-probability cases'.

Choose the threshold on validation data, never on test, it is a hyperparameter. Calibration is the separate question of whether the probabilities mean what they say: a calibrated model's predictions of 0.7 come true about 70% of the time. Rank metrics like ROC-AUC are blind to calibration, but the moment you do expected-value arithmetic, probability times loan amount, probability times customer lifetime value, or use the cost-ratio threshold formula above, calibration becomes load-bearing.

Modern gradient boosting is often reasonably calibrated out of the box, but class reweighting, resampling, and small trees distort it; deep networks are famously overconfident. Diagnose with a reliability diagram (predicted probability bins versus observed frequency) and Brier score or log loss; fix with Platt scaling (fit a logistic regression on the model's scores, good for small data and sigmoid-shaped distortion) or isotonic regression (non-parametric, needs more data, fixes arbitrary monotonic distortion), both fit on a held-out calibration set, not training data.

import numpy as np
from sklearn.calibration import CalibratedClassifierCV
from sklearn.metrics import precision_recall_curve, brier_score_loss

# Calibrate on held-out folds, then pick threshold from business costs
calibrated = CalibratedClassifierCV(base_model, method="isotonic", cv=5)
calibrated.fit(X_train, y_train)
proba = calibrated.predict_proba(X_val)[:, 1]
print("Brier:", round(brier_score_loss(y_val, proba), 4))

# Cost-based threshold: miss costs 50x a false alarm
c_fp, c_fn = 1, 50
print("cost threshold:", c_fp / (c_fp + c_fn))  # ~0.02, not 0.5

# Constraint-based: max recall subject to precision >= 0.80
prec, rec, thr = precision_recall_curve(y_val, proba)
ok = np.where(prec[:-1] >= 0.80)[0]
best = ok[np.argmax(rec[ok])]
print(f"threshold={thr[best]:.3f}  precision={prec[best]:.2f}  recall={rec[best]:.2f}")

Answer

Gradient boosting builds an additive model: F_m(x) = F_{m-1}(x) + lr * h_m(x), where each new small tree h_m is trained to correct the current ensemble's errors. The 'gradient' part is the key generalisation: rather than fitting residuals directly (which is the special case for squared error), each tree is fit to the negative gradient of the loss with respect to the current predictions, the pseudo-residuals. That is what lets one algorithm optimise log loss for classification, squared or absolute error for regression, and pairwise objectives for ranking: swap the loss, the gradients change, the machinery does not.

The learning rate shrinks each tree's contribution, and the core practical trade is lower learning rate plus more trees generalises better. Boosting reduces bias sequentially, which is the mirror image of bagging's variance reduction, and is why boosted models can overfit with too many rounds while forests cannot. What XGBoost added when it appeared, and why it swept tabular ML: a second-order formulation, using both gradient and Hessian of the loss, which yields a closed-form quality score for any candidate split and optimal leaf weights; built-in regularization in the objective itself (gamma for minimum split gain, lambda for L2 on leaf weights) rather than only structural constraints; sparsity-aware split finding with a learned default direction for missing values; column and row subsampling per tree; and systems engineering, histogram-based approximate splits, cache-aware layout, parallelised split search across features, that made it fast enough for industrial datasets. Interviewers commonly probe two contrasts: boosting versus bagging (sequential bias reduction versus parallel variance reduction), and why the learning rate exists at all (it is shrinkage regularization; taking full-size steps in function space overfits the pseudo-residuals of early rounds).

import numpy as np
from sklearn.tree import DecisionTreeRegressor

# Gradient boosting from scratch for squared error (gradient = residual)
rng = np.random.default_rng(0)
X = rng.uniform(-3, 3, size=(2000, 1))
y = np.sin(2 * X[:, 0]) + rng.normal(0, 0.2, 2000)

lr, n_rounds = 0.1, 200
pred = np.full_like(y, y.mean())      # F_0 = base prediction
trees = []
for m in range(n_rounds):
    pseudo_residuals = y - pred        # -dL/dF for squared error
    tree = DecisionTreeRegressor(max_depth=3)
    tree.fit(X, pseudo_residuals)
    pred += lr * tree.predict(X)       # F_m = F_{m-1} + lr * h_m
    trees.append(tree)
    if m % 50 == 0:
        print(f"round {m:3d}  mse={np.mean((y - pred) ** 2):.4f}")

Answer

All three are gradient-boosted decision trees with second-order optimisation and histogram-based split finding; the differences are in growth strategy, categorical handling, and defaults. LightGBM grows trees leaf-wise (best-first): it repeatedly expands the leaf with the highest gain, producing deep, asymmetric trees that reach lower loss with fewer nodes; this makes it typically the fastest of the three and the default choice for large tabular datasets, at the cost of easier overfitting on small data, controlled via num_leaves and min_data_in_leaf rather than depth alone. It also pioneered GOSS (keeping large-gradient rows and subsampling small-gradient ones) and exclusive feature bundling for sparse data.

XGBoost grows level-wise by default (though it has a lossguide mode), is the most battle-tested with the deepest ecosystem support (SHAP integrations, serving runtimes, every cloud platform), and its modern histogram tree method closed most of the historical speed gap with LightGBM. CatBoost's differentiator is categorical features: ordered target statistics computed over random permutations of the data give leak-resistant target encoding automatically, and ordered boosting reduces the target leakage inherent in standard boosting; it tends to win when data is dominated by high-cardinality categoricals (city, merchant, SKU) and is famously strong with default hyperparameters, at the price of slower training. Honest guidance for an interview: on most tabular problems the three land within noise of each other once tuned, so choose on workflow, LightGBM for iteration speed on big data, CatBoost to skip categorical engineering and tuning effort, XGBoost for ecosystem maturity and serving; and say you would benchmark all three under the same CV split, because that is what winning Kaggle and industry teams actually do before ensembling them.

Answer

Rank the knobs by leverage. First tier: learning_rate and n_estimators together (lower rate with more trees generalises better; fix the rate at 0.05-0.1 and let early stopping find the tree count), and tree complexity (max_depth for XGBoost, typically 4-8; num_leaves for LightGBM, typically 31-255, with min_data_in_leaf as the brake). Second tier: subsample (row fraction per tree) and colsample_bytree (feature fraction), both usually 0.7-0.9, adding randomness that decorrelates trees and fights overfitting.

Third tier: regularization terms, min_child_weight or min_data_in_leaf against tiny overfit leaves, gamma (min split gain), lambda/alpha for L2/L1 on leaf weights, and scale_pos_weight for imbalance. Tuning grids over all of these is wasteful; tune tier one, then tier two, or hand the space to Optuna. Early stopping is where candidates slip.

The mechanism: pass an eval_set, and training halts when the validation metric has not improved for early_stopping_rounds (typically 50-100) iterations, keeping the best iteration. The subtleties that get probed: the early-stopping set must be a genuinely held-out set, if you early-stop on the same fold you report, your metric is optimistically biased because you selected the tree count on it; inside k-fold CV, either carve a stopping set out of each fold's training portion, or accept the small bias and say so explicitly. After choosing hyperparameters, a common production pattern is to retrain on all data with the tree count fixed at roughly the average best iteration from CV (scaled up slightly for the larger data). And never early-stop on training loss, boosting reduces training loss essentially forever, so it would never stop.

import lightgbm as lgb
from sklearn.model_selection import train_test_split

X_tr, X_stop, y_tr, y_stop = train_test_split(
    X_train, y_train, test_size=0.15, stratify=y_train, random_state=42
)

model = lgb.LGBMClassifier(
    n_estimators=5000,          # ceiling; early stopping picks the real count
    learning_rate=0.05,
    num_leaves=63,
    min_data_in_leaf=50,
    subsample=0.8,
    colsample_bytree=0.8,
    reg_lambda=1.0,
    n_jobs=-1,
)
model.fit(
    X_tr, y_tr,
    eval_set=[(X_stop, y_stop)],   # held out from training, NOT the test set
    eval_metric="auc",
    callbacks=[lgb.early_stopping(100), lgb.log_evaluation(200)],
)
print("best iteration:", model.best_iteration_)

Answer

Three ensembling philosophies. Bagging trains many high-variance models independently on bootstrap samples and averages them: it attacks variance, trains in parallel, and is hard to overfit by adding members; random forest is bagging plus feature subsampling. Boosting trains weak learners sequentially, each correcting the ensemble's current errors via gradients of the loss: it attacks bias, must train serially, and will overfit if you let rounds run unchecked, which is why learning rate and early stopping exist.

The one-line contrast interviewers want: bagging averages away independent mistakes; boosting compounds corrections to shared mistakes. Stacking is different in kind: train several diverse base models, then train a meta-model whose inputs are the base models' predictions and whose output is the final prediction. The critical implementation detail is that the meta-model must be trained on out-of-fold predictions: each base model predicts each training row from a fold where that row was held out.

Training the meta-learner on in-sample base predictions teaches it to trust whichever base model memorised training data hardest, and the stack collapses in production. Keep the meta-model simple, logistic or ridge regression is standard, because its job is learning blend weights, not new structure. When to actually use stacking: competitions, where the last 0.2% of metric decides rankings, and high-value batch problems (credit risk, insurance pricing) where a small lift is worth real money and inference latency is irrelevant.

When not to: most production systems, because you now serve, monitor, retrain, and debug N+1 models for a marginal gain; a well-tuned single LightGBM plus better features usually beats a mediocre stack. Weighted averaging of two or three diverse models captures most of stacking's benefit at a fraction of its complexity, and saying so is a strong production-judgment signal.

Answer

Gain-based (impurity) importance, the default plot everyone prints from XGBoost, sums the loss reduction each feature contributed across all its splits. It is free, but it is computed on training data, so it inflates features the model used to overfit; it splits credit arbitrarily among correlated features; and impurity-based variants are biased toward high-cardinality and continuous features simply because they offer more split points. Treat it as a rough sketch, never as evidence.

Permutation importance asks a better question: shuffle one feature's values on held-out data and measure how much the metric drops. It is model-agnostic and evaluated out-of-sample, so it reflects what the model actually relies on for generalisation. Its failure mode is correlated features: shuffling one of a correlated pair creates unrealistic rows (age 23 with 30 years of work experience) and understates importance because the model recovers the signal from the surviving twin; grouping correlated features and permuting them together mitigates this.

SHAP computes per-prediction additive attributions grounded in Shapley values: for each row, feature contributions sum exactly to the difference between that prediction and the base rate, with TreeSHAP making this fast for tree ensembles. That gives you both global views (mean absolute SHAP, dependence plots that expose interactions) and the local explanations you need for 'why was this specific loan declined?', which matters under India's DPDP-era expectations and lender explainability norms. SHAP's caveats: it explains the model, not the world, a leaky feature gets high SHAP too; correlated features still split credit; and the interventional-versus-observational estimation choice changes numbers. The mature workflow: gain for a quick look, permutation for honest global ranking, SHAP for per-decision explanations and debugging suspicious features.

import numpy as np
import shap
from sklearn.inspection import permutation_importance

model.fit(X_train, y_train)

# 1) Gain importance: fast, train-data biased, sketch only
gain = model.feature_importances_

# 2) Permutation importance: out-of-sample, what the model relies on
perm = permutation_importance(
    model, X_val, y_val, scoring="roc_auc", n_repeats=10, random_state=42
)
for i in np.argsort(perm.importances_mean)[::-1][:5]:
    print(f"{feature_names[i]:30s} drop={perm.importances_mean[i]:.4f}")

# 3) SHAP: per-row attributions that sum to (prediction - base rate)
explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X_val)
shap.summary_plot(shap_values, X_val, feature_names=feature_names)
# One declined application, explained:
# shap.force_plot(explainer.expected_value, shap_values[i], X_val.iloc[i])

Answer

Grid search evaluates every combination in a Cartesian grid. It is exhaustive, embarrassingly parallel, and exponentially wasteful: with six hyperparameters at five values each you are running 15,625 fits, most of them varying parameters that barely matter. Random search samples configurations independently from specified distributions, and the classic Bergstra-Bengio result explains why it dominates grid: when only two of six dimensions really matter, random search effectively probes many distinct values along the important dimensions, while grid search wastes its budget re-testing the same few values of them.

Random search also lets you use continuous and log-uniform distributions (learning rate should be sampled log-uniformly, e.g. 1e-3 to 0.3) and can be stopped at any budget. Bayesian optimisation goes further: it fits a cheap surrogate model (Gaussian process or, in Optuna's TPE, tree-structured density estimators) mapping hyperparameters to observed scores, and picks the next trial by balancing exploitation of promising regions against exploration of uncertain ones. Add pruning (successive halving / Hyperband, or Optuna's median pruner), which kills clearly-losing trials after a few CV folds or boosting rounds, and you typically reach grid-quality optima in a tenth of the compute.

Practical protocol worth stating: fix a CV scheme and seed first so trials are comparable; tune the high-leverage parameters, learning rate, tree complexity, subsampling, over log-scaled ranges; use early stopping inside each trial rather than tuning n_estimators; and hold out an untouched test set, because the tuning process overfits the validation folds, the more trials you run the more the best CV score flatters you. Also know when not to tune: gradient boosting near-defaults are strong, and an hour of feature engineering routinely beats a day of hyperparameter search on tabular problems.

import optuna
import lightgbm as lgb
from sklearn.model_selection import cross_val_score, StratifiedKFold

cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)

def objective(trial):
    params = {
        "learning_rate": trial.suggest_float("learning_rate", 1e-3, 0.3, log=True),
        "num_leaves": trial.suggest_int("num_leaves", 15, 255, log=True),
        "min_data_in_leaf": trial.suggest_int("min_data_in_leaf", 10, 200),
        "subsample": trial.suggest_float("subsample", 0.6, 1.0),
        "colsample_bytree": trial.suggest_float("colsample_bytree", 0.6, 1.0),
        "reg_lambda": trial.suggest_float("reg_lambda", 1e-3, 10, log=True),
        "n_estimators": 800,
    }
    model = lgb.LGBMClassifier(**params, n_jobs=-1, verbosity=-1)
    return cross_val_score(model, X, y, cv=cv, scoring="roc_auc").mean()

study = optuna.create_study(direction="maximize",
                            pruner=optuna.pruners.MedianPruner())
study.optimize(objective, n_trials=60, show_progress_bar=False)
print(study.best_value, study.best_params)

Answer

The tool is ColumnTransformer wrapping per-type sub-pipelines, composed with the estimator into one Pipeline object. Numeric columns get imputation then scaling; categorical columns get imputation then encoding (one-hot for low cardinality, target encoding for high); text columns get TF-IDF or an embedding transformer. The payoff is not tidiness, it is correctness and deployability, and that is what the question is really testing.

Correctness: every statistic the preprocessing learns, imputation medians, scaler means, encoder categories, TF-IDF vocabulary and IDF weights, is fit only on training folds when the whole pipeline goes through cross_val_score or a hyperparameter search, which structurally eliminates preprocessing leakage; and a search can tune preprocessing choices jointly with model parameters (should min_frequency for rare categories be 20 or 100?), which is impossible when preprocessing happens in ad-hoc notebook cells. Deployability: the fitted pipeline is one artefact; pickle or ONNX it, and serving code calls predict on raw-ish rows with no risk of the training notebook and the serving service disagreeing about how to encode 'city_tier', which is one of the classic sources of training-serving skew. Details that separate candidates: handle_unknown='ignore' on OneHotEncoder so an unseen category at inference produces zeros instead of an exception; remainder='drop' versus 'passthrough' as an explicit decision, silently passing through an ID column is a leakage bug; set_output(transform='pandas') to keep feature names flowing for SHAP and debugging; and custom logic belongs in a FunctionTransformer or a small custom transformer implementing fit/transform, so it rides inside the artefact instead of living as a preprocessing script someone forgets to run in production.

from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression

num_cols = ["age", "monthly_income", "orders_90d"]
cat_cols = ["city_tier", "device_os"]
text_col = "last_support_ticket"

pre = ColumnTransformer(
    [
        ("num", Pipeline([("imp", SimpleImputer(strategy="median")),
                           ("sc", StandardScaler())]), num_cols),
        ("cat", Pipeline([("imp", SimpleImputer(strategy="most_frequent")),
                           ("ohe", OneHotEncoder(handle_unknown="ignore",
                                                  min_frequency=25))]), cat_cols),
        ("txt", TfidfVectorizer(max_features=3000, ngram_range=(1, 2)), text_col),
    ],
    remainder="drop",  # explicit: never silently pass through IDs
)

clf = Pipeline([("pre", pre), ("model", LogisticRegression(max_iter=1000))])
clf.fit(df_train, y_train)          # one artefact: preprocessing + model
print(clf.score(df_val, y_val))

Answer

Organise the answer around a prediction timestamp, because every feature must be computable strictly before it. Recency-frequency-monetary is the backbone: days since last transaction, transaction counts over multiple windows (7/30/90 days), total and average spend per window. Trend features usually outrank levels for churn: ratio of 30-day activity to 90-day activity, week-over-week decline in sessions, slope of monthly spend, a user whose orders fell from twelve to three is a stronger churn signal than a user steady at three.

Behavioural mix: category diversity, share of orders using discounts (discount-only users churn when coupons stop), payment-method mix, complaint and refund counts, support-ticket recency. Lifecycle: account age, days since first order, gap statistics like mean and variance of inter-purchase intervals, a user whose typical gap is 5 days going 20 days silent is anomalous, while the same silence is normal for a monthly shopper, which is why normalised recency (current gap divided by personal median gap) is one of the strongest single features in practice. Calendar context: festival-season flags matter in India because Diwali-month activity spikes make raw recency misleading.

The care list is what earns the offer. Every aggregate must be as-of the prediction date: 'lifetime order count' computed over the full table includes post-label orders and leaks. The label itself needs a definition with a window ('no order in the next 60 days'), and features must stop where the label window starts.

Use a point-in-time join (as implemented by feature stores like Feast) or generate multiple historical snapshots per user rather than one row at 'now'. And check feature freshness in serving: a 30-day aggregate computed nightly is stale by up to 24 hours, and if training used perfectly fresh values you have built in training-serving skew.

import pandas as pd

# txns: user_id, txn_time, amount; snapshot_date = prediction moment
snapshot = pd.Timestamp("2026-06-01")
past = txns[txns.txn_time < snapshot]          # HARD as-of cutoff: no future rows

def window_aggregates(days):
    w = past[past.txn_time >= snapshot - pd.Timedelta(days=days)]
    g = w.groupby("user_id").agg(
        **{f"txn_cnt_{days}d": ("amount", "size"),
           f"spend_{days}d": ("amount", "sum")})
    return g

feat = window_aggregates(30).join(window_aggregates(90), how="outer").fillna(0)
feat["trend_30_90"] = feat.txn_cnt_30d / feat.txn_cnt_90d.clip(lower=1)

last_seen = past.groupby("user_id").txn_time.max()
feat["recency_days"] = (snapshot - last_seen).dt.days
gaps = past.sort_values("txn_time").groupby("user_id").txn_time.diff().dt.days
feat["recency_vs_median_gap"] = feat.recency_days / gaps.groupby(
    past.user_id).median().clip(lower=1)

# Label: churn = no transaction in (snapshot, snapshot + 60d]; built separately.

Answer

A linear SVM finds the separating hyperplane that maximises the margin, the distance to the nearest training points, on the argument that a fat margin generalises better than one that merely separates. Only the points on or inside the margin, the support vectors, determine the solution; every other point could be deleted without changing the model. Real data is not separable, so the soft-margin formulation adds slack variables and the C parameter, which prices margin violations: large C tolerates few violations (tight fit, higher variance), small C buys a wider margin at the cost of training errors (more bias, more regularization).

Equivalently, the SVM minimises hinge loss plus L2, a useful framing because it connects to the regularized-loss view of every other linear model. The kernel trick: the dual formulation touches training data only through inner products x_i . x_j, so you can replace every inner product with a kernel function k(x_i, x_j) that equals an inner product in some higher-dimensional (possibly infinite-dimensional) space, without ever computing coordinates there. The RBF kernel exp(-gamma * ||x_i - x_j||^2) is the workhorse: gamma controls locality, high gamma yields wiggly, nearly-KNN behaviour (variance), low gamma approaches linear (bias); C and gamma are tuned together on a log grid.

The honest 2026 positioning interviewers respect: kernel SVMs scale poorly, training is roughly quadratic-plus in n and prediction cost grows with support-vector count, so they lost tabular ML to gradient boosting and perception to deep nets. They remain sensible for small-to-medium datasets (up to tens of thousands of rows) with many features, especially wide-data domains like bioinformatics and text with TF-IDF where linear SVMs (LinearSVC, or SGDClassifier with hinge loss for streaming scale) are still strong, fast baselines. Remember they output margins, not probabilities; getting probabilities requires calibration.

Answer

Without non-linearities, depth is an illusion: a stack of linear layers composes to W3(W2(W1 x)) = (W3 W2 W1) x, a single linear map, so a 50-layer 'network' has exactly the expressive power of one matrix multiply. Non-linear activations between layers break this collapse and give depth its meaning: each layer can bend the representation, and the universal approximation results only hold with non-linearity present. Choosing among them is mostly a story about gradients.

Sigmoid squashes to (0, 1) and saturates at both ends, where its derivative (at most 0.25) vanishes; chained through many layers, gradients shrink geometrically, which is the classic vanishing-gradient failure, and its non-zero-centred output slows optimisation. Verdict: never in hidden layers; correct at the output for binary probability. Tanh is the zero-centred cousin, still saturating, mostly seen inside legacy recurrent cells.

ReLU, max(0, x), made deep training practical: no saturation for positive inputs, gradient exactly 1 there, and computationally trivial. Its flaw is dying ReLU: a unit pushed into the negative region for all inputs gets zero gradient forever, often triggered by high learning rates; LeakyReLU gives the negative side a small slope as insurance. GELU, x times the Gaussian CDF of x, is a smooth ReLU relative that weights inputs by their magnitude probabilistically; it is the default in transformers (BERT and the GPT family onward), with SiLU/Swish close cousins appearing across modern LLM and vision architectures. Practical guidance to close with: ReLU remains a fine default for MLPs and CNNs, GELU/SiLU for transformer-style architectures, sigmoid/softmax only at outputs matched to the loss, and pair your final layer with the numerically stable combined losses (BCEWithLogitsLoss, CrossEntropyLoss on raw logits) rather than applying sigmoid or softmax manually.

import torch
import torch.nn as nn

class TabularNet(nn.Module):
    def __init__(self, n_features, hidden=128):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(n_features, hidden),
            nn.GELU(),                      # smooth ReLU relative; transformer default
            nn.Dropout(0.2),
            nn.Linear(hidden, hidden),
            nn.GELU(),
            nn.Dropout(0.2),
            nn.Linear(hidden, 1),           # raw logit: no sigmoid here
        )

    def forward(self, x):
        return self.net(x).squeeze(-1)

model = TabularNet(n_features=40)
loss_fn = nn.BCEWithLogitsLoss()            # sigmoid fused in, numerically stable
x = torch.randn(256, 40)
y = torch.randint(0, 2, (256,)).float()
loss = loss_fn(model(x), y)
loss.backward()                              # gradients flow through GELU
print(loss.item())

Answer

Backpropagation is reverse-mode automatic differentiation applied to the network's computation graph: an efficient way to compute the gradient of one scalar loss with respect to every parameter, in a single backward pass costing roughly the same as the forward pass. The forward pass computes each layer's output and caches the intermediate activations. The backward pass starts with dLoss/dOutput and applies the chain rule layer by layer in reverse: each layer receives the gradient of the loss with respect to its output, multiplies by its local derivatives, and emits two things, gradients with respect to its weights (accumulated for the optimiser step) and the gradient with respect to its input (passed to the previous layer).

For a linear layer y = Wx + b, the local computations are concrete: dL/dW = dL/dy times x-transpose, dL/db = dL/dy summed over the batch, dL/dx = W-transpose times dL/dy. That is the whole algorithm; a framework like PyTorch records the graph during the forward pass and loss.backward() replays it in reverse. The details interviewers probe: why cache activations, because local derivatives need forward values (dReLU needs the sign of the pre-activation; dL/dW needs x), and this cache is exactly why training memory scales with batch size and depth while inference memory does not, and why gradient checkpointing trades recomputation for memory when fine-tuning large models.

Why reverse mode rather than forward mode: reverse mode computes gradients of one output with respect to millions of inputs in one pass, which matches the loss-versus-parameters shape of deep learning; forward mode would need one pass per parameter. And the practical bugs: forgetting optimizer.zero_grad() so gradients accumulate across steps, detaching tensors accidentally so gradients stop flowing, and in-place operations that corrupt cached values needed by the backward pass.

Answer

During backpropagation, gradients are products of many per-layer Jacobians. When those factors are typically smaller than one, the product shrinks geometrically with depth: early layers receive vanishingly small gradients and effectively stop learning, which is why pre-2015 deep networks with sigmoid or tanh activations stalled. When factors exceed one, the product blows up: exploding gradients produce wild parameter swings and NaN losses, classically in recurrent networks where the same weight matrix multiplies the gradient at every time step.

The remedies stack. Activation choice: ReLU-family functions keep the derivative at 1 on the active side, removing the saturation shrink factor. Initialisation: He or Xavier scaling sets weight variances so activation and gradient magnitudes stay roughly constant across layers at the start.

Batch normalization normalises each layer's pre-activations to zero mean and unit variance over the mini-batch (with learned scale and shift), which keeps activations out of saturation regions, smooths the optimisation landscape, permits larger learning rates, and adds mild regularization noise; its serving subtlety is that it behaves differently at inference (running statistics, not batch statistics), which is a classic train/eval-mode bug, and it degrades with very small batches, which is why layer normalization (per-sample, batch-independent) is what transformers use. Residual connections attack depth directly: a block computes x + F(x), so the gradient reaching earlier layers includes an identity term, a highway that neither shrinks nor explodes, letting gradients skip past many blocks. That is what made 100+ layer ResNets trainable and it is equally load-bearing in transformers, where every attention and MLP block is wrapped in a residual. For explosion specifically, gradient clipping (rescale the gradient norm above a threshold) remains the standard fix, and clipping norms are worth watching as a training-health metric.

Answer

Dropout zeroes each hidden unit independently with probability p during training and scales the survivors by 1/(1-p) (inverted dropout), so the expected activation magnitude matches inference, where dropout is off entirely. Two complementary intuitions: it prevents co-adaptation, no unit can rely on a specific partner existing, so features must be individually useful; and it approximates training an exponential ensemble of thinned sub-networks whose predictions are averaged at test time. The train/eval distinction is the classic bug: forgetting model.eval() at inference leaves dropout active and silently degrades predictions, and it also breaks reproducibility of validation metrics.

Typical rates: 0.1-0.3 for tabular MLPs and transformer blocks, up to 0.5 in old-style large dense layers; dropout on inputs is rarer and gentler. It composes oddly with batch normalization (variance mismatch), one reason modern conv nets lean on BN plus data augmentation instead of heavy dropout. The rest of the regularization toolkit you should name: weight decay, L2 shrinkage implemented properly as decoupled AdamW; early stopping on a validation metric, the cheapest and most universal; data augmentation, the strongest regularizer where it applies (crops and flips for vision, mixup/cutmix, token masking and paraphrase for text), because it encodes invariances directly; label smoothing, softening one-hot targets to reduce overconfidence; reducing capacity or sharing weights; and normalization layers' own noise. The senior framing: in deep learning, more data and augmentation beat penalty terms, early stopping is always on, and dropout is one dial among several rather than the answer; on small tabular datasets, the honest regularizer is often switching to gradient boosting.

import torch
import torch.nn as nn

mlp = nn.Sequential(
    nn.Linear(64, 256), nn.GELU(), nn.Dropout(p=0.3),
    nn.Linear(256, 256), nn.GELU(), nn.Dropout(p=0.3),
    nn.Linear(256, 1),
)
opt = torch.optim.AdamW(mlp.parameters(), lr=1e-3, weight_decay=1e-2)

x = torch.randn(8, 64)

mlp.train()
print("train mode differs across calls (dropout active):")
print(mlp(x)[0].item(), mlp(x)[0].item())   # two different outputs

mlp.eval()                                   # dropout off, deterministic
with torch.no_grad():
    print("eval mode is stable:", mlp(x)[0].item(), mlp(x)[0].item())
# Forgetting model.eval() in serving is the classic dropout bug.

Answer

Plain SGD steps opposite the mini-batch gradient. Momentum adds an exponential moving average of past gradients (beta around 0.9) and steps along that velocity instead: consistent directions accumulate speed while oscillating directions cancel, which damps the zig-zag across ravines in the loss surface and accelerates progress along the valley floor. Adam keeps two EMAs per parameter, the mean of gradients (first moment, momentum) and the mean of squared gradients (second moment), and divides the step by the root of the second moment: parameters with consistently large gradients get smaller effective steps, parameters with small or rare gradients get larger ones.

That per-parameter adaptivity makes Adam robust to learning-rate choice and ill-scaled problems, and it converges fast out of the box, which is why it became the deep learning default. Bias correction compensates for the zero-initialised EMAs early in training. Adam's subtle flaw: adding classic L2 penalty to the loss gets divided by the adaptive denominator too, so effective weight decay varies per parameter and regularization is inconsistent.

AdamW decouples the decay, applying it directly to weights outside the adaptive update, which restores clean regularization; this is why AdamW is the standard optimiser for transformers and most modern training recipes, with typical betas (0.9, 0.999 or 0.95 for LLMs) and weight decay 0.01-0.1. Selection guidance: AdamW is the safe default for transformers, tabular MLPs, and fine-tuning; SGD with momentum plus a tuned schedule can still edge out Adam on final generalisation for conv nets, at the cost of more tuning effort; and whatever the optimiser, the learning rate and its schedule dominate, an optimiser change never rescues a bad schedule. Mention that optimiser state doubles or triples memory per parameter (two EMAs), which is a real constraint when fine-tuning large models and the motivation for memory-efficient variants like 8-bit optimisers.

Answer

An embedding is a learned mapping from a discrete item, a word, a user ID, a product, a pincode, into a dense real-valued vector, trained so that geometric closeness encodes task-relevant similarity. Mechanically it is a lookup table, an n_items by d matrix whose rows are updated by backpropagation like any other weights; the item's index selects a row, and gradients flow into exactly that row. Embeddings solve the two failures of one-hot encoding at scale: dimensionality (100,000 products become 64 dense dimensions instead of 100,000 sparse ones) and, more importantly, generalisation, one-hot vectors are all equidistant, carrying no notion that two Mumbai pincodes are more alike than a Mumbai and a Guwahati one, while trained embeddings place similar items near each other so the model transfers what it learns across them.

Training routes: as a component of a supervised model (an embedding layer for each categorical feature in a click-through or churn model, the standard deep-tabular pattern); via self-supervised objectives (word2vec's predict-the-neighbour idea generalises to 'product2vec' on co-purchase sequences and session2vec on browsing); or via contrastive learning, pulling positive pairs together and pushing negatives apart, which is how modern sentence and image embedding models are built. Uses beyond NLP that interviewers want to hear: candidate retrieval in recommenders (embed users and items, retrieve nearest items for a user vector, the two-tower pattern behind feeds at Flipkart-scale marketplaces), semantic search over support tickets and job descriptions, clustering and deduplication of catalogue entries, cold-start mitigation by embedding content attributes, and anomaly detection as distance from normal-behaviour clusters. Practical notes: embedding dimension is a tuned hyperparameter (a common heuristic is the fourth root of cardinality, and fast.ai uses 1.6 * n^0.56, but CV decides), rare categories need a shared 'unknown' row to avoid overfit rows, and cosine similarity on L2-normalised vectors is the default retrieval metric.

import torch
import torch.nn as nn

class ChurnNet(nn.Module):
    """Deep-tabular pattern: embeddings for categoricals + numeric features."""
    def __init__(self, n_pincodes=19000, n_devices=500, n_numeric=20):
        super().__init__()
        self.pin_emb = nn.Embedding(n_pincodes, 32)   # 19000 -> 32 dims
        self.dev_emb = nn.Embedding(n_devices, 8)
        self.mlp = nn.Sequential(
            nn.Linear(32 + 8 + n_numeric, 128), nn.GELU(), nn.Dropout(0.2),
            nn.Linear(128, 1),
        )

    def forward(self, pin_idx, dev_idx, numeric):
        z = torch.cat([self.pin_emb(pin_idx),
                       self.dev_emb(dev_idx), numeric], dim=-1)
        return self.mlp(z).squeeze(-1)

model = ChurnNet()
logit = model(torch.tensor([1204]), torch.tensor([17]), torch.randn(1, 20))
# After training, model.pin_emb.weight rows cluster similar pincodes:
sim = torch.cosine_similarity(model.pin_emb.weight[1204],
                              model.pin_emb.weight[1205], dim=0)

Answer

TF-IDF represents a document as a sparse vector of vocabulary size, weighting each term by its frequency in the document discounted by how common it is across the corpus. It is exact-match by construction: 'payment failed' and 'transaction declined' share no tokens, so their TF-IDF similarity is zero despite meaning nearly the same thing. Its strengths are real and current: training-free, interpretable (you can point at the words responsible for a match or classification), extremely fast, strong on keyword-heavy tasks and rare exact tokens (error codes, SKUs, legal citations), and with a linear SVM or logistic regression on top it remains a startlingly hard baseline for topical text classification.

Learned embeddings from sentence-transformer-style models map text to dense vectors where semantic similarity becomes geometric proximity, so paraphrases land together, and multilingual models place Hindi, Hinglish, and English expressions of one intent near each other, directly relevant for Indian support-ticket routing and job-description matching where users mix scripts. Costs: an inference pass per text, a fixed context window, domain mismatch when the pretraining corpus differs from yours, and lost token-level interpretability. Decision rules to state: for search and retrieval, hybrid is the 2026 production standard, BM25 (the TF-IDF-family ranker) catches exact identifiers while dense retrieval catches paraphrase, and reciprocal-rank fusion or a reranker merges them; for classification with abundant labelled data, benchmark TF-IDF plus linear model first, it wins more often than candidates expect and deploys trivially; for few-shot or paraphrase-heavy tasks (dedup, intent, semantic FAQ matching), embeddings win decisively; and for anything user-facing where you must explain matches, TF-IDF's transparency is itself a feature. Saying 'I would benchmark both, TF-IDF is my baseline' signals engineering maturity better than defaulting to the fanciest encoder.

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity
from sentence_transformers import SentenceTransformer

docs = [
    "payment failed during checkout",
    "transaction declined at the payment step",
    "how do I change my delivery address",
]

# Sparse lexical: no shared tokens -> near-zero similarity for paraphrases
tfidf = TfidfVectorizer().fit_transform(docs)
print("tfidf sim(0,1):", round(cosine_similarity(tfidf[0], tfidf[1])[0, 0], 3))

# Dense semantic: paraphrases land close together
enc = SentenceTransformer("all-MiniLM-L6-v2")
emb = enc.encode(docs, normalize_embeddings=True)
print("dense sim(0,1):", round(float(emb[0] @ emb[1]), 3))  # high
print("dense sim(0,2):", round(float(emb[0] @ emb[2]), 3))  # low
# Production search: BM25 for exact codes + dense for meaning, fused.

Answer

Self-attention lets every token in a sequence gather information from every other token in one step. Each token's representation is projected into three vectors: a query (what am I looking for), a key (what do I contain), and a value (what do I contribute). Token i's output is a weighted average of all tokens' values, with weights softmax(q_i . k_j / sqrt(d)): the dot product scores relevance, the sqrt(d) scaling keeps softmax gradients healthy, and the softmax turns scores into a distribution.

Multi-head attention runs several such attentions in parallel over subspaces, letting different heads track different relationships (syntax, coreference, positional patterns), and concatenates the results. A transformer block wraps attention and a position-wise feed-forward network with residual connections and layer normalization; stacking blocks builds the model. Because attention itself is permutation-invariant, positional information must be injected explicitly, learned positions in early models, rotary embeddings (RoPE) in most modern LLMs.

Encoder-style models (BERT lineage) attend bidirectionally and suit understanding tasks; decoder-style models (GPT lineage) use causal masks so each token sees only its past, enabling autoregressive generation. Why it displaced RNNs: recurrence processes tokens serially, so training cannot parallelise across sequence length and information between distant tokens must survive many squashed steps, vanishing along the way; attention connects any two positions in one hop and trains fully in parallel on GPUs, which unlocked the scale that made modern LLMs possible. The trade you should name: attention is quadratic in sequence length in compute and memory, which is why long-context work centres on KV caching for inference, and attention variants (sliding-window, linear, sparse) for efficiency. For a working-level interview, being able to write the softmax(QK^T/sqrt(d))V formula and explain causal masking is usually the bar.

Answer

Both start from a pretrained model. Feature extraction freezes the backbone and trains only a new head: pass your data through the frozen network, treat an internal representation as fixed features, and fit a small classifier (even logistic regression) on top. Fine-tuning unfreezes some or all backbone weights and continues training on your task, adapting the representations themselves.

The decision hinges on data size and domain distance. Little data plus a domain close to pretraining (classifying product photos with an ImageNet-pretrained backbone, routing English support tickets with a standard sentence encoder): feature extraction, because updating millions of parameters on thousands of examples mostly buys overfitting. More data or a distant domain (medical imaging, legal Hindi text, code): fine-tune, since frozen features encode the wrong invariances; the standard middle path unfreezes the top few blocks while keeping early generic layers frozen.

Fine-tuning hygiene interviewers listen for: learning rates one to two orders smaller than training from scratch (1e-5 to 1e-4 for transformers), warmup, and optionally discriminative rates that decay toward earlier layers, all to avoid catastrophically forgetting the pretrained knowledge in the first noisy steps. LoRA (low-rank adaptation) reframes fine-tuning for large models: freeze the pretrained weight matrices W and learn only a low-rank update, W + BA where B and A have rank r (say 8-64), cutting trainable parameters by orders of magnitude, shrinking optimiser state and memory accordingly, and making each task's adaptation a few-megabyte adapter you can hot-swap over one shared base model. QLoRA pushes further by keeping the frozen base in 4-bit quantisation, which is what lets a 7-billion-parameter model be fine-tuned on a single consumer GPU. In 2026, parameter-efficient fine-tuning is the default for LLM adaptation, while full fine-tuning persists where data is plentiful and the budget allows.

import torch.nn as nn
from torchvision import models

# Feature extraction: freeze backbone, train the new head only
model = models.resnet50(weights="IMAGENET1K_V2")
for p in model.parameters():
    p.requires_grad = False
model.fc = nn.Linear(model.fc.in_features, 4)   # new head: trainable

# Staged fine-tuning: also unfreeze the last block, small LR, param groups
for p in model.layer4.parameters():
    p.requires_grad = True

import torch
opt = torch.optim.AdamW([
    {"params": model.fc.parameters(), "lr": 1e-3},
    {"params": model.layer4.parameters(), "lr": 1e-5},  # gentle on pretrained
], weight_decay=1e-2)

trainable = sum(p.numel() for p in model.parameters() if p.requires_grad)
print(f"trainable params: {trainable:,}")  # tiny fraction of the full model

Answer

This remains true in 2026 and the benchmarks agree: on medium-sized tabular datasets, tuned gradient boosting (XGBoost, LightGBM, CatBoost) matches or beats deep tabular architectures most of the time, and it does so with minutes of training and little preprocessing. The reasons are structural, and articulating them is the point of the question. Tabular data has heterogeneous features with different units, scales, and meanings, and its target functions are frequently irregular: sharp thresholds (regulatory cutoffs at age 18, credit-score bands, free-delivery minimums), interactions among small feature subsets, and no smoothness between neighbouring feature values.

Trees are built for exactly this: axis-aligned splits express thresholds natively, are invariant to monotonic transforms, handle missing values and mixed types, and are indifferent to feature scale. Neural networks carry a smoothness bias from continuous activations and gradient descent, excellent for images and text where nearby inputs share meaning, but a mismatch for cliff-shaped tabular functions; they also lack the spatial or sequential structure that convolutions and attention exploit, so the architecture brings no useful prior. Uninformative features hurt MLPs more than trees, and typical tabular sample sizes (thousands to low millions of rows) are small by deep learning standards.

Deep learning wins on tabular when specific conditions hold: very large datasets with high-cardinality categoricals where learned embeddings shine (click-through prediction at ad-tech scale); multimodal problems where tabular columns join text, images, or sequences in one network; representation reuse across many related tasks; and online-learning settings where continuous gradient updates fit the serving pattern. The pragmatic answer that lands in interviews at Flipkart or Razorpay: baseline with LightGBM, always; add embedding-based deep models when text and behaviour sequences enter the problem; and consider that TabPFN-style pretrained tabular transformers are now credible for small datasets, a recent development worth naming.

Answer

The task: given orders(user_id, order_time, amount), compute for each month how many of that month's active users were also active the following month. This is the canonical ML-adjacent SQL question at product companies, because retention cohorts feed churn labels and dashboards alike. The clean structure is three steps in CTEs: first, deduplicate to user-months (one row per user per active month), because a user with thirty orders in June must count once; second, self-join user-months to the next month with a one-month offset on the same user; third, aggregate to a rate per month.

Interviewers check specific things. Deduplication first: skipping SELECT DISTINCT (or GROUP BY) inflates both numerator and denominator and is the most common failure. The join direction: LEFT JOIN from the current month keeps users who did not return, so the denominator is all active users, not just retained ones; an INNER JOIN silently computes the wrong rate.

Correct month arithmetic across year boundaries: date functions, not month-number arithmetic, so December to January works. COUNT(DISTINCT ...) discipline in the final aggregate. And an articulate follow-up matters: extending to cohort retention (group users by first-ever month and track each cohort across offsets of 1, 2, 3 months) is the usual second ask, which reuses the same skeleton with a MIN(month) OVER (PARTITION BY user_id) cohort assignment. Voicing the edge cases unprompted, timezone of order_time, whether refunded orders count as activity, and that calendar-month boundaries make a user active on the 31st and 1st look retained after one day, is what separates a memorised answer from someone who has built retention reporting for real.

WITH user_months AS (
    SELECT DISTINCT
        user_id,
        DATE_TRUNC('month', order_time) AS m
    FROM orders
),
paired AS (
    SELECT
        cur.m,
        cur.user_id,
        nxt.user_id IS NOT NULL AS retained
    FROM user_months cur
    LEFT JOIN user_months nxt
      ON nxt.user_id = cur.user_id
     AND nxt.m = cur.m + INTERVAL '1 month'
)
SELECT
    m AS activity_month,
    COUNT(DISTINCT user_id) AS active_users,
    COUNT(DISTINCT CASE WHEN retained THEN user_id END) AS retained_next_month,
    ROUND(
      COUNT(DISTINCT CASE WHEN retained THEN user_id END)::numeric
      / COUNT(DISTINCT user_id), 4
    ) AS retention_rate
FROM paired
GROUP BY m
ORDER BY m;

Answer

Four failure classes account for most corrupted training sets. Row multiplication on merge: joining orders to a users table where user_id is unexpectedly duplicated (a re-registered device, a dirty dimension table) silently multiplies rows, so downstream aggregates double-count and the model trains on phantom data. The defence is pandas' validate argument ('many_to_one' raises immediately if the right side has duplicate keys) plus asserting row counts after every merge.

Silent row loss on inner joins: merge defaults to inner, so users missing from a dimension table vanish along with their labels, biasing the sample toward well-populated records, usually richer, older accounts; merge with how='left' deliberately and then inspect the null pattern with indicator=True. Index-alignment surprises: assigning a Series produced by one operation to a differently-indexed frame aligns on index labels, not position, quietly scattering NaNs; reset_index or use .values intentionally. And aggregation leakage: computing group statistics (mean spend per city) over the full frame before the train/test split injects test-set rows into training features, the pandas edition of preprocessing leakage; compute such statistics on the training partition and map them onto validation.

Also worth naming: groupby(...).apply on large frames is slow and better expressed with named aggregations in .agg; transform is the correct tool for broadcasting a group statistic back to every row; observed=True on categorical groupbys avoids phantom empty groups; and chained indexing (df[a][b] = x) may modify a copy, which is why pandas 2-era copy-on-write semantics and .loc assignments are the standard. Interviewers rarely ask these as trivia; they hand you a small dirty dataset and watch whether you check shapes, nulls, and duplicates after each step, so narrate those checks as you code.

import pandas as pd

# 1) Merge with validation: blow up loudly instead of multiplying rows
feat = orders.merge(
    users[["user_id", "city", "signup_date"]],
    on="user_id", how="left",
    validate="many_to_one",       # raises if users has duplicate user_id
    indicator=True,
)
assert len(feat) == len(orders), "merge changed the row count"
print(feat["_merge"].value_counts())   # how many failed to match?

# 2) Group statistic WITHOUT leakage: fit on train, map to validation
city_avg = train.groupby("city")["order_value"].mean()
train["city_avg_value"] = train["city"].map(city_avg)
val["city_avg_value"] = val["city"].map(city_avg)  # train stats only
val["city_avg_value"] = val["city_avg_value"].fillna(city_avg.mean())

# 3) transform broadcasts a group stat back to rows (same length as input)
train["share_of_city"] = train["order_value"] / train.groupby("city")[
    "order_value"].transform("sum")

Answer

Random k-fold assumes rows are exchangeable: any subset is statistically like any other. Time-indexed data violates this twice over. First, direct temporal leakage: shuffled folds train on the future and validate on the past, and features with any temporal memory, lagged values, rolling means, trend components, let the model effectively see tomorrow while being scored on yesterday, inflating offline metrics that then collapse in production.

Second, autocorrelation: adjacent observations are near-duplicates, so even without explicit time features, a random split places a row's near-twin in training, and the validation score measures interpolation between neighbours rather than genuine forward prediction. The correct scheme is forward chaining: every validation window lies strictly after its training window. sklearn's TimeSeriesSplit implements expanding-window splits (train on months 1-6, validate 7; train 1-7, validate 8; and so on); rolling-origin variants fix the training window length instead, which is preferable when old regimes should be forgotten. Refinements that mark real experience: an embargo gap between train and validation windows when features contain rolling aggregates, so a 30-day rolling feature computed at the boundary cannot straddle both sets; grouping by entity as well as time when panels have many users, so the same user-week cannot leak across the divide; evaluating per-window and inspecting the spread across windows, because a model that aced the stable months and failed the festival-season month has a seasonality problem a single averaged metric hides; and benchmarking against naive baselines (last value, seasonal-naive) since a forecasting model that cannot beat 'predict last Diwali's number' is decoration. Finally, the same discipline applies to any scoring model, churn, default, fraud, retrained over time: the honest final evaluation is out-of-time, train through month t, test on t+1 onward, because that is precisely how the model will live.

import numpy as np
from sklearn.model_selection import TimeSeriesSplit
from lightgbm import LGBMRegressor
from sklearn.metrics import mean_absolute_error

# df sorted by date; features already built as-of each row's date
df = df.sort_values("date").reset_index(drop=True)
X, y = df[feature_cols].values, df["target"].values

tscv = TimeSeriesSplit(n_splits=5, gap=7)   # 7-day embargo for rolling feats
maes, naive_maes = [], []
for fold, (tr, va) in enumerate(tscv.split(X)):
    model = LGBMRegressor(n_estimators=500, learning_rate=0.05)
    model.fit(X[tr], y[tr])
    pred = model.predict(X[va])
    maes.append(mean_absolute_error(y[va], pred))
    naive_maes.append(mean_absolute_error(y[va], y[va - 7]))  # week-ago naive
    print(f"fold {fold}: model={maes[-1]:.3f}  naive={naive_maes[-1]:.3f}")

print(f"mean MAE {np.mean(maes):.3f} vs naive {np.mean(naive_maes):.3f}")
# Report the per-fold spread too: one bad seasonal window matters.
# Caveat: y[va - 7] and gap=7 assume one row per day; on panel data
# compute the week-ago offset and embargo by date, not row position.

Answer

Decide from the freshness the decision actually needs, not from engineering ambition. Batch scoring runs the model on a schedule, nightly churn scores written to a table, weekly propensity lists for a CRM, and it is dramatically simpler: no latency budget, no service to keep alive, failures retry quietly, and features can be computed with heavyweight SQL over the warehouse. If the business acts on scores daily, real-time serving adds cost without value, and saying so is a senior signal.

Real-time serving is forced when the prediction depends on request-time information or must gate an action in-flight: fraud checks inside a payment authorisation (double-digit milliseconds at a Razorpay-scale gateway), search ranking, dynamic pricing, recommendation on page load. The minimal real-time stack: serialise the trained pipeline (preprocessing fused with the model, never separate), wrap it in a lightweight HTTP service, load the artefact once at startup rather than per request, and containerise it so the Python environment is pinned; from there, horizontal replicas behind a load balancer, a p99 latency SLO, and request/response logging for later analysis. The parts interviewers probe: feature access is the hard half, request-time features arrive in the payload, but historical aggregates ('user's 30-day order count') must come from a low-latency store precomputed by the same logic as training, which is the feature-store problem; version every model artefact and log model version with each prediction so incidents are attributable; keep a shadow or canary path for new models; and know the latency levers, ONNX or native-booster inference instead of pickled sklearn wrappers, batching requests to amortise overhead where traffic allows. A hybrid pattern covers many real systems: precompute expensive user-level scores in batch, serve them from a cache, and reserve true real-time inference for the request-dependent part.

# serve.py: minimal FastAPI model service
import joblib
import pandas as pd
from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()
MODEL_VERSION = "churn-lgbm-2026-08-01"
model = joblib.load("model_pipeline.joblib")   # preprocessing + model, ONE artefact

class ScoreRequest(BaseModel):
    user_id: int
    recency_days: float
    txn_cnt_30d: int
    txn_cnt_90d: int
    city_tier: str

@app.post("/score")
def score(req: ScoreRequest):
    row = pd.DataFrame([req.model_dump()])
    proba = float(model.predict_proba(row.drop(columns=["user_id"]))[0, 1])
    # Log (user_id, features, proba, MODEL_VERSION) for monitoring/audit
    return {"user_id": req.user_id, "churn_probability": round(proba, 4),
            "model_version": MODEL_VERSION}

# uvicorn serve:app --workers 4   |   docker build w/ pinned deps

Answer

Training-serving skew is any systematic difference between the data a model saw in training and the data it receives in production, and it is the most common reason a model with a strong offline evaluation quietly underperforms live. It comes in three flavours. Implementation skew: features computed by two codepaths, a pandas/SQL pipeline for training and a Java/Go service for serving, drift apart in details nobody notices: different null handling, different rounding, timezone offsets (IST versus UTC bucketing shifts every daily aggregate), a 30-day window implemented as 30 calendar days in one place and 720 hours in the other.

Temporal skew: training features were computed with the leisure of the warehouse, effectively fresh as-of each historical moment, while serving reads a nightly-refreshed store, so the live model sees staler aggregates than any it trained on; the offline evaluation never measured this. Distribution skew: the serving population differs from training, new user segments, a marketing campaign, a new city launch, which overlaps with drift but shows up on day one. Engineering it away: define each feature once, in one artefact, and reuse it on both sides, either by fusing preprocessing into the model pipeline (the sklearn Pipeline / ONNX route) or by adopting a feature store (Feast-style) whose point-in-time-correct offline retrieval generates training sets and whose online store serves the same definitions.

Log the features actually used at serving time alongside each prediction, then run a skew audit: join logged serving features to regenerated training-style features for the same entities and timestamps and compare distributions per feature; any gap is a bug you can now localise. Add freshness metadata to features and alert when staleness exceeds what training assumed. And for launches, shadow-score live traffic with the new model before it gates decisions, comparing its online score distribution to offline expectations; a mismatch there is skew announcing itself before it costs money.

Answer

Data drift (covariate shift) means the input distribution P(X) moved: transaction amounts trend up, a new Android version changes device features, a marketing push floods the funnel with a new demographic. The mapping X to y may be intact, but the model is extrapolating into regions it saw rarely. Concept drift means the relationship P(y|X) itself changed: fraudsters adapt to your rules, a repricing changes what predicts churn, a pandemic rewrites demand curves.

The distinction matters because their remedies differ: data drift can sometimes be absorbed by reweighting or refreshed training data, while concept drift makes accumulated history actively misleading and often calls for shorter training windows. Detection is a monitoring pipeline, not a one-off test. For inputs, compare live feature distributions per window against a training reference: Population Stability Index is the industry workhorse (banded rule of thumb: below 0.1 stable, 0.1-0.25 watch, above 0.25 investigate), with KS tests for continuous features and chi-square for categorical; watch the model's output score distribution too, since a shifting score histogram is often the earliest, cheapest alarm.

For concept drift you need outcomes, and labels lag: fraud confirmations take weeks, churn is defined over 60-day windows. Standard practice under label delay: monitor proxies that arrive fast (chargeback requests, support contacts, early repayment misses), evaluate rolling metrics on the labels as they mature and align them to prediction time, and treat rising disagreement between a stable input distribution and a moving proxy metric as concept-drift evidence. Response playbook: triage which features drifted and whether a pipeline bug (a feed gone null) explains it, since 'drift' alerts are broken pipelines more often than economics; retrain on a window that includes the new regime; and if drift is chronic, move to scheduled retraining with champion-challenger evaluation rather than ad-hoc firefighting. State the false-alarm caveat: with hundreds of features, uncorrected per-feature tests will page you weekly; prioritise by feature importance and effect size, not p-values alone.

import numpy as np

def psi(reference, live, bins=10):
    """Population Stability Index for one numeric feature."""
    edges = np.quantile(reference, np.linspace(0, 1, bins + 1))
    edges[0], edges[-1] = -np.inf, np.inf          # cover new extremes
    ref_pct = np.histogram(reference, edges)[0] / len(reference)
    live_pct = np.histogram(live, edges)[0] / len(live)
    ref_pct = np.clip(ref_pct, 1e-6, None)         # avoid log(0)
    live_pct = np.clip(live_pct, 1e-6, None)
    return float(np.sum((live_pct - ref_pct) * np.log(live_pct / ref_pct)))

for col in ["txn_amount", "recency_days", "score"]:   # score drift = early alarm
    value = psi(train_df[col].values, last_week_df[col].values)
    flag = "OK" if value < 0.1 else ("WATCH" if value < 0.25 else "ALERT")
    print(f"{col:15s} PSI={value:.3f}  {flag}")

Answer

Monitor four layers, in the order failures actually occur. System health first: latency percentiles, error rates, throughput, memory, because a model that times out is a failed model regardless of its AUC, and the serving SLO is part of the model's contract. Data health second, where most incidents originate: schema changes, per-feature null rates and out-of-range rates, feature freshness lag, and volume anomalies; a single upstream feed silently going 90% null will degrade predictions long before any statistical drift test fires, so per-feature null-rate alerts are the highest-value cheap monitor you can build.

Prediction health third: the score distribution per window (mean, histogram distance from the training reference), segment-level breakdowns (new versus tenured users, city tiers, platforms), and business-facing action rates, the fraction flagged, approved, or targeted, since a decision threshold tuned for one score distribution silently changes the action volume when scores shift. Outcome health last, as labels mature: rolling AUC or PR-AUC aligned back to prediction time, calibration (are 0.7s coming true 70% of the time), and the business metric the model exists to move, with fast proxies standing in while true labels lag. Retraining triggers come in three types and mature teams use all of them: scheduled (weekly or monthly, matched to the domain's natural drift rate, the predictable backbone); metric-triggered (matured performance or calibration breaching a threshold, or sustained PSI alerts on important features); and event-triggered (new market launch, pricing change, fraud-pattern incident, upstream schema migration).

Every retrain flows through the same gate: evaluate champion versus challenger on identical out-of-time data, require the challenger to win by a margin exceeding evaluation noise, then canary or shadow before full rollout, with one-click rollback to the previous artefact. The anti-pattern to name: retraining on an alert without diagnosis, because if the alert was a broken pipeline, the retrain launders the bug into the model.

Answer

Structure the answer as design, metrics, then pitfalls. Design: randomise at the unit that experiences the model and matches the business metric, users for recommendation and churn interventions, not requests, because one user hit by both models contaminates behaviour and violates independence. Split traffic at a stable hash of the unit ID, hold the assignment constant for the experiment's life, and size the test with a power calculation before launch: from the metric's baseline and variance, the minimum detectable effect you care about, and standard alpha 0.05 / power 0.8, derive the sample size and hence duration; run at least one full weekly cycle regardless, since weekday and weekend behaviour differ.

Before the real test, an A/A run validates the plumbing, if A versus A shows a 'significant' difference, your assignment or logging is broken. Metrics: one primary business metric declared in advance (conversion, fraud loss per transaction, retained revenue), a small set of guardrails that must not degrade (latency, cancellation rate, support contacts, approval rate), and the model's offline metric logged for diagnosis but never as the success criterion, the whole point of the online test is that offline AUC gains do not automatically become business gains. The mistakes that invalidate results, which is what the interviewer is really asking: peeking, repeatedly testing significance and stopping on the first p < 0.05 inflates false positives badly, so fix the horizon in advance or use sequential testing methods designed for continuous monitoring; multiple comparisons across many segments and metrics, if you slice until something is significant, you will find noise, so pre-register the segments; interference between arms, a fraud model that deters fraudsters changes what the control arm sees, and marketplace models can cannibalise shared inventory, which may require switchback (time-sliced) designs; novelty effects that decay, check whether the lift persists in the final week; and Simpson's-paradox traffic imbalances when ramp-up percentages changed mid-test, analyse only the stable period. Close with the operational detail: log model version per decision, and keep the holdout going after launch, a small persistent control quantifies the model's ongoing value and catches slow regressions.

Answer

Interviewers score structure, so narrate stages and the decisions inside each. Problem framing first, and slowly: define churn precisely, for a fintech with UPI-style engagement, something like 'no transaction in the next 60 days' for a user active in the prior 90; state that the definition is a product decision you would pressure-test against reactivation curves (if most 60-day-silent users never return, the label is good). Clarify the intervention, because it shapes everything: if the output feeds a retention team with capacity for 50,000 contacts a month, you need good ranking in the top 1%, not global calibration.

Data and labels: build snapshots, not one dataset 'as of today', for each historical month-end, features strictly from before the snapshot, label from the following 60 days; this yields many training rows per user, an out-of-time test by construction (train on Jan-Apr snapshots, test on May), and structurally prevents the leakage that plagues churn models. Features: RFM levels and trends, normalised recency against each user's own cadence, product-mix breadth (bill pay, investments, cards), balance and cashback patterns, support-ticket and app-error counts, KYC and onboarding completeness, with every aggregate as-of the snapshot. Modelling: LightGBM with class weighting (churn base rate maybe 5-15%), evaluated on PR-AUC and, decisively, recall and precision at the intervention budget; a logistic baseline for sanity and explainability comparison.

Validation: out-of-time, plus segment slices, new users versus tenured, because a model that only predicts 'new users churn' is rediscovering onboarding, not adding lift. Then the parts that win senior loops: SHAP-driven review of top features for leakage smells; deployment as monthly batch scoring into the CRM (real-time adds nothing here, say so); threshold set by contact-capacity economics, expected retained revenue against contact cost; drift monitoring on inputs and score distribution with quarterly champion-challenger retrains; and measurement through an uplift lens, holding out a random control from the retention campaign, because the business question is not 'who churns' but 'who is saved by contact', and mentioning uplift modelling as the v2 is the strongest possible close.

Answer

Frame the constraints before any modelling: the score must land inside the authorisation flow, so the budget is tens of milliseconds end to end including feature retrieval; positives are well under 1%; labels are delayed and noisy (chargebacks confirm over weeks, many frauds are never reported); adversaries adapt, so concept drift is guaranteed, not hypothetical; and both error directions carry real cost, missed fraud loses money directly while false declines lose good customers and merchant trust. Architecture: a tiered system, not one model. Tier one, deterministic rules and velocity checks (same card, many merchants, minutes) that catch obvious patterns cheaply and give compliance an auditable layer.

Tier two, the ML scorer: gradient boosting on features retrieved from an online store, card and device aggregates over sliding windows (counts and amounts over 5 minutes, 1 hour, 24 hours, 30 days), deviation features (this amount versus the card's typical amount, hour-of-day versus habit), merchant risk profiles, and network features like how many cards this device has touched. The sliding-window aggregates must be maintained by streaming pipelines, and their training-time counterparts must be point-in-time correct, this is where training-serving skew kills fraud systems, so say 'same feature definitions both sides' explicitly. Tier three, the decision layer: not binary, but approve, step-up (send to OTP or additional verification), or decline, with thresholds set from the cost matrix; step-up is the pressure valve that buys recall without pure declines.

Evaluation: PR-AUC offline, but the numbers that matter are fraud-loss basis points and false-decline rate at the operating thresholds, sliced by merchant category and ticket size. Handle label delay by training on matured windows and monitoring fast proxies (chargeback filings, customer disputes). Counter adaptation with frequent scheduled retrains, champion-challenger gating, and monitoring for score drift; keep the rules layer as a fast-response channel for novel attack patterns while the model catches up. Close with review-queue economics: analyst capacity bounds how many alerts are useful, so precision at the queue size is the honest ceiling on the system.

Answer

Start by decomposing the target, because the single number 'delivery time' hides three sub-processes with different drivers: restaurant preparation time (kitchen load, dish complexity, historical prep behaviour of that restaurant at that hour), rider assignment and travel to restaurant (rider supply nearby, batching decisions), and last-mile travel (distance, traffic, rain, building access). Swiggy-style systems model components separately or jointly with shared features, and saying 'I would decompose' immediately distinguishes you from candidates who fit one regressor on raw features. Features follow the decomposition: restaurant-hour historical prep quantiles, live kitchen queue depth, order composition; rider supply-demand ratio in the zone, batching state; haversine and road distance, time-of-day and day-of-week, weather, and locality effects best captured with learned embeddings for restaurant and zone IDs.

What makes it non-standard, and what the interviewer is fishing for: first, the loss is asymmetric, an ETA that is too optimistic creates a broken promise, cancellations, and support load, while a slightly conservative ETA merely looks less attractive at order time; so you do not minimise MSE, you either train quantile regression (predict the 70th-80th percentile) or a custom asymmetric loss penalising underestimates more. Second, the prediction is an intervention: the displayed ETA changes user behaviour (order or not) and operations (rider allocation deadlines), so online evaluation needs guardrails on conversion, not just error metrics. Third, feedback loops and selection: you only observe delivery times for orders actually placed, and batching decisions made using the ETA affect the realised time, a mild form of the prediction influencing its own label.

Fourth, distribution shift is constant and spiky: rain, festivals like Diwali and Eid, cricket finals; the model needs live-updating features (current zone-level delays as a feature) because no offline feature set survives a sudden downpour. Evaluation: MAE is fine for dashboards, but the product metrics are promise-breach rate (deliveries exceeding the shown ETA) against ETA competitiveness, evaluated per zone and per daypart, and improved via online A/B with switchback designs because zones interfere. This is a case where a well-chosen loss function is worth more than a bigger model, and closing on that sentence lands well.

Answer

The loss defines what the model's prediction means, and that is the sentence to lead with. Minimising MSE yields the conditional mean: differentiable everywhere, heavily penalising large errors, and therefore sensitive to outliers, one wild delivery time drags the whole fit toward it. Minimising MAE yields the conditional median: robust to outliers, but its gradient is a constant sign (plus or minus one), carrying no magnitude information, which makes gradient-based training slower and its zero-second-derivative awkward for second-order boosters like LightGBM, which substitute a workaround Hessian.

Huber loss interpolates: quadratic inside a delta-band, linear outside, so you keep informative gradients near the optimum and robustness in the tails; delta is a tunable knob that effectively decides which errors count as outliers. Quantile (pinball) loss generalises MAE asymmetrically: for quantile tau it charges tau times the error on underestimates and (1 - tau) on overestimates, so minimising it yields the conditional tau-quantile; train tau = 0.1, 0.5, 0.9 models and you have prediction intervals, and train tau = 0.8 when the business cost of under-promising and over-promising is asymmetric, the ETA case exactly. Log-cosh approximates Huber with smooth second derivatives.

The mapping from business cost to loss choice is the interview's real content: symmetric costs and trustworthy data, MSE; heavy-tailed noisy targets (income, claim severity, latency), MAE or Huber; asymmetric costs or interval requirements, quantile; count targets, Poisson or Tweedie deviance, standard for insurance frequency-severity in Indian insurers' pricing teams. Implementation-wise, gradient boosting frameworks accept custom objectives as functions returning per-row gradient and Hessian of the loss with respect to the raw prediction, which is exactly the second-order machinery XGBoost popularised; supply them and the tree-growing logic optimises your business loss directly rather than a proxy. Two cautions: always pair a custom objective with a matching custom evaluation metric for early stopping, and sanity-check gradients numerically, a sign error trains a model that actively optimises the wrong direction while appearing to converge.

import numpy as np
import lightgbm as lgb

def asymmetric_l2(underestimate_penalty=3.0):
    """Custom objective: squared error, 3x heavier when we UNDER-predict.
    (e.g. ETA: promising 25 min and taking 40 is worse than the reverse)"""
    def objective(y_true, y_pred):
        residual = y_pred - y_true
        weight = np.where(residual < 0, underestimate_penalty, 1.0)
        grad = 2.0 * weight * residual          # dL/dpred
        hess = 2.0 * weight                     # d2L/dpred2
        return grad, hess
    return objective

def asymmetric_l2_eval(y_true, y_pred):
    residual = y_pred - y_true
    weight = np.where(residual < 0, 3.0, 1.0)
    return "asym_l2", float(np.mean(weight * residual ** 2)), False

model = lgb.LGBMRegressor(objective=asymmetric_l2(3.0),
                          n_estimators=2000, learning_rate=0.05)
model.fit(X_train, y_train,
          eval_set=[(X_val, y_val)], eval_metric=asymmetric_l2_eval,
          callbacks=[lgb.early_stopping(100)])
# Alternative for intervals: objective="quantile", alpha=0.8

Answer

A production recommender at Flipkart or Swiggy scale is a funnel: retrieval selects a few hundred candidates from a catalogue of millions in milliseconds, then a ranking model scores that small set with rich features. The two-tower architecture is the standard retrieval workhorse. One tower encodes the user (profile features, behaviour-sequence embeddings, context like hour and city) into a d-dimensional vector; a second tower independently encodes each item (category, price band, text and image embeddings, popularity statistics); relevance is the dot product or cosine of the two vectors.

Training is contrastive: positive user-item pairs from engagement logs (clicks, orders), against negatives, and the negative-sampling strategy is where the craft lives. Random negatives are too easy; in-batch negatives, treating other users' positives in the same batch as this user's negatives, are the efficient standard, corrected for popularity bias (popular items appear as negatives too often, and the logQ correction compensates); hard negatives (items the user saw and skipped) sharpen the boundary but overdone they teach the model to fight noise. The architectural constraint that makes it all work: the towers never interact until the final dot product, so item vectors depend on nothing about the user, letting you precompute embeddings for the whole catalogue offline and reduce serving to 'embed the user, find the nearest item vectors'.

That nearest-neighbour search over millions of vectors cannot be exact at latency; approximate nearest neighbour indexes solve it, HNSW graphs (high recall, memory-hungry, the default in most vector databases) or IVF-PQ (coarse clustering plus product quantisation, compressing vectors for RAM-constrained scale), trading a few points of recall for orders-of-magnitude speedups. Operational realities worth naming: the item index is rebuilt or incrementally updated as catalogue and embeddings refresh, and user vectors are computed at request time (or near-line for heavy sequence encoders); cold-start items lean on content features in the item tower, which is precisely why the tower takes attributes and not just an ID embedding; and retrieval is evaluated with recall@k against the ranker's eventual choices, not AUC, because its only job is to not drop the good candidates.

Answer

Escalate through cheap options first; the interviewer is testing judgment about when distributed infrastructure is actually warranted. Step one, shrink the data honestly: most 100 GB training sets are 100 GB of waste. Downcast dtypes (float64 to float32, int64 to int32 or int8, objects to categoricals) for a typical 4-8x reduction; load only needed columns from columnar formats (Parquet with column projection, never CSV); and ask whether all rows earn their place, for imbalanced problems, keeping every positive and downsampling negatives with importance weights preserves the signal at a fraction of the size, and learning curves tell you when more rows stopped helping.

Step two, stream instead of load: LightGBM and XGBoost construct compressed histogram representations and train comfortably on datasets larger than naive pandas handling suggests, XGBoost's external-memory mode and QuantileDMatrix exist precisely for this; scikit-learn's SGDClassifier and other partial_fit estimators do true out-of-core learning over chunked reads; and PyTorch DataLoaders stream batches from disk by design, memory-mapping large arrays. Step three, swap the dataframe engine: Polars with lazy scanning executes query plans over larger-than-RAM Parquet with spilling, and DuckDB runs SQL aggregations directly against Parquet files at warehouse speed on a laptop, both routinely replacing 'we need a cluster' for feature engineering at the tens-to-hundreds-of-GB scale. Push heavy joins and aggregations upstream into the warehouse (BigQuery, Snowflake, Redshift) and pull down only the model-ready matrix.

Step four, only now, distribute: Spark or Ray, Dask for pandas-shaped workflows, distributed LightGBM/XGBoost when a single beefy machine (which cloud vendors happily rent with 700+ GB of RAM) is genuinely exceeded, or when the pipeline must run inside an existing Spark platform anyway. The closing point that reads as experience: distributed training buys real costs, cluster tuning, shuffle failures, subtle nondeterminism, harder debugging, so the honest sequence is sample, stream, single big machine, and distribute last, with learning curves justifying each step up.

import pandas as pd

# 1) Shrink: dtypes + column projection (Parquet, never CSV)
cols = ["user_id", "txn_amount", "city_tier", "label"]
df = pd.read_parquet("txns.parquet", columns=cols)
df["txn_amount"] = df["txn_amount"].astype("float32")
df["city_tier"] = df["city_tier"].astype("category")
print(df.memory_usage(deep=True).sum() / 1e9, "GB after downcast")

# 2) True out-of-core: stream Parquet row batches, never load the full file
import pyarrow.parquet as pq
from sklearn.linear_model import SGDClassifier
clf = SGDClassifier(loss="log_loss")
pf = pq.ParquetFile("txns.parquet")
for batch in pf.iter_batches(batch_size=1_000_000,
                             columns=["txn_amount", "label"]):
    chunk = batch.to_pandas()   # only this batch is ever in memory
    clf.partial_fit(chunk[["txn_amount"]], chunk["label"], classes=[0, 1])

# 3) Larger-than-RAM aggregation without a cluster: DuckDB over Parquet
import duckdb
feat = duckdb.sql("""
    SELECT user_id, COUNT(*) AS txn_cnt, SUM(txn_amount) AS spend
    FROM 'txns.parquet' GROUP BY user_id
""").df()

Answer

Both attack the same production problem, inference cost and latency, from different directions, and they compose. Knowledge distillation trains a small student model to imitate a large teacher: instead of (or alongside) the hard labels, the student learns from the teacher's full output distribution, softened by a temperature parameter in the softmax. The soft targets carry 'dark knowledge', how confidently the teacher ranks the wrong classes, which regularises the student and transfers inter-class structure a one-hot label cannot; the loss is a weighted blend of KL divergence to the teacher's softened outputs and cross-entropy on true labels, and variants also match intermediate representations.

Distillation changes the architecture itself, so the wins are structural: a 6-layer student of a 12-layer teacher (the DistilBERT recipe) keeps most of the quality at roughly half the size and materially faster inference, and distilling a large teacher's scores into a small ranker is standard practice in recommendation stacks. It costs a real training effort and needs data to distil on. Quantization keeps the architecture and shrinks the numbers: weights (and optionally activations) drop from 32- or 16-bit floats to INT8 or 4-bit representations, cutting memory footprint proportionally and exploiting faster integer kernels.

Post-training quantization is nearly free, calibrate scales on a few hundred batches and convert, and INT8 typically costs little accuracy on well-behaved networks; quantization-aware training simulates quantization during fine-tuning and recovers accuracy when PTQ degrades, which matters more at 4-bit. For LLM serving in 2026, weight-only 4-bit and 8-bit quantization is the norm because memory bandwidth dominates decode latency, and it is often the difference between a model fitting one GPU or needing two. Choosing: quantization first, always, because it is cheap and composable; distillation when you need a structurally smaller or architecturally different model (edge deployment, strict latency SLOs, expensive teacher ensembles compressed into one servable); both together for the tightest budgets, distil then quantise.

Also say what does not need any of this: gradient-boosted trees, whose serving cost is usually trivial, this toolkit is about neural networks. Verify on your own eval set after compression, per-segment, because compression losses are rarely uniform and can concentrate in minority slices.