Y

Machine learning · Beginner

Your first classifier, done properly

3 min read

Why it matters

Machine learning appeared in 84% of the Data Scientist and 100% of the AI/ML Engineer job descriptions we read. Before any fancy model, interviewers check the workflow: what is the target, how did you split the data, what baseline did you beat? Get this right once and every later model is built on solid ground.

The concept

Supervised learning learns a mapping from features (inputs, X) to a label (target, y) using examples where the answer is known.

The workflow:

  1. Define the target and the decision the model supports.

  2. Split data into training and test sets before looking closely. The test set simulates new data and is used once, at the end.

  3. Baseline: the score of a trivial rule, such as always predicting the most common class. A model that cannot beat it is useless.

  4. Pipeline: preprocessing plus model in one object, so the exact same steps run in training and prediction.

  5. Fit on training data, evaluate on test data.

scikit-learn gives every model the same interface: fit, predict, predict_proba, score.

Worked example: penguin species

The Palmer Penguins data (CC0) has 344 penguins; 333 have every measurement.

penguins_classifier.py
import pandas as pd
from sklearn.dummy import DummyClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, confusion_matrix
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler

url = "https://raw.githubusercontent.com/allisonhorst/palmerpenguins/main/inst/extdata/penguins.csv"
penguins = pd.read_csv(url).dropna()
X = penguins[["bill_length_mm", "bill_depth_mm", "flipper_length_mm", "body_mass_g"]]
y = penguins["species"]

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42, stratify=y)

baseline = DummyClassifier(strategy="most_frequent").fit(X_train, y_train)
print("baseline accuracy", accuracy_score(y_test, baseline.predict(X_test)))

model = Pipeline([
    ("scale", StandardScaler()),
    ("classify", LogisticRegression(max_iter=1000)),
]).fit(X_train, y_train)
pred = model.predict(X_test)
print("model accuracy", accuracy_score(y_test, pred))
print(confusion_matrix(y_test, pred, labels=["Adelie", "Chinstrap", "Gentoo"]))

Output: the baseline (always "Adelie") scores 0.44 on the 84 test penguins; logistic regression scores 1.00, with every species correct in the confusion matrix.

A perfect score should make you suspicious, not happy. Here it is plausible: bill shape and flipper length separate these three species very cleanly, and biologists can do the same by eye. On real business problems, a perfect score usually means leakage (see the feature pipelines shelf).

Note stratify=y: it keeps the species mix the same in both splits, which matters for small or imbalanced data.

Common mistakes

  • No baseline. "92% accuracy" means nothing if 90% of rows are one class.

  • Preprocessing before splitting. Fitting a scaler on all data leaks test information into training. Put preprocessing inside the pipeline.

  • Tuning on the test set. Every time you look at test results and change something, the test set becomes training data. Use validation or cross-validation for choices.

  • Forgetting the decision. A model is only useful if its predictions change something someone does.

Interview angle

"Walk me through how you would build a model for X." Answer in workflow order: target and decision, data and split, baseline, simple model first, metric chosen for the decision, then iteration. Starting with "I would use XGBoost" is a red flag for interviewers.

Practice task

  1. Replace logistic regression with KNeighborsClassifier and DecisionTreeClassifier. Compare test accuracy.

  2. Use only body_mass_g. How far does accuracy fall, and which species get confused?

  3. Predict sex instead of species. What is the baseline, and how much does the model beat it?

Further reading