Project · Beginner · about 8 hours
SMS spam classifier with an honest baseline
Build and compare text classifiers on 5,572 real SMS messages, and explain why the simple one wins.
Skills:LLMs and NLP, Machine learning, Python
Part of: Data Scientist, AI/ML Engineer
Data and licences
- UCI SMS Spam Collection ↗CC BY 4.0
Brief
A mobile operator wants to filter spam SMS. Blocking a real message (a bank OTP, a family message) is worse than letting some spam through. Build and compare classifiers, then recommend one with a threshold.
What to build
Load the tab-separated file: 4,825 ham and 747 spam messages.
Build a TF-IDF plus logistic regression baseline inside a pipeline.
Build an embedding classifier with an open sentence embedding model.
Compare on the same test set. Choose a threshold that keeps false positives below 1 in 1,000 legitimate messages, and report the spam recall you get.
Read 20 errors and group them by cause.
Deliverables
spam.ipynb,errors.md,recommendation.md
How it is graded
Reviewers rerun the notebook and check the threshold logic against the stated cost of false positives. A clear, cost-aware recommendation scores highest.
Stretch
Add a zero-shot LLM classifier on 200 test messages and compare quality, latency and cost per 1,000 messages.
Deliverables
- A notebook comparing TF-IDF and embedding classifiers on one fixed test set
- An error analysis of 20 misclassified messages
- A short recommendation for a phone operator
How a reviewer grades it
| Criterion | Weight | What good looks like |
|---|---|---|
| Fixed split, suitable metrics (precision, recall, F1 for spam) | 30% | |
| Two approaches compared fairly | 25% | |
| Error analysis finds real patterns | 25% | |
| Recommendation weighs false positives against missed spam | 20% |