Route banking questions to 77 intents

#Brief

A bank's support team wants incoming chat messages routed to the right queue. Banking77 has 10,003 training and 3,080 test queries, each labelled with one of 77 intents such as card_arrival, lost_or_stolen_card and exchange_rate. Many intents are close neighbours, which makes this a realistic routing problem.

As a reference point, a TF-IDF (word 1 and 2-grams) plus logistic regression baseline scores about 89% accuracy on the test split, and logistic regression on all-MiniLM-L6-v2 sentence embeddings scores about 93%. Your job is to reproduce a baseline, improve on it honestly and explain what is still wrong.

#What to build

1. Load train.csv and test.csv. Hold out 10% of training data as validation for any tuning.

2. Train the TF-IDF baseline. Report accuracy and macro F1.

3. Train a classifier on sentence embeddings from a small open model. Report the same metrics, the time to embed and the model size.

4. Build a confusion table and read the ten most confused pairs. Are the labels ambiguous, or is the model wrong?

5. Pick an operating rule: when the top score is low, send the message to a human. Show how accuracy on the routed messages changes with the share sent to humans.

#Deliverables

• Repository with notebooks/, src/, a pinned requirements.txt and README.md

#How it is graded

Reviewers rerun your notebook and read the error analysis. The key question: would a support lead trust your routing rule, and do they understand where it fails?

#Stretch

Add a zero-shot or few-shot LLM classifier on 300 test queries and compare accuracy, latency and cost per 1,000 messages with your trained models.