Serve a model on Kubernetes with autoscaling and a safe rollout

#Brief

A classifier in a notebook is not a product. Take the Banking77 intent model (TF-IDF or embeddings) and run it the way a platform team would: in containers, on Kubernetes, observable and safe to update.

Use a local cluster such as kind or minikube. Nothing here needs a cloud account.

#What to build

1. Train the model and save it with its version and training data hash.

2. Serve it with FastAPI: /predict, /healthz and /readyz (ready only after the model is loaded), plus Prometheus metrics.

3. Build a multi-stage image that runs as a non-root user. Record its size.

4. Deploy with resource requests and limits set from a quick load test, liveness and readiness probes, and an HPA on CPU.

5. Load test with a tool such as k6 or Locust. Show pods scaling out and latency staying within a target you set.

6. Ship version 2 with a rolling update while the load test runs. Show zero failed requests, then roll back.

7. Validate manifests in CI (for example with kubeconform) and build the image on every push.

#Deliverables

• Repository with service/, Dockerfile, k8s/, load/, .github/workflows/, RUNBOOK.md

#How it is graded

Reviewers create the cluster from your README, run your load test and trigger a rollout. They check the dashboard and whether anything failed.

#Stretch

Add a canary with two Deployments behind one Service and compare the new version's error rate before shifting all traffic.