AI / ML · Cloud-Native
From model training to a live, auto-scaling API — automatically
A pipeline that trains several versions of a model, picks the best one and puts it live as a service that grows and shrinks with demand. No manual hand-offs between data science and operations.
Proof of concept: Kubeflow Pipelines + KServe: end-to-end MLOps on Kubernetes
Where this helps
- Teams that train models but still deploy them by hand
- Making machine-learning results reproducible and auditable
- Serving models whose traffic goes up and down
The rest of this page is the technical deep dive — how it was built and what its limits are.
The technical problem
Training, comparing and deploying models by hand is slow and hard to reproduce. Teams need a repeatable path from training run to a scalable inference endpoint.
How the architecture works
- 01 · Train ×3
Parallel training components, one per learning rate (0.001, 0.0005, 0.0001).
- 02 · Select
A selection component compares validation accuracy and picks the best model.
- 03 · Store
The chosen model is stored in MinIO (S3-compatible object storage).
- 04 · Serve
A deployment component creates a KServe InferenceService that scales from 1 to 3 pods on request concurrency.
What was being explored
Machine learning work breaks the assumptions of ordinary CI/CD: a model is the combination of code, data, hyperparameters and runtime environment. The goal was to show that a complete lifecycle — parallel training, comparison, selection and serving — can run as one reproducible workflow on Kubernetes.
Why this approach
Kubeflow Pipelines adds ML-aware orchestration on top of Kubernetes, and KServe turns a stored model into a standard REST inference endpoint with request-based autoscaling through Knative. Using both keeps training and serving on the same platform and under the same access controls.
What was implemented
Pipeline components for training, model selection and KServe deployment; a training container image; KServe RBAC and storage-secret manifests; a compiled pipeline manifest; and an inference test script. The pipeline is parameterised (epochs, hidden units) and runs from the Kubeflow UI.
What it demonstrates
The handover from “best experiment” to “running service” can be automated inside the pipeline itself, removing a manual and error-prone step. The repository documents the RBAC and storage wiring that this handover needs, which is usually where such setups fail.
Limitations and next steps
- Uses the Fashion-MNIST sample dataset; it demonstrates the workflow, not model quality.
- Hyperparameter search is a fixed grid of learning rates rather than a tuning service.
- Demo storage credentials are provided as plain Kubernetes manifests and should be replaced by a secret manager in real environments.
Related writing
From Reproducible Pipelines to Auto-Scaled Inference: ML on Kubernetes with Kubeflow and KServe
Machine learning needs its own operational layer. The article walks through orchestrating parallel training, model selection and auto-scaled serving on Kubernetes with Kubeflow Pipelines and KServe.