Let’s be honest. Most “AI projects” start as a notebook named Untitled12_final_final_v2.ipynb. It lives on a laptop that hasn’t rebooted in three weeks, runs on a Python version that no longer exists, and relies on a CSV file Bob from accounting emailed you once.
Then management says, “Great, let’s put it in production.” And that is when the screaming starts.
To survive the transition from “hacky script” to “enterprise system” without losing your sanity, you must stop treating ML like a science fair project and treat it like software engineering. Vertex AI is the toolkit for this. Let’s break down the parts that actually matter.
Vertex AI Experiments: The “Messy Desk”
Before you have a model, you have a mess. You are tweaking hyperparameters, swapping datasets, and trying eight architectures. If you do not track this, you will forget which combination yielded 98% accuracy.
Vertex AI Experiments is a logbook for the chaos. It tracks:
– Parameters: Learning rate, batch size, dropout
– Metrics: Accuracy, loss, F1 score
– Context: Which dataset version you used
Think of it as Git commit messages for math. Later, when your boss asks why the new model is “worse,” you pull up the Dashboard and prove that actually, this model is 10x faster and 80% cheaper, even if it is 0.1% less accurate.
Model Registry: The “Filing Cabinet”
Once you have a model that isn’t terrible, put it somewhere safe. That is NOT an S3 bucket or a desktop folder. The Model Registry is the source of truth for “which version is running in prod?”
- Versioning: It handles v1, v2, v3 automatically.
- Aliasing: Tag models as
default,staging, orproduction. - Governance: See exactly who trained it and when.
The Registry does not store the massive weight files (those live in Cloud Storage); it stores the metadata and pointers. It is the library card catalog.
The Workflow:
1. Experiments: Try 50 things.
2. Winner: Pick the best one.
3. Registry: Register the winner. It is now an immutable artifact.
Model Garden: The “Shopping Mall”
Sometimes you do not need to build a car; you just need to rent a taxi. Model Garden is Google’s catalog of pre-trained models.
- First-Party: Gemini, PaLM, Imagen, Chirp.
- Open Source: Llama, BERT, Mistral.
The cynical take: It is great for prototyping or generic tasks. But if you have a highly specific domain (detecting defects in underwater welding seams), you still need fine-tuning. Treat Model Garden as a starting point, not a magic wand.
Endpoints: Where Rubber Meets Road
An “Endpoint” is a URL that accepts data and spits out predictions. In the cloud, nothing is “just” a URL.
Public Endpoints:
Standard internet IP. Secured by IAM (needs a Google token).
Best for: Mobile apps, public web services.
Private Endpoints:
Private Service Connect (PSC) exposes the model as an internal service inside your VPC. No public internet exposure.
Best for: Internal enterprise apps, sensitive financial data.
The superpower: Traffic Splitting (Canary Deployments).
You can deploy two models to the same endpoint ID:
“Send 90% of traffic to Model v1, and 10% to Model v2.”
If v2 throws errors, flip the switch back to 100% v1. Zero downtime. If you aren’t doing this, you are deploying on hope.
Pipelines: Modularizing at 3 AM
If you run training by manually clicking “Run” in a notebook, you are doing it wrong. Vertex AI Pipelines (based on Kubeflow) break your massive script into small, reusable Components:
- Ingestion Component: Pulls data from BigQuery
- Preprocessing Component: Cleans data
- Training Component: Crunches numbers
- Evaluation Component: Checks if the model sucks
- Deployment Component: Pushes to Registry if it passes
If Step 3 fails, you fix Step 3. You do not re-run the whole universe. It saves money (caching) and sanity.
The Bottom Line
Vertex AI is complex because production ML is complex.
– Use Experiments to make your mess searchable.
– Use Registry to lock down winners.
– Use Endpoints with traffic splitting so you don’t sweat bullets deploying.
– Use Pipelines so you don’t manually babysit scripts at 3 AM.
