Article 18 – Load Balancing in Google Cloud

So your application is getting serious traffic. You started with a single VM, and it worked fine for a while. But now it is struggling under the load. Worse, if that one VM goes down, your entire application is offline. Single point of failure. Performance bottleneck. Classic problem.

The fix is obvious — run multiple copies of your application behind a Managed Instance Group (MIG). But now you have a dozen VMs, each with its own IP address. Which IP do you give to your users? How do you distribute traffic evenly? How do you stop sending traffic to a VM that just crashed?

You need a Load Balancer.

What a Load Balancer Actually Does

A load balancer sits in front of your servers and acts as a single entry point for all incoming traffic. Its job is straightforward:

  • Provide one stable IP address for your users
  • Distribute incoming requests across your pool of backend servers
  • Monitor health continuously and only send traffic to healthy instances

Now, in Google Cloud, “Load Balancer” is not a single product. It is an entire suite of different products, each designed for a specific type of traffic and use case. Picking the right one matters.

The Decision Tree

Choosing the right load balancer comes down to three questions:

  1. Global or Regional? — Do you need to serve users worldwide from a single IP, or are your servers in one region?
  2. External or Internal? — Is traffic coming from the public internet, or flowing between services inside your VPC?
  3. HTTP(S) or TCP/UDP? — Are you balancing web traffic (Layer 7) where you can route based on URLs, or raw TCP/UDP traffic (Layer 4)?

Let me walk through each type.

The Global External HTTP(S) Load Balancer

This is the flagship. If you are running a website or web application with users around the world, this is what you want.

The scenario: You have an e-commerce site. Users are in New York, London, and Tokyo. You want a single IP address (www.yoursite.com) but you want traffic served by the VMs closest to each user.

Here is what makes this load balancer special:

  • Single Anycast IP — You get one public IP address that is announced from all of Google’s edge locations globally. When a user in London hits that IP, their traffic enters Google’s network in Europe automatically.

  • Global Backend Support — It can balance traffic across instance groups in multiple regions (us-east1, europe-west2, asia-northeast1). It routes users to the closest healthy backend with available capacity.

  • SSL Offloading — Instead of making each VM handle encryption/decryption, the load balancer does it at the edge. This frees up CPU on your backends. You can even use free, Google-managed SSL certificates that auto-renew.

  • Cloud CDN Integration — One checkbox enables Google’s Content Delivery Network to cache static content at the edge.

If the question involves global reach, HTTP/HTTPS, or a single IP for multiple regions — this is almost always the answer.

External Network Load Balancer (TCP/UDP)

The scenario: You are not balancing a website. You are running a backend for an online game that uses UDP on port 7777. All your game servers are in us-central1.

HTTP(S) is irrelevant here. You need a Layer 4 load balancer that handles raw TCP or UDP traffic.

  • Regional Scope — It distributes traffic to backends within a single region
  • Layer 4 — It understands TCP and UDP but has no knowledge of HTTP headers or URLs. It just forwards packets.
  • Pass-Through — It does not proxy connections, which means extremely high throughput and low latency. Perfect for real-time applications.

If traffic is external, regional, and NOT HTTP(S) — like SSH, RDP, SMTP, or custom protocols — this is your tool.

Internal TCP/UDP Load Balancer

The scenario: You have a three-tier application inside your VPC. Web servers (tier 1) need to talk to application servers (tier 2). You want to load balance that traffic, but these app servers must NOT be exposed to the internet.

  • Internal IP Address — The load balancer gets a private IP from your subnet. Only reachable within your VPC or peered VPCs.
  • Layer 4 — Balances TCP/UDP for internal service-to-service communication.
  • Managed and HA — Google handles the availability.

If traffic is service-to-service inside your VPC and TCP/UDP based — this is the answer.

Internal HTTP(S) Load Balancer

The scenario: Your internal architecture is microservices communicating over REST APIs. You want to route /users to the user-service and /inventory to the inventory-service through a single internal entry point.

The Internal TCP/UDP LB cannot do this — it does not understand URLs. You need the Internal HTTP(S) Load Balancer.

  • Internal Layer 7 — Advanced URL-based routing inside your VPC
  • Proxy-Based — Handles complex routing and policy enforcement between microservices

If internal microservices need HTTP/HTTPS-based routing — this is it.

The Anatomy of a Load Balancer

Behind the scenes, a load balancer is not a single box. It is a chain of configured resources:

  1. Forwarding Rule — The front door. Defines the IP, protocol, and port the LB listens on.
  2. Target Proxy / Target Pool — Terminates user connections.
  3. URL Map (Layer 7 only) — Routes requests based on path or host to specific backend services.
  4. Backend Service — Defines how traffic is distributed and points to the health check.
  5. Backend (Instance Group or NEG) — The actual group of VMs or containers receiving traffic.
  6. Health Check — The LB constantly probes backends (e.g., hitting /healthz and looking for a 200 OK). If a backend fails, traffic stops going to it until it recovers.
Setting Up a Global HTTP(S) Load Balancer

Let me walk you through the gcloud commands to create one:

# Step 1: Create a Managed Instance Group
gcloud compute instance-groups managed create web-mig \
    --template=web-template \
    --size=3 \
    --zone=us-central1-a

# Step 2: Create a health check
gcloud compute health-checks create http web-health-check \
    --port=80

# Step 3: Create a backend service
gcloud compute backend-services create web-backend \
    --protocol=HTTP \
    --health-checks=web-health-check \
    --global

# Step 4: Add the MIG to the backend service
gcloud compute backend-services add-backend web-backend \
    --instance-group=web-mig \
    --instance-group-zone=us-central1-a \
    --global

# Step 5: Create a URL map
gcloud compute url-maps create web-map \
    --default-service=web-backend

# Step 6: Create a target HTTP proxy
gcloud compute target-http-proxies create web-proxy \
    --url-map=web-map

# Step 7: Create the forwarding rule (the actual listener)
gcloud compute forwarding-rules create web-rule \
    --address=my-static-ip \
    --global \
    --target-http-proxy=web-proxy \
    --ports=80

That is a lot of steps, but each piece has a purpose. The forwarding rule is the front door, the URL map is the router, the backend service decides where to send traffic, and the health check keeps things reliable.

Common Pitfalls and Best Practices

Pitfall: Choosing a Regional LB when your application has a global user base. Users far away will experience high latency.
Best Practice: Use the Global External HTTP(S) LB for any web app with geographically distributed users.

Pitfall: Exposing internal backend services to the internet with an external LB.
Best Practice: Use internal load balancers for all service-to-service communication within your VPC.

Pitfall: Health checks that are too aggressive or do not accurately reflect application health. Healthy backends get removed from rotation.
Best Practice: Create a dedicated health check endpoint (/healthz) that returns 200 only when the app is fully functional.

Pitfall: Forgetting firewall rules to allow traffic from the LB to your backends.
Best Practice: Always create a firewall rule allowing ingress from Google’s health check IP ranges: 35.191.0.0/16 and 130.211.0.0/22.

Related Posts