LAB47_GCP_Caching: Memorystore (Redis) and Legacy Serverless Connectors

Caching: Memorystore (Redis) and Legacy Serverless Connectors

☁ GCP 🔥 Intermediate Memorystore Redis Cloud Functions Serverless VPC Access

Scenario

Your serverless API is experiencing high latency querying BigQuery. You want to introduce a caching layer. You will deploy GCP's managed Redis offering: Memorystore. Because Memorystore instances live entirely on internal VPC IP addresses and cannot have public IPs, your serverless Cloud Function requires a network bridge to reach it. This lab teaches the legacy (but still widely used) method: Serverless VPC Access Connectors.

Your Objectives
  • Deploy a VPC Network and Subnetwork.
  • Deploy a google_redis_instance in the VPC.
  • Deploy a google_vpc_access_connector in the same region, attaching it to the network.
  • Deploy a google_cloudfunctions2_function. Inside its vpc_connector block, reference the Connector.
  • Pass the Redis IP address to the Cloud Function as an environment variable natively.

Additional Context

VPC Access Connectors: How do they work? A connector spins up a fleet of hidden `e2-micro` compute instances in the background. Your Cloud Functions traffic routes to these proxy instances, which then forward the traffic into your VPC. They are billed accurately as Compute Engine instances!

Direct Egress vs Connectors: Direct VPC Egress is faster, cheaper, and more scalable. However, Serverless Connectors are still required for certain older GCP architectures (like Cloud Functions Gen 1) and specific Shared VPC topologies, so Terraform engineers must know how to provision them.

main.tf

provider "google" {
  project = var.project_id
  region  = "us-central1"
}

# 1. VPC Network
resource "google_compute_network" "cache_vpc" {
  name                    = "redis-vpc"
  auto_create_subnetworks = false
}

# Subnet for the connector (Must be a dedicated /28 range)
resource "google_compute_subnetwork" "connector_subnet" {
  name          = "connector-sub"
  network       = google_compute_network.cache_vpc.id
  region        = "us-central1"
  cidr_range    = "10.8.0.0/28"
}

# 2. Serverless VPC Access Connector
resource "google_vpc_access_connector" "redis_connector" {
  name          = "cf-to-redis"
  region        = "us-central1"
  subnet {
    name = google_compute_subnetwork.connector_subnet.name
  }
  
  # Scaling settings for the hidden proxy VMs
  machine_type  = "e2-micro"
  min_instances = 2
  max_instances = 3
}

# 3. Memorystore (Redis) Instance
resource "google_redis_instance" "cache" {
  name           = "app-cache"
  memory_size_gb = 1
  tier           = "BASIC" # No High Availability replica
  region         = "us-central1"

  # Attach it to the VPC
  authorized_network = google_compute_network.cache_vpc.id
  connect_mode       = "DIRECT_PEERING"
}

# 4. Dummy Source Code
data "archive_file" "src" {
  type        = "zip"
  output_path = "func.zip"
  source {
    content  = "import redis, os\ndef hello_http(req):\n    r = redis.Redis(host=os.environ.get('REDIS_IP'))\n    return 'connected'"
    filename = "main.py"
  }
}
resource "random_id" "bkt" { byte_length = 4 }
resource "google_storage_bucket" "b" { name = "gcf-src-${random_id.bkt.hex}"; location = "US" }
resource "google_storage_bucket_object" "z" { name = "func.zip"; bucket = google_storage_bucket.b.name; source = data.archive_file.src.output_path }

# 5. The Cloud Function using the Connector
resource "google_cloudfunctions2_function" "api" {
  name     = "caching-api"
  location = "us-central1"
  
  build_config {
    runtime     = "python311"
    entry_point = "hello_http"
    source {
      storage_source { bucket = google_storage_bucket.b.name; object = google_storage_bucket_object.z.name }
    }
  }

  service_config {
    environment_variables = {
      REDIS_IP = google_redis_instance.cache.host
    }
    
    # Enable the Connector Legacy Bridge
    vpc_connector = google_vpc_access_connector.redis_connector.id
    # Route ALL outbound traffic from the function through the connector into the VPC
    vpc_connector_egress_settings = "ALL_TRAFFIC"
  }
}

Workflow Commands

terraform init
terraform apply -var="project_id=my-gcp-project-123"

# Note: Redis creation can take 5-10 minutes.

terraform destroy -var="project_id=my-gcp-project-123"

Related Posts