LAB07_GCP_Asynchronous Messaging: Pub/Sub Topics and Subscriptions

Asynchronous Messaging: Pub/Sub Topics and Subscriptions

☁ GCP 🟢 Beginner Pub/Sub Messaging Decoupling

Scenario

In event-driven architectures, services communicate indirectly. When a user creates an order, the "Orders Service" publishes an event, and the "Invoicing Service" listens for it. In Google Cloud, this is handled by Cloud Pub/Sub. You need to create a Pub/Sub topic for order events, and a pull subscription that retains unacknowledged messages for 7 days.

Your Objectives
  • Create a google_pubsub_topic named order-events.
  • Create a google_pubsub_subscription named invoice-processor attached to the topic.
  • Set the subscription's message retention duration to 7 days (the maximum allowed).
  • Set the acknowledgement deadline to 60 seconds (giving the subscriber a minute to process a message before it is re-delivered).
  • Output the topic name and subscription name.

Additional Context

Pull vs Push: A Pull subscription requires the client application to constantly ask Pub/Sub "Do you have any new messages?" (polling). A Push subscription tells Pub/Sub to actively send HTTP POST requests to an endpoint (like a Cloud Function) whenever a message arrives. In this lab, we are creating the default Pull subscription.

Retention Duration: Unacknowledged messages stay in the subscription until this duration expires. By default, it's 7 days. Once a message is acknowledged, it is removed from the queue immediately unless "retain acknowledged messages" is explicitly turned on.

variables.tf

variable "project_id" { type = string }

main.tf

provider "google" {
  project = var.project_id
}

# The Pub/Sub Topic
resource "google_pubsub_topic" "orders" {
  name = "order-events"

  labels = {
    domain = "commerce"
  }
}

# The Pub/Sub Pull Subscription
resource "google_pubsub_subscription" "invoice_worker" {
  name  = "invoice-processor-sub"
  topic = google_pubsub_topic.orders.id

  # Duration format requires an "s" for seconds. 7 days = 604800s
  message_retention_duration = "604800s" 
  
  # Give the worker 60 seconds to process the message before re-delivery
  ack_deadline_seconds = 60

  # Retain messages that have been acknowledged (good for replay ability, costs more config)
  retain_acked_messages = false

  expiration_policy {
    # If the subscription is inactive (no connections/pulls) for 30 days, delete it automatically
    ttl = "2592000s"
  }

  labels = {
    worker = "invoicing"
  }
}

outputs.tf

output "topic_name" {
  value = google_pubsub_topic.orders.name
}

output "subscription_name" {
  value = google_pubsub_subscription.invoice_worker.name
}

Workflow Commands

terraform init
terraform apply -var="project_id=my-gcp-project-123"

# Publish a message to the topic via gcloud
gcloud pubsub topics publish $(terraform output -raw topic_name) \
  --message="{'order_id': 991, 'amount': 45.00}"

# Pull the message from the subscription (auto-acknowledged)
gcloud pubsub subscriptions pull $(terraform output -raw subscription_name) \
  --auto-ack

terraform destroy -var="project_id=my-gcp-project-123"
✓ Expected Output After Apply
Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Outputs:

subscription_name = "invoice-processor-sub"
topic_name        = "order-events"

Related Posts