LAB08_GCP_Cron Jobs: Cloud Scheduler Triggering Pub/Sub

Cron Jobs: Cloud Scheduler Triggering Pub/Sub

☁ GCP 🟢 Beginner Cloud Scheduler Pub/Sub Cron

Scenario

You have a batch process that needs to run every night at 2:00 AM to aggregate analytics. Instead of keeping a VM running just to execute a cron tab, you will use Cloud Scheduler, a fully managed enterprise-grade cron job scheduler. The scheduler will publish an event payload to a Pub/Sub topic, which could then trigger a Cloud Function or Cloud Run job to do the actual aggregation.

Your Objectives
  • Create a google_pubsub_topic named analytics-trigger.
  • Create a google_cloud_scheduler_job that runs at 0 2 * * * (2:00 AM daily).
  • Set the scheduler target to pubsub_target pointing to the topic, and encode a simple JSON payload in the message body.
  • Output the Job ID.

Additional Context

Targets: Cloud Scheduler can hit HTTP/HTTPS endpoints, Pub/Sub topics, or App Engine HTTP endpoints. Pub/Sub is highly recommended for internal triggers because it decouples the timing from the execution: the scheduler doesn't need to worry about timeouts if your background job takes 2 hours to execute.

Data Encoding: When sending a payload to Pub/Sub via Cloud Scheduler in Terraform, the `data` field must be a base64-encoded string. You can use Terraform's base64encode() function to wrap a JSON template.

variables.tf

variable "project_id" { type = string }
variable "gcp_region" { type = string; default = "us-central1" }

main.tf

provider "google" {
  project = var.project_id
  region  = var.gcp_region
}

# 1. The Pub/Sub Topic to trigger
resource "google_pubsub_topic" "trigger" {
  name = "analytics-trigger"
}

# 2. The Cloud Scheduler Job
resource "google_cloud_scheduler_job" "nightly_analytics" {
  name        = "nightly-analytics-aggregation"
  description = "Triggers analytics aggregation at 2 AM every day"
  
  # Standard unix cron format: minute hour day-of-month month day-of-week
  schedule    = "0 2 * * *"
  
  # Define the Timezone to ensure 2AM means 2AM locally
  time_zone   = "America/New_York"

  pubsub_target {
    topic_name = google_pubsub_topic.trigger.id

    # Base64 encoded payload that gets sent to the Pub/Sub subscriber
    data = base64encode(jsonencode({
      "action": "aggregate",
      "target_table": "daily_stats"
    }))
  }
}

outputs.tf

output "scheduler_job_id" {
  value = google_cloud_scheduler_job.nightly_analytics.id
}

Workflow Commands

terraform init
terraform apply -var="project_id=my-gcp-project-123"

# Manually trigger the job for testing right now (don't wait for 2 AM)
gcloud scheduler jobs run nightly-analytics-aggregation \
  --location=us-central1

# Or view the job status
gcloud scheduler jobs describe nightly-analytics-aggregation \
  --location=us-central1

terraform destroy -var="project_id=my-gcp-project-123"
✓ Expected Output After Apply
Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Outputs:

scheduler_job_id = "projects/my-gcp-project/locations/us-central1/jobs/nightly-analytics-aggregation"

Related Posts