LAB02_GCP_Cloud Storage: Versioned Buckets and Lifecycle Rules

Cloud Storage: Versioned Buckets and Lifecycle Rules

☁ GCP 🟢 Beginner Cloud Storage Versioning Lifecycle

Scenario

Your data science team needs a place to store training datasets. They require a bucket that prevents accidental deletion of files (by keeping previous versions). To manage costs, any non-current version of a file should automatically transition to Nearline storage after 30 days, and be permanently deleted after 90 days.

Your Objectives
  • Create a google_storage_bucket with a globally unique name using a random_id.
  • Enable uniform bucket-level access and enforce public access prevention.
  • Enable object versioning on the bucket.
  • Add a lifecycle_rule that sets the storage class to NEARLINE for non-current versions older than 30 days.
  • Add a second lifecycle_rule that deletes non-current versions older than 90 days.
  • Output the bucket URL and name.

Additional Context

Uniform Bucket-Level Access enforces that only IAM policies grant access to objects. It disables legacy Object ACLs, which is highly recommended by Google for security and auditing.

force_destroy: If a bucket contains objects (even deleted versions), Terraform cannot destroy it unless you set force_destroy = true, which tells the underlying API to delete all contents before deleting the bucket itself.

variables.tf

variable "project_id" { type = string }
variable "gcp_region" { type = string; default = "us-central1" }

main.tf

provider "google" {
  project = var.project_id
  region  = var.gcp_region
}

# Generate a random suffix for the bucket name
resource "random_id" "bucket_suffix" {
  byte_length = 4
}

# The Cloud Storage Bucket
resource "google_storage_bucket" "datasets" {
  name          = "ml-datasets-${random_id.bucket_suffix.hex}"
  location      = var.gcp_region
  storage_class = "STANDARD"

  # Allows terraform destroy to succeed even if the bucket has objects
  force_destroy = true

  # Google's recommended security setting
  uniform_bucket_level_access = true
  public_access_prevention    = "enforced"

  # Enable versioning to protect against accidental overwrites/deletes
  versioning {
    enabled = true
  }

  # Lifecycle Rule 1: Move non-current versions to Nearline after 30 days
  lifecycle_rule {
    action {
      type          = "SetStorageClass"
      storage_class = "NEARLINE"
    }
    condition {
      age = 30
      with_state = "ARCHIVED" # ARCHIVED means non-current version
    }
  }

  # Lifecycle Rule 2: Delete non-current versions after 90 days
  lifecycle_rule {
    action {
      type = "Delete"
    }
    condition {
      age = 90
      with_state = "ARCHIVED"
    }
  }

  labels = {
    environment = "dev"
    team        = "data-science"
  }
}

outputs.tf

output "bucket_name" {
  value = google_storage_bucket.datasets.name
}

output "bucket_url" {
  value = google_storage_bucket.datasets.url
}

Workflow Commands

terraform init
terraform apply -var="project_id=my-gcp-project-123"

# Upload an empty file to test
touch test.csv
gcloud storage cp test.csv gs://$(terraform output -raw bucket_name)/

# Upload it again to create a new version
gcloud storage cp test.csv gs://$(terraform output -raw bucket_name)/

# List all versions
gcloud storage ls --all-versions gs://$(terraform output -raw bucket_name)/

terraform destroy -var="project_id=my-gcp-project-123"
✓ Expected Output After Apply
Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Outputs:

bucket_name = "ml-datasets-a1b2c3d4"
bucket_url  = "gs://ml-datasets-a1b2c3d4"

Related Posts