LAB50_AWS_Capstone: Production Deployment — State Management, Drift Detection, and Lifecycle Control

Capstone: Production Deployment — State Management, Drift Detection, and Lifecycle Control

☁ AWS 🔥 Advanced State Management Drift Detection terraform import prevent_destroy ignore_changes moved blocks Replacing Resources

Scenario

You've been handed ownership of a production AWS account. Somebody manually created an RDS instance and an S3 bucket directly in the console — they are not in any Terraform state. The S3 bucket's versioning is toggled on/off by an automated process each weekend (intentional drift that you want Terraform to ignore). The RDS instance was manually tagged with a BackupOwner tag that changes nightly via a tagging Lambda (also intentional drift). You need to bring these resources under Terraform state control, handle intentional drift, and lock critical resources against accidental destroy.

Your Objectives
  • Use terraform import to bring the existing S3 bucket and RDS instance into state without recreating them.
  • Add lifecycle { ignore_changes = [versioning, tags["BackupOwner"]] } on the appropriate resources to suppress intentional drift from causing spurious plan diffs.
  • Add lifecycle { prevent_destroy = true } on the RDS instance to block accidental terraform destroy.
  • Demonstrate a moved block: the S3 bucket was originally called aws_s3_bucket.legacy in someone's local state; rename it to aws_s3_bucket.primary without destroying and recreating it.
  • Show how to use terraform plan -refresh-only to detect drift between state and real infrastructure, and terraform apply -replace to force replacement of a specific resource.

Additional Context

terraform import updates Terraform state to point to an existing resource, but it does not generate the HCL configuration. You must write the resource block yourself first, then import. After import, run terraform plan — any differences between your HCL and the actual resource will show as changes. Fix the HCL until the plan shows "no changes" before committing.

moved blocks (introduced in Terraform 1.1) allow you to rename or move resources between module paths in the state without destroy/recreate. They are idempotent — once the move is applied, you can (and should) remove the moved block from your code. They are preferable to terraform state mv because the intent is captured in code and applied as part of a normal plan/apply.

main.tf — after import, with lifecycle controls

provider "aws" { region = "us-east-1" }

# Step 1: Write the resource block matching the existing bucket, then import.
# terraform import aws_s3_bucket.primary my-existing-bucket-name
resource "aws_s3_bucket" "primary" {
  bucket = "my-existing-prod-bucket"

  lifecycle {
    # The bucket's versioning is toggled by an automated weekend process.
    # Tell Terraform to ignore differences in versioning state.
    ignore_changes = [versioning]

    # Prevent any plan from proposing to destroy this bucket.
    prevent_destroy = true
  }
}

# Separate resources for versioning — allows ignore_changes on the bucket itself.
resource "aws_s3_bucket_versioning" "primary" {
  bucket = aws_s3_bucket.primary.id
  versioning_configuration { status = "Enabled" }

  lifecycle {
    # Let the automated process toggle this without Terraform fighting it.
    ignore_changes = [versioning_configuration]
  }
}

# Step 2: Import the existing RDS instance.
# terraform import aws_db_instance.prod my-existing-db-identifier
resource "aws_db_instance" "prod" {
  identifier        = "my-existing-prod-db"
  instance_class    = "db.t3.medium"
  engine            = "mysql"
  engine_version    = "8.0"
  username          = "dbadmin"
  # Do NOT put the actual password here — use a data source or ignore_changes.
  password          = "placeholder-replaced-by-ignore"
  allocated_storage = 100
  skip_final_snapshot = false
  final_snapshot_identifier = "prod-final-snapshot"

  tags = {
    Environment = "prod"
    ManagedBy   = "terraform"
    # BackupOwner tag is managed by a Lambda — do not manage here.
  }

  lifecycle {
    # The nightly Lambda modifies BackupOwner tag and password rotation
    # changes the password. Ignore these so they don't cause plan diffs.
    ignore_changes = [
      password,
      tags["BackupOwner"],
    ]

    # CRITICAL: prevent any accidental terraform destroy of the production database.
    prevent_destroy = true
  }
}

# Step 3: moved block — rename resource address without destroying/recreating.
# Situation: the old code used aws_s3_bucket.legacy, new code uses aws_s3_bucket.primary.
# This tells Terraform to move the state entry — no destroy/recreate happens.
moved {
  from = aws_s3_bucket.legacy
  to   = aws_s3_bucket.primary
}
# After this apply succeeds, remove the moved block in the next commit.

Key Workflow — Import, Drift Detection, and Targeted Replace

### IMPORT existing resources into state ###
# 1. Write the resource blocks in main.tf (as above)
# 2. Import the S3 bucket
terraform import aws_s3_bucket.primary my-existing-prod-bucket

# 3. Import the RDS instance
terraform import aws_db_instance.prod my-existing-prod-db

# 4. Run plan — adjust HCL until no changes are shown
terraform plan  # fix any differences between HCL and actual config

# 5. Apply to converge state
terraform apply


### DRIFT DETECTION — check what changed in AWS vs state ###
# -refresh-only updates state to match real AWS, does not propose resource changes
terraform plan -refresh-only

# If drift is detected and you want to accept the real AWS state:
terraform apply -refresh-only
# This updates state to match AWS without changing any resources


### TARGETED RESOURCE REPLACEMENT ###
# Force destroy and recreate a specific resource (e.g., a compromised EC2 instance)
# Does NOT destroy other resources
terraform apply -replace="aws_instance.web"

# Alternatively, mark in state (deprecated in favour of -replace):
# terraform taint aws_instance.web


### STATE MANIPULATION ###
# List all state addresses
terraform state list

# Show details of a specific resource
terraform state show aws_db_instance.prod

# Manually remove a resource from state (without destroying it in AWS)
# Use when handing a resource to another team's state
terraform state rm aws_s3_bucket.legacy_temp

# Move resource between state addresses (same as moved block but CLI-only)
terraform state mv aws_s3_bucket.legacy aws_s3_bucket.primary


### PREVENT_DESTROY in action ###
# Try to destroy with prevent_destroy = true — will fail at plan with a clear error
terraform destroy
# │ Error: Instance cannot be destroyed
# │ Resource aws_db_instance.prod has lifecycle.prevent_destroy set to true.


### MOVED BLOCK verification ###
# Before moved block: state has aws_s3_bucket.legacy
# After adding moved block and running apply:
terraform plan   # shows: "aws_s3_bucket.legacy has moved to aws_s3_bucket.primary"
terraform apply  # executes the move — no destroy/recreate
# Remove the moved {} block from code after successful apply

outputs.tf

output "bucket_name"   { value = aws_s3_bucket.primary.bucket }
output "bucket_arn"    { value = aws_s3_bucket.primary.arn }
output "db_endpoint"   { value = aws_db_instance.prod.endpoint }
output "db_identifier" { value = aws_db_instance.prod.identifier }

Summary of State Management Commands

Command                              Purpose
─────────────────────────────────────────────────────────────────────────────
terraform import           Import existing resource into state
terraform plan -refresh-only         Detect drift (what changed in AWS)
terraform apply -refresh-only        Accept drift into state without changes
terraform apply -replace=      Force destroy + recreate one resource
terraform state list                 List all state addresses
terraform state show           Show full state for one resource
terraform state rm             Remove from state (does not destroy in AWS)
terraform state mv         Rename in state (prefer moved {} block)
moved { from = ...; to = ... }       Code-based rename — tracked in git history
✓ Expected: Plan After Import (should show no changes)
terraform import aws_s3_bucket.primary my-existing-prod-bucket
Import successful!

terraform import aws_db_instance.prod my-existing-prod-db
Import successful!

terraform plan
# After matching HCL to actual config:
No changes. Your infrastructure matches the configuration.

# After adding moved block:
terraform plan
  # aws_s3_bucket.legacy has moved to aws_s3_bucket.primary
  # Plan: 0 to add, 0 to change, 0 to destroy.