LAB44_AWS_Terraform Workspaces + Remote State Data Sources: Env Pipeline

Terraform Workspaces + Remote State Data Sources: Env Pipeline

☁ AWS 🔥 Advanced Workspaces Remote State terraform_remote_state S3 Backend DynamoDB Lock Layered Architecture

Scenario

Your infrastructure is split into two layers: a networking layer (VPCs, subnets, route tables) managed by the networking team, and an application layer (ECS, RDS, ALB) managed by the app team. Each layer needs to run in three environments (dev, staging, prod) without duplicating configuration. The app layer must read VPC IDs and subnet IDs from the networking layer's state without hard-coding them.

Your Objectives
  • Set up an S3 backend with DynamoDB locking for the networking layer. Use workspaces (dev, staging, prod) and terraform.workspace to vary resource sizing by environment.
  • In the networking layer, use terraform.workspace in CIDR calculations (e.g., dev=10.10.x.x, staging=10.20.x.x, prod=10.30.x.x) using a lookup map.
  • Set up the application layer with its own S3 backend. Use terraform_remote_state to read the networking layer's outputs (vpc_id, private_subnet_ids) without hard-coding them.
  • Use locals to size resources by workspace (e.g., RDS instance class: db.t3.micro in dev, db.t3.small in staging, db.t3.medium in prod).
  • Output the current workspace name and the instance class being used.

Additional Context

Workspaces vs separate state files: Workspaces store each workspace's state in a separate key within the same backend (env:/dev/terraform.tfstate, env:/prod/terraform.tfstate). This is convenient for simple multi-environment setups. For strict environment isolation (different AWS accounts per env), use separate backend configurations or Terraform Cloud workspaces mapped to different AWS credential sets.

terraform_remote_state requires the producing state to have the right outputs defined. Any output in the networking layer's outputs.tf is accessible in the consuming configuration as data.terraform_remote_state.network.outputs.<output_name>. This creates a soft dependency between teams — the networking team controls the output contract.

Networking Layer: backend.tf

terraform {
  backend "s3" {
    bucket         = "my-tf-state-bucket"
    key            = "networking/terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "tf-state-locks"
    encrypt        = true
  }
}
# Workspace state is stored at: env://networking/terraform.tfstate

Networking Layer: main.tf

provider "aws" { region = "us-east-1" }

# Use workspace to select different CIDR ranges per environment.
locals {
  vpc_cidrs = {
    dev     = "10.10.0.0/16"
    staging = "10.20.0.0/16"
    prod    = "10.30.0.0/16"
  }
  vpc_cidr = lookup(local.vpc_cidrs, terraform.workspace, "10.99.0.0/16")

  tags = { Environment = terraform.workspace, ManagedBy = "terraform" }
}

resource "aws_vpc" "main" {
  cidr_block           = local.vpc_cidr
  enable_dns_support   = true
  enable_dns_hostnames = true
  tags = merge(local.tags, { Name = "vpc-${terraform.workspace}" })
}

resource "aws_subnet" "private" {
  count             = 2
  vpc_id            = aws_vpc.main.id
  cidr_block        = cidrsubnet(local.vpc_cidr, 8, count.index + 10)
  availability_zone = "us-east-1${element(["a","b"], count.index)}"
  tags = merge(local.tags, { Name = "private-${count.index}-${terraform.workspace}" })
}

resource "aws_subnet" "public" {
  count                   = 2
  vpc_id                  = aws_vpc.main.id
  cidr_block              = cidrsubnet(local.vpc_cidr, 8, count.index + 1)
  availability_zone       = "us-east-1${element(["a","b"], count.index)}"
  map_public_ip_on_launch = true
  tags = merge(local.tags, { Name = "public-${count.index}-${terraform.workspace}" })
}

resource "aws_internet_gateway" "main" {
  vpc_id = aws_vpc.main.id
  tags   = merge(local.tags, { Name = "igw-${terraform.workspace}" })
}

Networking Layer: outputs.tf

output "vpc_id"              { value = aws_vpc.main.id }
output "vpc_cidr"            { value = aws_vpc.main.cidr_block }
output "private_subnet_ids"  { value = aws_subnet.private[*].id }
output "public_subnet_ids"   { value = aws_subnet.public[*].id }
output "igw_id"              { value = aws_internet_gateway.main.id }
output "workspace"           { value = terraform.workspace }

Application Layer: backend.tf

terraform {
  backend "s3" {
    bucket         = "my-tf-state-bucket"
    key            = "application/terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "tf-state-locks"
    encrypt        = true
  }
}

Application Layer: main.tf

provider "aws" { region = "us-east-1" }

# Read the networking layer's state for the current workspace.
data "terraform_remote_state" "network" {
  backend = "s3"
  config = {
    bucket = "my-tf-state-bucket"
    key    = "env:/${terraform.workspace}/networking/terraform.tfstate"
    region = "us-east-1"
  }
}

# Size resources based on the current workspace.
locals {
  instance_classes = {
    dev     = "db.t3.micro"
    staging = "db.t3.small"
    prod    = "db.t3.medium"
  }
  rds_class = lookup(local.instance_classes, terraform.workspace, "db.t3.micro")
  tags = { Environment = terraform.workspace, ManagedBy = "terraform" }
}

# Consume networking outputs — no hard-coded IDs.
resource "aws_db_subnet_group" "main" {
  name       = "app-db-subnet-${terraform.workspace}"
  subnet_ids = data.terraform_remote_state.network.outputs.private_subnet_ids
  tags       = local.tags
}

resource "aws_security_group" "rds" {
  name   = "app-rds-sg-${terraform.workspace}"
  vpc_id = data.terraform_remote_state.network.outputs.vpc_id  # from remote state
  ingress { from_port = 5432; to_port = 5432; protocol = "tcp"
    cidr_blocks = [data.terraform_remote_state.network.outputs.vpc_cidr] }
  egress  { from_port = 0;    to_port = 0;    protocol = "-1"; cidr_blocks = ["0.0.0.0/0"] }
  tags = local.tags
}

resource "random_password" "db" { length = 24; special = false }

resource "aws_db_instance" "app" {
  identifier           = "app-db-${terraform.workspace}"
  engine               = "postgres"; engine_version = "15"
  instance_class       = local.rds_class  # sized by workspace
  allocated_storage    = terraform.workspace == "prod" ? 100 : 20
  db_name              = "appdb"
  username             = "appuser"
  password             = random_password.db.result
  db_subnet_group_name = aws_db_subnet_group.main.name
  vpc_security_group_ids = [aws_security_group.rds.id]
  skip_final_snapshot  = true
  tags                 = local.tags
}

Application Layer: outputs.tf

output "workspace"       { value = terraform.workspace }
output "rds_class"       { value = local.rds_class }
output "vpc_id"          { value = data.terraform_remote_state.network.outputs.vpc_id }
output "db_endpoint"     { value = aws_db_instance.app.endpoint }

Workflow Commands

### Networking layer
cd networking/
terraform init
terraform workspace new dev         # create the 'dev' workspace
terraform workspace new staging
terraform workspace new prod

terraform workspace select dev
terraform apply                     # creates 10.10.0.0/16 VPC

terraform workspace select staging
terraform apply                     # creates 10.20.0.0/16 VPC

terraform workspace select prod
terraform apply                     # creates 10.30.0.0/16 VPC

### Application layer (run after networking)
cd ../application/
terraform init
terraform workspace new dev && terraform workspace select dev
terraform apply                     # uses db.t3.micro, reads VPC from remote state

terraform output workspace          # → dev
terraform output rds_class          # → db.t3.micro

terraform workspace select prod
terraform apply                     # uses db.t3.medium, reads prod VPC
✓ Expected Output After Apply (prod workspace)
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.

Outputs:

db_endpoint = "app-db-prod.abc123.us-east-1.rds.amazonaws.com:5432"
rds_class   = "db.t3.medium"
vpc_id      = "vpc-0xxx..."   (read from networking layer remote state)
workspace   = "prod"

Related Posts