LAB26_AWS_Three-Tier VPC: Public, Private App, and Private Data Subnets with Flow Logs

Three-Tier VPC: Public, Private App, and Private Data Subnets with Flow Logs

☁ AWS 🔥 Advanced VPC Flow Logs VPC Endpoints NACL Multi-AZ

Scenario

Your security team has audited your AWS environment and identified the following gaps: the VPC has no flow logging (no audit trail of network connections), private instances are sending all S3 and DynamoDB traffic through the NAT Gateway (unnecessary egress cost), and the data tier subnets share a route table with the application tier (no isolation).

Design and build a production-grade three-tier VPC from scratch: a public tier for load balancers, a private application tier for compute, and a private data tier for databases — each isolated with its own route tables and NACLs. Enable VPC Flow Logs to CloudWatch and add Gateway VPC Endpoints for S3 and DynamoDB so that traffic to those services never leaves the AWS network.

Your Objectives
  • Create a VPC with DNS support enabled. Use for_each over a map to create 6 subnets (2 public, 2 private-app, 2 private-data) across 2 AZs from a single resource block.
  • Create an Internet Gateway and a public route table. Create a NAT Gateway (with EIP) in each public subnet and separate private route tables per AZ pointing to their local NAT Gateway.
  • Create separate route tables for the data tier — with no NAT route (databases should not reach the internet).
  • Add Network ACLs: the data tier NACL should deny all inbound traffic except from the app tier CIDR range on port 3306/5432.
  • Create a CloudWatch Log Group and an IAM role to enable VPC Flow Logs for the entire VPC.
  • Add Gateway VPC Endpoints for S3 and DynamoDB, associated with the private route tables.
  • Output all subnet IDs grouped by tier, NAT Gateway IPs, and the flow log ID.

Additional Context

Gateway Endpoints vs Interface Endpoints: Gateway endpoints (S3, DynamoDB) are free and work by injecting entries into your route table pointing to the AWS service prefix list. They do not require a private DNS name and do not consume an ENI or IP address. Interface endpoints (most other services) create an ENI in your subnet — they cost ~$0.01/hour per AZ plus data processing fees.

Dual NAT Gateway pattern: In production, create one NAT Gateway per AZ. If a single NAT Gateway fails (they are AZ-scoped), all private instances across all AZs lose outbound internet access. Using one NAT per AZ ensures AZ-level fault isolation at the cost of an additional NAT Gateway fee (~$0.045/hour each).

variables.tf

variable "aws_region"   { type = string; default = "us-east-1" }
variable "environment"  { type = string; default = "prod" }
variable "vpc_cidr"     { type = string; default = "10.0.0.0/16" }

# Map-driven subnet definitions — key = logical name, value = config object.
variable "subnets" {
  type = map(object({ cidr = string; az = string; tier = string }))
  default = {
    "public-a"       = { cidr = "10.0.1.0/24";  az = "us-east-1a"; tier = "public" }
    "public-b"       = { cidr = "10.0.2.0/24";  az = "us-east-1b"; tier = "public" }
    "private-app-a"  = { cidr = "10.0.11.0/24"; az = "us-east-1a"; tier = "private-app" }
    "private-app-b"  = { cidr = "10.0.12.0/24"; az = "us-east-1b"; tier = "private-app" }
    "private-data-a" = { cidr = "10.0.21.0/24"; az = "us-east-1a"; tier = "private-data" }
    "private-data-b" = { cidr = "10.0.22.0/24"; az = "us-east-1b"; tier = "private-data" }
  }
}

main.tf

# main.tf

provider "aws" { region = var.aws_region }

locals {
  # Filter subnets by tier for route table associations.
  public_subnets   = { for k, v in var.subnets : k => v if v.tier == "public" }
  app_subnets      = { for k, v in var.subnets : k => v if v.tier == "private-app" }
  data_subnets     = { for k, v in var.subnets : k => v if v.tier == "private-data" }
  azs = distinct(values(var.subnets)[*].az)  # ["us-east-1a","us-east-1b"]
  tags = { Environment = var.environment, ManagedBy = "terraform" }
}

# VPC
resource "aws_vpc" "main" {
  cidr_block           = var.vpc_cidr
  enable_dns_support   = true
  enable_dns_hostnames = true
  tags = merge(local.tags, { Name = "main-vpc-${var.environment}" })
}

# All subnets — one for_each covers all 6 from the variable map.
resource "aws_subnet" "all" {
  for_each          = var.subnets
  vpc_id            = aws_vpc.main.id
  cidr_block        = each.value.cidr
  availability_zone = each.value.az
  map_public_ip_on_launch = each.value.tier == "public"
  tags = merge(local.tags, { Name = "${each.key}-${var.environment}", Tier = each.value.tier })
}

# Internet Gateway.
resource "aws_internet_gateway" "main" {
  vpc_id = aws_vpc.main.id
  tags   = merge(local.tags, { Name = "igw-${var.environment}" })
}

# Public route table — routes 0.0.0.0/0 to IGW.
resource "aws_route_table" "public" {
  vpc_id = aws_vpc.main.id
  route { cidr_block = "0.0.0.0/0"; gateway_id = aws_internet_gateway.main.id }
  tags = merge(local.tags, { Name = "public-rt-${var.environment}" })
}

resource "aws_route_table_association" "public" {
  for_each       = local.public_subnets
  subnet_id      = aws_subnet.all[each.key].id
  route_table_id = aws_route_table.public.id
}

# EIP + NAT Gateway per AZ (fault-isolated outbound internet for private tiers).
resource "aws_eip" "nat" {
  for_each = local.public_subnets
  domain   = "vpc"
  tags     = merge(local.tags, { Name = "nat-eip-${each.key}" })
}

resource "aws_nat_gateway" "main" {
  for_each      = local.public_subnets
  allocation_id = aws_eip.nat[each.key].id
  subnet_id     = aws_subnet.all[each.key].id
  tags          = merge(local.tags, { Name = "nat-${each.key}" })
}

# Private app route tables — one per AZ, routes to AZ-local NAT Gateway.
resource "aws_route_table" "private_app" {
  for_each = local.app_subnets
  vpc_id   = aws_vpc.main.id
  # Route to the NAT Gateway in the matching public subnet for this AZ.
  route {
    cidr_block     = "0.0.0.0/0"
    nat_gateway_id = aws_nat_gateway.main[replace(each.key, "private-app", "public")].id
  }
  tags = merge(local.tags, { Name = "private-app-rt-${each.key}" })
}

resource "aws_route_table_association" "private_app" {
  for_each       = local.app_subnets
  subnet_id      = aws_subnet.all[each.key].id
  route_table_id = aws_route_table.private_app[each.key].id
}

# Data tier route tables — NO NAT route. Databases should not reach the internet.
resource "aws_route_table" "private_data" {
  for_each = local.data_subnets
  vpc_id   = aws_vpc.main.id
  # Intentionally NO default route — data tier is fully isolated.
  tags = merge(local.tags, { Name = "private-data-rt-${each.key}" })
}

resource "aws_route_table_association" "private_data" {
  for_each       = local.data_subnets
  subnet_id      = aws_subnet.all[each.key].id
  route_table_id = aws_route_table.private_data[each.key].id
}

# NACL for data tier — allow only MySQL (3306) from app tier CIDRs.
resource "aws_network_acl" "data" {
  vpc_id     = aws_vpc.main.id
  subnet_ids = [for k, v in local.data_subnets : aws_subnet.all[k].id]

  ingress { rule_no = 100; action = "allow"; protocol = "tcp"; from_port = 3306; to_port = 3306; cidr_block = "10.0.11.0/23" }
  ingress { rule_no = 200; action = "allow"; protocol = "tcp"; from_port = 1024; to_port = 65535; cidr_block = "10.0.0.0/16" }
  ingress { rule_no = 32766; action = "deny"; protocol = "-1"; from_port = 0; to_port = 0; cidr_block = "0.0.0.0/0" }
  egress  { rule_no = 100; action = "allow"; protocol = "tcp"; from_port = 1024; to_port = 65535; cidr_block = "10.0.0.0/16" }
  egress  { rule_no = 32766; action = "deny"; protocol = "-1"; from_port = 0; to_port = 0; cidr_block = "0.0.0.0/0" }
  tags = merge(local.tags, { Name = "data-tier-nacl-${var.environment}" })
}

# VPC Flow Logs — captures all accepted/rejected connection metadata.
resource "aws_cloudwatch_log_group" "flow_logs" {
  name              = "/vpc/flow-logs/${var.environment}"
  retention_in_days = 90
}

data "aws_iam_policy_document" "flow_logs_assume" {
  statement {
    effect  = "Allow"
    actions = ["sts:AssumeRole"]
    principals { type = "Service"; identifiers = ["vpc-flow-logs.amazonaws.com"] }
  }
}

data "aws_iam_policy_document" "flow_logs_write" {
  statement {
    effect    = "Allow"
    actions   = ["logs:CreateLogGroup","logs:CreateLogStream","logs:PutLogEvents","logs:DescribeLogGroups","logs:DescribeLogStreams"]
    resources = ["${aws_cloudwatch_log_group.flow_logs.arn}:*"]
  }
}

resource "aws_iam_role" "flow_logs" {
  name               = "vpc-flow-logs-role-${var.environment}"
  assume_role_policy = data.aws_iam_policy_document.flow_logs_assume.json
}

resource "aws_iam_role_policy" "flow_logs" {
  name   = "vpc-flow-logs-write"
  role   = aws_iam_role.flow_logs.id
  policy = data.aws_iam_policy_document.flow_logs_write.json
}

resource "aws_flow_log" "main" {
  vpc_id         = aws_vpc.main.id
  traffic_type   = "ALL"   # log ACCEPT, REJECT, and ALL traffic
  iam_role_arn   = aws_iam_role.flow_logs.arn
  log_destination = aws_cloudwatch_log_group.flow_logs.arn
  tags           = merge(local.tags, { Name = "vpc-flow-log-${var.environment}" })
}

# Gateway VPC Endpoints — free, routed via route table. No ENI, no cost.
data "aws_route_tables" "private_all" {
  depends_on = [aws_route_table.private_app, aws_route_table.private_data]
  vpc_id = aws_vpc.main.id
  filter { name = "tag:ManagedBy"; values = ["terraform"] }
}

resource "aws_vpc_endpoint" "s3" {
  vpc_id            = aws_vpc.main.id
  service_name      = "com.amazonaws.${var.aws_region}.s3"
  vpc_endpoint_type = "Gateway"
  route_table_ids   = concat(
    [for rt in aws_route_table.private_app  : rt.id],
    [for rt in aws_route_table.private_data : rt.id]
  )
  tags = merge(local.tags, { Name = "vpce-s3-${var.environment}" })
}

resource "aws_vpc_endpoint" "dynamodb" {
  vpc_id            = aws_vpc.main.id
  service_name      = "com.amazonaws.${var.aws_region}.dynamodb"
  vpc_endpoint_type = "Gateway"
  route_table_ids   = [for rt in aws_route_table.private_app : rt.id]
  tags = merge(local.tags, { Name = "vpce-dynamodb-${var.environment}" })
}

outputs.tf

output "vpc_id"          { value = aws_vpc.main.id }
output "public_subnets"  { value = { for k, v in local.public_subnets  : k => aws_subnet.all[k].id } }
output "app_subnets"     { value = { for k, v in local.app_subnets     : k => aws_subnet.all[k].id } }
output "data_subnets"    { value = { for k, v in local.data_subnets    : k => aws_subnet.all[k].id } }
output "nat_gateway_ips" { value = { for k, v in aws_eip.nat           : k => v.public_ip } }
output "flow_log_id"     { value = aws_flow_log.main.id }

Workflow Commands

terraform init
terraform apply

# Verify S3 gateway endpoint is in the route tables
aws ec2 describe-route-tables \
  --filters "Name=vpc-id,Values=$(terraform output -raw vpc_id)" \
  --query "RouteTables[].Routes[?DestinationPrefixListId!=null]"

# Query flow logs (allow 5-10 minutes for first entries)
aws logs tail /vpc/flow-logs/prod --follow

terraform destroy
✓ Expected Output After Apply
Apply complete! Resources: 21 added, 0 changed, 0 destroyed.

Outputs:

app_subnets     = { "private-app-a" = "subnet-...", "private-app-b" = "subnet-..." }
data_subnets    = { "private-data-a" = "subnet-...", "private-data-b" = "subnet-..." }
flow_log_id     = "fl-0abc1234567890def"
nat_gateway_ips = { "public-a" = "54.201.10.1", "public-b" = "54.201.20.2" }
public_subnets  = { "public-a" = "subnet-...", "public-b" = "subnet-..." }
vpc_id          = "vpc-0abc123456def7890"