LAB47_AWS_In-Memory Caching: ElastiCache Redis Cluster with Subnet Group and Auth

In-Memory Caching: ElastiCache Redis Cluster with Subnet Group and Auth

☁ AWS 🔥 Advanced ElastiCache Redis Replication Group Auth Token Encryption Subnet Group

Scenario

Your application's database is under heavy read load because every API response requires multiple queries for session data, product catalog lookups, and user preferences. You need an in-memory caching layer using Amazon ElastiCache for Redis with high availability (Multi-AZ with automatic failover), encryption in-transit and at-rest, and AUTH token authentication to prevent unauthorized access.

Your Objectives
  • Create an ElastiCache subnet group covering private subnets in at least 2 AZs.
  • Create an ElastiCache parameter group for Redis 7.x that sets maxmemory-policy to allkeys-lru (evict least recently used keys when memory is full).
  • Generate a secure AUTH token using random_password and store it in Secrets Manager.
  • Create an aws_elasticache_replication_group with 2 nodes (cache.r7g.large), Multi-AZ automatic failover, TLS (transit_encryption_enabled), at-rest encryption (at_rest_encryption_enabled), and the AUTH token.
  • Create a security group allowing Redis port (6379) only from the application tier CIDR.
  • Output the primary endpoint, the reader endpoint, and the secret ARN containing the auth token.

Additional Context

Replication Group vs Cluster Mode: A replication group without cluster mode has one primary shard with up to 5 read replicas — great for read-heavy workloads. Cluster mode splits data across multiple shards for write scaling (e.g., 5 shards × 2 nodes = 10 nodes). For most applications, a single-shard replication group with replicas is the right starting point.

AUTH token requirement: When transit_encryption_enabled (TLS) is enabled and an auth_token is set, clients must use the AUTH command with the token before running any Redis commands. Without TLS, the AUTH token cannot be set (Redis rejects it). The token must be 16-128 characters.

variables.tf

variable "aws_region"    { type = string; default = "us-east-1" }
variable "vpc_id"        { type = string }
variable "subnet_ids"    { type = list(string); description = "Private subnets in 2+ AZs." }
variable "app_cidr"      { type = string; default = "10.0.0.0/16"; description = "App tier CIDR allowed to connect." }
variable "environment"   { type = string; default = "prod" }
variable "cache_name"    { type = string; default = "app-cache" }

main.tf

provider "aws" { region = var.aws_region }
locals { tags = { Environment = var.environment, ManagedBy = "terraform" } }

# AUTH token — must be 16-128 characters, stored in Secrets Manager.
resource "random_password" "redis_auth" {
  length  = 32
  special = false  # Redis AUTH token must not contain @, : or /
}

resource "aws_secretsmanager_secret" "redis_auth" {
  name                    = "${var.cache_name}-auth-token-${var.environment}"
  recovery_window_in_days = 7
  tags                    = local.tags
}
resource "aws_secretsmanager_secret_version" "redis_auth" {
  secret_id     = aws_secretsmanager_secret.redis_auth.id
  secret_string = random_password.redis_auth.result
}

# Subnet group — must span 2+ AZs for Multi-AZ.
resource "aws_elasticache_subnet_group" "main" {
  name       = "${var.cache_name}-subnet-group-${var.environment}"
  subnet_ids = var.subnet_ids
  tags       = local.tags
}

# Parameter group — configure eviction policy.
resource "aws_elasticache_parameter_group" "redis7" {
  name   = "${var.cache_name}-params-${var.environment}"
  family = "redis7"

  # evict least recently used keys when memory is full.
  parameter { name = "maxmemory-policy"; value = "allkeys-lru" }

  # Increase the number of databases.
  parameter { name = "databases"; value = "16" }

  tags = local.tags
}

# Security group — only app tier can connect on Redis port.
resource "aws_security_group" "redis" {
  name   = "${var.cache_name}-sg-${var.environment}"
  vpc_id = var.vpc_id

  ingress {
    description = "Redis from app tier"
    from_port   = 6379; to_port = 6379; protocol = "tcp"
    cidr_blocks = [var.app_cidr]
  }
  egress { from_port = 0; to_port = 0; protocol = "-1"; cidr_blocks = ["0.0.0.0/0"] }
  tags = merge(local.tags, { Name = "${var.cache_name}-sg" })
}

# ElastiCache Replication Group — Redis 7 with Multi-AZ, TLS, and AUTH.
resource "aws_elasticache_replication_group" "main" {
  replication_group_id = "${var.cache_name}-${var.environment}"
  description          = "Redis cache for ${var.cache_name} — ${var.environment}"

  node_type            = "cache.r7g.large"
  num_cache_clusters   = 2  # 1 primary + 1 replica
  automatic_failover_enabled = true
  multi_az_enabled           = true

  engine_version       = "7.1"
  port                 = 6379
  parameter_group_name = aws_elasticache_parameter_group.redis7.name
  subnet_group_name    = aws_elasticache_subnet_group.main.name
  security_group_ids   = [aws_security_group.redis.id]

  # Encryption — both required when using AUTH token.
  transit_encryption_enabled = true
  at_rest_encryption_enabled = true
  auth_token                 = random_password.redis_auth.result

  maintenance_window         = "sun:05:00-sun:06:00"
  snapshot_retention_limit   = 7  # daily snapshots for 7 days
  snapshot_window            = "03:00-04:00"

  # Avoid destroy/recreate on minor version upgrades.
  apply_immediately = false

  tags = local.tags
}

outputs.tf

output "primary_endpoint" {
  value = aws_elasticache_replication_group.main.primary_endpoint_address
}
output "reader_endpoint" {
  value = aws_elasticache_replication_group.main.reader_endpoint_address
}
output "redis_port"     { value = 6379 }
output "auth_secret_arn" { value = aws_secretsmanager_secret.redis_auth.arn }
output "auth_token"     { value = random_password.redis_auth.result; sensitive = true }

Workflow Commands

terraform init
terraform apply -var="vpc_id=vpc-xxxx" -var='subnet_ids=["subnet-aaa","subnet-bbb"]'

# Retrieve auth token from Secrets Manager
AUTH=$(aws secretsmanager get-secret-value \
  --secret-id $(terraform output -raw auth_secret_arn) \
  --query SecretString --output text)

# Test connectivity (from EC2 in the app tier)
redis-cli -h $(terraform output -raw primary_endpoint) \
  -p 6379 --tls --no-auth-warning -a "$AUTH" ping
# Expected: PONG

# Test a key-value operation
redis-cli -h $(terraform output -raw primary_endpoint) \
  -p 6379 --tls --no-auth-warning -a "$AUTH" \
  SET session:user123 '{"role":"admin","lastLogin":"2024-01-15"}' EX 3600

redis-cli -h $(terraform output -raw reader_endpoint) \
  -p 6379 --tls --no-auth-warning -a "$AUTH" GET session:user123

terraform destroy -var="vpc_id=..." -var='subnet_ids=["..."]'
✓ Expected Output After Apply
Apply complete! Resources: 7 added, 0 changed, 0 destroyed.

Outputs:

auth_secret_arn  = "arn:aws:secretsmanager:us-east-1:123456789012:secret:app-cache-auth-token-prod-xxxxxx"
auth_token       = <sensitive>
primary_endpoint = "app-cache-prod.abc123.ng.0001.use1.cache.amazonaws.com"
reader_endpoint  = "app-cache-prod-ro.abc123.ng.0001.use1.cache.amazonaws.com"
redis_port       = 6379

Related Posts