Skip to main content

atmos terraform backend

Use these commands to manage Terraform state backend infrastructure. This solves the Terraform bootstrap problem by automatically provisioning backend storage with secure defaults, making it compatible with any Terraform-managed backend.

atmos terraform backend --help

Usage​

atmos terraform backend <subcommand> [options]

Available Subcommands​

create <component>
Provision backend infrastructure for a component
list
List all backends in a stack
describe <component>
Show backend configuration from stack
update <component>
Update backend configuration (idempotent)
delete <component>
Delete backend infrastructure (requires --force)

Creating Backends​

Provision backend infrastructure for a component in a specific stack:

atmos terraform backend create <component> --stack <stack>

The backend must have provision.backend.enabled: true in its stack configuration.

Examples​

Create Backend​

Provision an S3 bucket with secure defaults:

atmos terraform backend create vpc --stack dev

This creates an S3 bucket (if it doesn't exist) with:

  • Versioning enabled
  • AES-256 encryption
  • Public access blocked
  • Resource tags applied

List Backends​

Show all backend configurations in a stack:

atmos terraform backend list --stack dev

Describe Backend Configuration​

View a component's backend configuration from the stack:

atmos terraform backend describe vpc --stack dev
atmos terraform backend describe vpc --stack dev --format json

Update Backend​

Apply configuration changes to existing backend (idempotent):

atmos terraform backend update vpc --stack dev

Delete Backend​

Remove backend infrastructure (requires --force for safety):

atmos terraform backend delete vpc --stack dev --force

Provision Multiple Backends​

atmos terraform backend create vpc --stack dev
atmos terraform backend create eks --stack dev
atmos terraform backend create rds --stack dev

Arguments​

component
The Atmos component name (required for create, describe, update, delete)

Flags​

--stack / -s
Atmos stack name (required). Can also be set via ATMOS_STACK environment variable
--identity / -i
Identity to use for authentication. Overrides component's default identity. Use without value to select interactively
--format / -f
Output format for list and describe commands: table, yaml, json (default: varies by command)
--force
Force deletion without confirmation (delete command only)

Authentication​

Backend commands support both global and component-level authentication, consistent with regular Terraform commands.

Component-Level Identity​

Components can specify their own authentication identity with a default:

# stacks/dev.yaml
components:
  terraform:
    vpc:
      # Component-level auth configuration
      auth:
        identities:
          vpc-deployer:
            default: true  # Used automatically for this component
            kind: aws/permission-set
            via:
              provider: aws-sso
            principal:
              name: "NetworkAdmin"
              account:
                name: "network-dev"

With this configuration, backend commands automatically use the vpc-deployer identity:

# Automatically uses vpc-deployer identity
atmos terraform backend create vpc --stack dev
atmos terraform backend delete vpc --stack dev --force

Identity Flag Override​

The --identity flag overrides the component's default identity:

# Override component default with admin identity
atmos terraform backend create vpc --stack dev --identity admin

Global Identity​

Without component-level auth, backend commands use global auth configuration from atmos.yaml or the --identity flag.

See Authentication and Identity for complete auth configuration details.

How It Works​

Manual Provisioning​

When you run atmos terraform backend create:

  1. Load Configuration - Atmos loads the component's stack configuration
  2. Check Provisioning - Verifies provision.backend.enabled: true is set
  3. Select Provisioner - Chooses provisioner based on backend_type (s3, gcs, azurerm)
  4. Check Existence - Verifies if backend already exists (idempotent)
  5. Provision - Creates backend with hardcoded security defaults if needed
  6. Apply Settings - Configures versioning, encryption, access controls, and tags

Automatic Provisioning​

Backends are also provisioned automatically when running Terraform commands if provision.backend.enabled: true:

# Backend provisioned automatically before terraform init
atmos terraform plan vpc --stack dev
atmos terraform apply vpc --stack dev

The automatic flow:

Auth Setup (TerraformPreHook)
  ↓
Backend Provisioning (if enabled)
  ↓
Terraform Init
  ↓
Terraform Command

Solving the Terraform Bootstrap Problem​

Terraform has a chicken-and-egg problem: you need infrastructure to store state, but you need state to manage infrastructure. Atmos solves this by:

  1. Automatic Detection - Reads backend configuration from stack manifests
  2. Secure Defaults - Creates backend with hardcoded security settings
  3. Idempotent Operations - Safe to run multiple times
  4. Terraform Compatible - Works with any Terraform-managed backend

This eliminates manual backend setup and makes Terraform backends work like any other infrastructure resource.

Configuration​

Enable backend provisioning in your stack manifest:

# stacks/dev.yaml
components:
  terraform:
    vpc:
      backend_type: s3  # Must be at component level

      backend:
        bucket: acme-ue1-dev-tfstate
        key: vpc/terraform.tfstate
        region: us-east-1
        use_lockfile: true  # Enable native S3 locking (Terraform 1.10+)

      provision:
        backend:
          enabled: true  # Enable automatic provisioning

Configuration Hierarchy​

The provision.backend configuration supports Atmos's deep-merge system and can be specified at multiple levels in the stack hierarchy. This provides flexibility to set defaults at high levels and override at component level.

Global Default (Organization-Level)​

Enable provisioning for all components in an organization:

# stacks/orgs/acme/_defaults.yaml
terraform:
  provision:
    backend:
      enabled: true

All components in this organization will inherit provision.backend.enabled: true unless explicitly overridden.

Environment-Level Configuration​

Set different provisioning policies per environment:

# stacks/orgs/acme/plat/dev/_defaults.yaml
terraform:
  provision:
    backend:
      enabled: true  # Auto-provision in dev

# stacks/orgs/acme/plat/prod/_defaults.yaml
terraform:
  provision:
    backend:
      enabled: false  # Pre-provisioned backends in prod

Component Inheritance​

Use metadata.inherits to share provision configuration:

# stacks/catalog/vpc/defaults.yaml
components:
  terraform:
    vpc/defaults:
      provision:
        backend:
          enabled: true

# stacks/dev.yaml
components:
  terraform:
    vpc:
      metadata:
        inherits: [vpc/defaults]
      # Inherits provision.backend.enabled: true

Component-Level Override​

Override inherited settings per component:

components:
  terraform:
    vpc:
      provision:
        backend:
          enabled: false  # Disable for this specific component

Deep-Merge Behavior: Atmos combines configurations from all levels, giving you maximum flexibility:

  • Set defaults at organization or environment level
  • Override per component when needed
  • Use catalog inheritance for reusable patterns
  • Component-level configuration has highest precedence

Supported Backend Types​

S3 (AWS)​

Hardcoded Defaults:

  • Versioning: Enabled
  • Encryption: AES-256 (AWS-managed keys)
  • Public Access: Blocked (all 4 settings)
  • Locking: Native S3 locking (Terraform 1.10+)
  • Tags: Name, ManagedBy=Atmos

Required Configuration:

backend_type: s3  # Must be at component level

backend:
  bucket: my-terraform-state     # Required
  key: component/terraform.tfstate
  region: us-east-1              # Required
  use_lockfile: true             # Enable native S3 locking (Terraform 1.10+)

Cross-Account Support:

backend:
  bucket: my-terraform-state
  region: us-east-1
  use_lockfile: true  # Enable native S3 locking (Terraform 1.10+)
  assume_role:
    role_arn: arn:aws:iam::999999999999:role/TerraformStateAdmin

The provisioner assumes the role to create the bucket in the target account.

Error Handling​

Exit Codes​

Exit CodeError TypeAction
0SuccessBackend created or already exists
1General errorCheck error message for details
2Configuration errorFix provision.backend configuration
3Permission errorGrant required IAM permissions
4Resource conflictChange bucket name (globally unique)
5Network errorCheck network connectivity to cloud provider

Example Errors​

Missing Configuration:

Error: backend.bucket is required in backend configuration

Hint: Add bucket name to stack manifest
Example:
  backend:
    bucket: my-terraform-state
    region: us-east-1

Permission Denied:

Error: failed to create bucket: AccessDenied

Hint: Verify AWS credentials have s3:CreateBucket permission
Required IAM permissions:
  - s3:CreateBucket
  - s3:HeadBucket
  - s3:PutBucketVersioning
  - s3:PutBucketEncryption
  - s3:PutBucketPublicAccessBlock
  - s3:PutBucketTagging

Bucket Name Conflict:

Error: failed to create bucket: BucketAlreadyExists

Hint: S3 bucket names are globally unique across all AWS accounts
Try a different bucket name:
  - acme-ue1-dev-tfstate-123456789012 (add account ID)
  - acme-ue1-dev-tfstate-randomsuffix (add random suffix)

Required IAM Permissions​

S3 Backend​

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:CreateBucket",
        "s3:HeadBucket",
        "s3:PutBucketVersioning",
        "s3:PutBucketEncryption",
        "s3:PutBucketPublicAccessBlock",
        "s3:PutBucketTagging"
      ],
      "Resource": "arn:aws:s3:::my-terraform-state*"
    }
  ]
}

For cross-account provisioning, also add:

{
  "Effect": "Allow",
  "Action": "sts:AssumeRole",
  "Resource": "arn:aws:iam::999999999999:role/TerraformStateAdmin"
}

Migrating to Terraform-Managed Backends​

Once your backend is provisioned, you can import it into Terraform for advanced management:

Step 1: Provision the Backend​

Use Atmos to create the backend with secure defaults:

atmos terraform backend create vpc --stack prod

Step 2: Import into Terraform​

Add the backend to your Terraform configuration and import it:

# Import the provisioned backend
import {
  to = aws_s3_bucket.terraform_state
  id = "acme-ue1-prod-tfstate"
}

resource "aws_s3_bucket" "terraform_state" {
  bucket = "acme-ue1-prod-tfstate"
}

Step 3: Add Advanced Features​

Extend the backend with production-specific features:

# Add lifecycle rules
resource "aws_s3_bucket_lifecycle_configuration" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id

  rule {
    id     = "delete-old-versions"
    status = "Enabled"

    noncurrent_version_expiration {
      noncurrent_days = 90
    }
  }
}

# Add replication
resource "aws_s3_bucket_replication_configuration" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id
  role   = aws_iam_role.replication.arn

  rule {
    id     = "replicate-state"
    status = "Enabled"

    destination {
      bucket        = aws_s3_bucket.terraform_state_replica.arn
      storage_class = "STANDARD_IA"
    }
  }
}

Step 4: Disable Automatic Provisioning​

Once Terraform manages the backend, disable automatic provisioning:

provision:
  backend:
    enabled: false  # Backend now managed by Terraform

See the Automatic Backend Provisioning docs for more migration patterns.

Idempotent Operations​

The provision command is idempotent—running it multiple times is safe:

$ atmos terraform backend create vpc --stack dev
Running backend provisioner...
Creating S3 bucket 'acme-ue1-dev-tfstate'...
✓ Successfully provisioned backend

$ atmos terraform backend create vpc --stack dev
Running backend provisioner...
S3 bucket 'acme-ue1-dev-tfstate' already exists (idempotent)
✓ Backend provisioning completed