Skip to main content

Terraform Planfiles

Atmos provides sophisticated planfile management for Terraform, enabling safe and predictable infrastructure changes through plan generation, storage, and application workflows.

Overview​

A Terraform planfile is a binary file that contains the exact set of changes Terraform will make to your infrastructure. Atmos enhances Terraform's native planfile functionality by:

  • Standardizing planfile naming and storage
  • Enabling seamless plan-and-apply workflows
  • Supporting both automatic and custom planfile management
  • Integrating with CI/CD pipelines and approval workflows

Why Planfiles Are Important​

Planfiles are one of Terraform's most powerful features for ensuring safe, predictable infrastructure changes. They solve a critical problem: ensuring that what you reviewed is exactly what gets applied.

The Problem Without Planfiles​

Without planfiles, there's a dangerous gap between plan and apply:

  1. You run terraform plan and review the changes
  2. Time passes (minutes, hours, or days)
  3. During terraform apply:
    • Someone else might have modified the infrastructure
    • Configuration files might have changed
    • Provider APIs might return different data
    • Remote state might have been updated
  4. Result: The applied changes differ from what was reviewed!

How Planfiles Solve This​

A planfile is a snapshot that captures:

  • The exact infrastructure state at planning time
  • The precise changes Terraform will make
  • All variable values and provider configurations
  • The complete dependency graph

When you apply a planfile, Terraform uses this snapshot instead of re-calculating changes, guaranteeing that what you reviewed is what gets applied.

Benefits of Using Planfiles​

1. Guaranteed Consistency​

  • No Drift: Changes applied match exactly what was planned, even if infrastructure or configuration changed
  • Immutable Plans: Once generated, a planfile cannot be modified, ensuring integrity
  • State Version Tracking: Planfiles capture the state version at plan time to detect if the state has changed before applying. Note that actual concurrency protection comes from your backend's state locking mechanisms, not from planfiles themselves

2. Enhanced Security and Compliance​

  • Audit Trail: Planfiles serve as immutable records of approved changes for compliance
  • Review Process: Security teams can analyze planfiles before approval
  • Approval Workflows: Enable separation of duties - developers plan, ops team approves and applies
  • Policy Validation: Run security scanners (Checkov, tfsec, OPA) against planfiles. Use atmos terraform generate planfile to create the JSON version needed by most security scanning tools

3. CI/CD Pipeline Safety​

  • Atomic Operations: Apply exactly what passed through your CI/CD tests
  • Artifact Storage: Store planfiles in external storage (e.g., S3, GitHub Actions artifacts). Atmos provides a planfile storage GitHub Action for this purpose
  • Change Documentation: Saved planfiles provide a record of what changes were planned, useful for understanding what was deployed (not for automated rollback)
  • Multi-Stage Pipelines: Separate plan generation, validation, approval, and application stages

4. Operational Efficiency​

  • Faster Apply: No need to refresh state or recalculate the dependency graph (can be 50-70% faster)
  • Reduced API Calls: Fewer cloud provider API calls during apply (important for rate limits)
  • Predictable Timing: Apply duration is predictable since the work is pre-calculated
  • Offline Review: Planfiles can be converted to JSON/YAML for review without Terraform access

5. Team Collaboration​

  • Async Workflows: One engineer can generate plans for another to review and apply
  • Change Windows: Generate plans ahead of maintenance windows, apply during the window
  • Knowledge Transfer: Planfiles document infrastructure changes for team learning

Planfile Limitations and Challenges​

While planfiles provide significant benefits, they also have important limitations to understand:

Planfile Staleness​

Planfiles become stale and must be recreated if:

  • Infrastructure changes externally (e.g., manual changes, other automation)
  • Terraform state is updated by other operations
  • Configuration files are modified after plan generation
  • Provider data sources return different values

Plan Comparison and Verification​

When recreating planfiles, you cannot automatically verify they're identical to the original. Use atmos terraform plan-diff to compare plans and determine if they are effectively the same or have material differences.

# Generate new planfile
atmos terraform plan vpc -s dev -out vpc-dev.planfile

# Compare with previous plan to detect drift
atmos terraform plan-diff vpc -s dev

IAM Role and Authentication Issues​

If different IAM roles are used for plan vs apply, the planfile will cause failures:

  • Planfiles embed the IAM role used during plan generation (via provider assume_role configuration)
  • If you apply with a different IAM role, Terraform will fail because the embedded credentials don't match
  • Solution: Use the same IAM role for both plan and apply, or regenerate the planfile with the apply role

Regeneration and Re-verification​

When any of these issues occur:

  1. Regenerate the planfile with current state and credentials
  2. Re-verify the plan matches expectations (use plan-diff if comparing to a previous plan)
  3. Re-run approval workflows if your process requires it

Backend Limitations​

Terraform/OpenTofu Core Limitation

Not all Terraform backends support saving planfiles locally with the -out flag. This is a limitation of Terraform and OpenTofu core, not Atmos. Atmos works within these constraints and provides workarounds where possible.

Understanding Backend Types​

Terraform backends fall into two categories regarding planfile support:

  1. Standard Backends: Store state only, support local planfile generation
  2. Enhanced Backends: Manage both state and operations remotely, don't support local planfiles
Security Warning: Not All Backends Are Created Equal

When using -backend-config or hardcoding backend credentials in your configuration, Terraform embeds these values in plan files. This creates serious security risks:

  • Credentials Exposed: Sensitive data like usernames, passwords, and tokens are stored in plaintext within the planfile
  • CI/CD Breakage: Job-specific or time-limited tokens become invalid between plan and apply stages
  • Compliance Violations: Planfiles containing secrets cannot be safely stored, shared, or committed to version control

The HTTP backend is particularly problematic: While it technically supports terraform plan -out, the embedded credentials make it effectively unusable in any production or CI/CD environment. For all practical purposes, treat the HTTP backend as not supporting planfiles.

Best Practice: Use environment variables for backend authentication or choose backends that support secure credential management.

Backends That DO NOT Support Local Planfiles​

The following backends have limitations with terraform plan -out=<file>:

BackendTypeActual LimitationRecommended Approach
Remote BackendEnhancedDoes not support saving execution plans locally. Returns error: "The 'remote' backend does not support saving the generated execution plan locally"Use remote execution mode or migrate to standard backend with skip_planfile: true in Atmos
Terraform CloudEnhancedRemote execution mode doesn't allow local plan outputUse VCS-driven workflow or configure skip_planfile: true in Atmos
Terraform EnterpriseEnhancedSame as Terraform Cloud - remote execution limitationUse VCS-driven workflow or configure skip_planfile: true in Atmos
HTTP BackendStandard⚠️ Security Risk: While technically supports -out, backend credentials are embedded in planfiles making it effectively unusableDo not use planfiles with HTTP backend. Use environment variables for auth or switch backends

Backends That Support Planfiles​

All standard backends support local planfile generation. These are the most commonly used:

BackendStorage TypeBest ForKey Features
S3AWS S3AWS infrastructure• DynamoDB for locking
• Encryption at rest
• Versioning support
GCSGoogle Cloud StorageGCP infrastructure• Native state locking
• Customer-managed encryption keys
• Versioning support
Azure BlobAzure StorageAzure infrastructure• Native locking via blob leases
• Encryption at rest
• Soft delete capability
LocalFilesystemDevelopment/testing• Simple setup
• No external dependencies
• Not for production teams
ConsulHashiCorp ConsulMulti-cloud setups• Distributed storage
• High availability
• Built-in encryption
PostgreSQLDatabaseCentralized management• ACID compliance
• Existing database infrastructure
• SQL-based queries
KubernetesK8s SecretsKubernetes-native• Native K8s integration
• RBAC support
• Namespace isolation
OSSAlibaba Cloud OSSAlibaba Cloud• TableStore for locking
• Server-side encryption
COSTencent Cloud COSTencent Cloud• Native locking support
• Cross-region replication

Identifying Backend Limitations​

Common Error Messages​

When using a backend that doesn't support planfiles, you'll see one of these errors:

Terraform Cloud/Enterprise:

Error: Saving a generated plan is currently not supported

The "remote" backend does not support saving the generated execution
plan locally at this time.

Remote Backend:

Error: Run variables are currently not supported

The "remote" backend does not support setting run variables at this time.
Currently the only to way to pass variables is by creating a '*.auto.tfvars'
variables file or by setting environment variables.

HTTP Backend with Credentials:

Error: Error loading state: HTTP remote state endpoint requires auth

The plan was created with different credentials than those being used
to apply it.

Testing Your Backend​

# Check your backend configuration
$ atmos describe component vpc -s dev | grep backend

# Test if planfiles are supported
$ atmos terraform plan vpc -s dev

# Success: You'll see a message like:
# Saved the plan to: dev-vpc.planfile

# Failure: You'll see one of the error messages above

Working with Non-Supporting Backends​

If your backend doesn't support planfiles, you have several options:

Option 1: Skip Planfiles​

Configure Atmos to skip planfile generation:

atmos.yaml
components:
  terraform:
    plan:
      skip_planfile: true  # Disable for all components

Or per-command:

$ atmos terraform plan vpc -s dev --skip-planfile

Option 2: Use Backend-Specific Workflows​

For Terraform Cloud/Enterprise:

atmos.yaml
components:
  terraform:
    backend:
      type: remote
      organization: your-org
    plan:
      skip_planfile: true  # Required for remote backend

How Atmos Manages Planfiles​

Automatic Planfile Generation​

When you run atmos terraform plan, Atmos automatically:

  1. Generates a planfile with a standardized name
  2. Stores it in the component's working directory
  3. Makes it available for subsequent apply operations

Automatic Planfile Naming​

When no custom planfile name is specified with -out, Atmos automatically generates a planfile name following this pattern:

# Without folder prefix
<stack>-<component>.planfile

# With folder prefix
<stack>-<folder_prefix>-<component>.planfile

# Examples:
dev-vpc.planfile                    # Stack: dev, Component: vpc
prod-us-east-1-database.planfile    # Stack: prod, Folder: us-east-1, Component: database
staging-platform-kubernetes.planfile # Stack: staging, Folder: platform, Component: kubernetes

This automatic naming ensures:

  • Consistent planfile identification across teams
  • No accidental planfile overwrites between runs
  • Easy correlation between planfiles and their stack/component

Planfile Storage Location​

Planfiles are stored in the component's working directory:


## Working with Planfiles in Practice

### Binary Planfiles (Native Terraform)

When using supported backends, Atmos automatically generates binary planfiles:

```bash
# Atmos generates: dev-vpc.planfile
atmos terraform plan vpc -s dev

# Apply the generated planfile
atmos terraform apply vpc -s dev --from-plan

# Or specify a custom planfile name
atmos terraform plan vpc -s dev -out=my-custom.tfplan
atmos terraform apply vpc -s dev --planfile my-custom.tfplan

JSON/YAML Planfiles with atmos terraform generate planfile​

For enhanced security scanning, policy validation, or when binary planfiles aren't suitable, use Atmos's JSON/YAML planfile generation:

# Generate JSON planfile for security scanning
atmos terraform generate planfile vpc -s dev --file-template plan-{{.stack}}-{{.component}}.json

# Generate YAML planfile for human review
atmos terraform generate planfile vpc -s dev --file-template plan-{{.stack}}-{{.component}}.yaml

Integration with Security and Compliance Tools​

JSON planfiles enable integration with security scanning and policy tools:

# Generate JSON planfile for security scanning
atmos terraform generate planfile vpc -s prod --file-template prod-vpc.json

# Scan with Wiz.io or similar tools
wiz-cli iac scan --format terraform-plan prod-vpc.json

# Check with Open Policy Agent (OPA)
opa eval -d policies/ -i prod-vpc.json "data.terraform.deny[msg]"

# Validate with Checkov
checkov --framework terraform_plan -f prod-vpc.json

# Analyze with Terrascan
terrascan scan -i tfplan -f prod-vpc.json

Benefits of JSON/YAML planfiles:

  • Security Scanning: Integrate with tools like Wiz.io, Snyk, Checkov
  • Policy as Code: Validate with OPA, Sentinel, or custom policies
  • Human Review: YAML format is easy to read and review
  • Version Control: Can be safely committed (no binary data)
  • Backend Independent: Works with any backend, including those that don't support binary planfiles
  • Audit Trail: Creates reviewable record of planned changes

Comparing Planfiles with atmos terraform plan-diff​

Compare any two planfiles to understand differences:

# Compare two JSON planfiles
atmos terraform plan-diff --file plan1.json --file2 plan2.json

# Compare different formats
atmos terraform plan-diff --file plan1.yaml --file2 plan2.json --format yaml

# Compare binary planfile with JSON (after converting)
atmos terraform generate planfile vpc -s dev --file-template current.json
atmos terraform plan-diff --file baseline.json --file2 current.json

This performs a semantic comparison of the plan data structures, not a naive text diff.

components/ └── terraform/ └── vpc/ ├── main.tf ├── variables.tf ├── outputs.tf └── dev-vpc.planfile # Generated planfile


## Working with Planfiles

### Basic Plan and Apply Workflow

The most common workflow is to generate a plan, review it, and then apply it:

```bash
# Step 1: Generate a plan (automatically creates a planfile)
$ atmos terraform plan vpc -s dev

# Review the plan output...

# Step 2: Apply the exact plan that was generated
$ atmos terraform apply vpc -s dev --from-plan

Custom Planfile Locations​

You can specify custom locations for planfiles using the standard Terraform -out flag:

# Generate plan to a custom location
$ atmos terraform plan vpc -s dev -out=/tmp/my-vpc-plan.tfplan

# Apply the custom planfile
$ atmos terraform apply vpc -s dev --planfile /tmp/my-vpc-plan.tfplan

Skipping Planfile Generation​

In some cases, you may want to run a plan without generating a planfile:

# Quick plan check without saving
$ atmos terraform plan vpc -s dev --skip-planfile

# Or configure globally in atmos.yaml
atmos.yaml
components:
  terraform:
    plan:
      skip_planfile: true  # Disable planfile generation globally

Planfile Flags and Options​

Plan Command Flags​

FlagDescriptionExample
-out=FILESave plan to specific fileatmos terraform plan vpc -s dev -out=my.planfile
--skip-planfileSkip planfile generationatmos terraform plan vpc -s dev --skip-planfile

Apply Command Flags​

FlagDescriptionExample
--from-planUse Atmos-generated planfileatmos terraform apply vpc -s dev --from-plan
--planfile FILEUse specific planfileatmos terraform apply vpc -s dev --planfile my.planfile

GitLab CI Example​

GitLab-Managed Terraform State

If using GitLab's built-in Terraform state management, planfiles are not supported. GitLab manages the complete plan/apply workflow internally. Use GitLab's native integration instead of Atmos planfiles.

For standard backends (S3, GCS, Azure) with GitLab CI:

.gitlab-ci.yml
stages:
  - plan
  - apply

variables:
  # Important: Use consistent credentials across jobs
  TF_VAR_token: ${CI_JOB_TOKEN}  # Same token for all jobs

plan:
  stage: plan
  script:
    - atmos terraform plan vpc -s prod
  artifacts:
    paths:
      - components/terraform/vpc/prod-vpc.planfile
    expire_in: 1 day

apply:
  stage: apply
  dependencies:
    - plan
  script:
    - atmos terraform apply vpc -s prod --from-plan
  only:
    - main
  when: manual  # Require manual approval

Terraform Cloud Integration​

Terraform Cloud doesn't support the -out flag for planfiles. When using Terraform Cloud:

Configuration​

atmos.yaml
components:
  terraform:
    plan:
      skip_planfile: true  # Required for Terraform Cloud

Environment Variable​

export ATMOS_COMPONENTS_TERRAFORM_PLAN_SKIP_PLANFILE=true

Command Line​

atmos terraform plan vpc -s dev --skip-planfile

Advanced Use Cases​

Plan Comparison​

Compare two different plans using atmos terraform plan-diff:

# Generate first plan
$ atmos terraform plan vpc -s dev -out=plan1.tfplan

# Make some configuration changes...

# Generate second plan
$ atmos terraform plan vpc -s dev -out=plan2.tfplan

# Compare the plans
$ atmos terraform plan-diff vpc -s dev --orig=plan1.tfplan --new=plan2.tfplan

JSON/YAML Planfiles​

Generate planfiles in JSON or YAML format for integration with other tools:

# Generate JSON planfile
$ atmos terraform generate planfile vpc -s dev --format=json

# Generate YAML planfile
$ atmos terraform generate planfile vpc -s dev --format=yaml

# Use with security scanning tools
$ atmos terraform generate planfile vpc -s dev --format=json
$ checkov --file dev-vpc.planfile.json --framework terraform_plan

Multi-Component Workflows​

Apply plans for multiple components in sequence:

#!/bin/bash
# generate-and-apply-all.sh

components=("vpc" "database" "kubernetes" "application")
stack="prod"

# Generate all plans
for component in "${components[@]}"; do
  echo "Planning $component..."
  atmos terraform plan "$component" -s "$stack"
done

# Review and apply
read -p "Apply all plans? (y/n) " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]; then
  for component in "${components[@]}"; do
    echo "Applying $component..."
    atmos terraform apply "$component" -s "$stack" --from-plan
  done
fi

Example 1: Multi-Environment Deployment​

Deploy the same configuration across multiple environments with confidence:

#!/bin/bash
# deploy-all-environments.sh

environments=("dev" "staging" "prod")
component="api-service"

# Generate all plans first
for env in "${environments[@]}"; do
  echo "Planning $component in $env..."
  atmos terraform plan "$component" -s "$env" \
    -out="${env}-${component}-$(date +%Y%m%d-%H%M%S).planfile"
done

# Review all plans
echo "Please review all generated plans..."
ls -la *.planfile

# Apply after approval
read -p "Apply all plans? (yes/no) " -r
if [[ $REPLY == "yes" ]]; then
  for env in "${environments[@]}"; do
    planfile=$(ls ${env}-${component}-*.planfile | head -n1)
    echo "Applying $planfile..."
    atmos terraform apply "$component" -s "$env" --planfile "$planfile"
  done
fi

Example 2: Blue-Green Deployment​

Use planfiles to ensure both environments are updated identically:

# Step 1: Plan both blue and green environments
atmos terraform plan app -s prod-blue -out=blue.planfile
atmos terraform plan app -s prod-green -out=green.planfile

# Step 2: Deploy to green (inactive) first
atmos terraform apply app -s prod-green --planfile green.planfile

# Step 3: Switch traffic to green
atmos terraform apply load-balancer -s prod --var="active_env=green"

# Step 4: Update blue (now inactive)
atmos terraform apply app -s prod-blue --planfile blue.planfile

Example 3: Compliance-Required Workflow​

For regulated environments requiring approval and audit trails:

# Developer generates plan
atmos terraform plan database -s prod -out=change-request-12345.planfile

# Convert to JSON for compliance scanning
atmos terraform generate planfile database -s prod \
  --planfile change-request-12345.planfile \
  --format=json \
  --file=change-request-12345.json

# Run compliance checks
checkov --file change-request-12345.json --framework terraform_plan
opa eval -d /policies -i change-request-12345.json "data.terraform.deny[msg]"

# After approval, operations team applies
atmos terraform apply database -s prod --planfile change-request-12345.planfile

# Archive for audit
aws s3 cp change-request-12345.planfile \
  s3://audit-bucket/terraform/$(date +%Y/%m/%d)/change-request-12345.planfile

Best Practices​

When to Use Planfiles​

Use planfiles when:

  • You need guaranteed consistency between plan and apply
  • Multiple team members review changes before applying
  • Compliance requires an audit trail of approved changes
  • You want to prevent drift between planning and execution
  • Running in automated pipelines where plans need approval

Skip planfiles when:

  • Working in development environments with rapid iteration
  • Using backends that don't support them (Remote, Terraform Cloud)
  • Backend credentials would be embedded (HTTP backend)
  • Real-time infrastructure state is critical

Security Considerations​

  1. Never commit binary planfiles to version control - They may contain sensitive data
  2. Use environment variables for backend authentication - Avoid -backend-config with sensitive values
  3. Consider JSON planfiles for sensitive environments - Use atmos terraform generate planfile for reviewable, scannable output
  4. Set appropriate file permissions - Planfiles should be readable only by authorized users
  5. Clean up old planfiles - Remove after applying or when no longer needed

Choosing the Right Approach​

ScenarioRecommended ApproachCommand
Production deployment with approvalBinary planfileatmos terraform plan vpc -s prod
Security scanning requiredJSON planfileatmos terraform generate planfile vpc -s prod --file-template plan.json
Using HTTP backendSkip planfilesatmos terraform plan vpc -s prod --skip-planfile
Rapid development iterationSkip planfilesatmos terraform deploy vpc -s dev
Compliance audit requiredJSON planfile + version controlatmos terraform generate planfile with Git

Security Considerations​

  1. Never commit binary planfiles to version control - They may contain sensitive data
  2. Use environment variables for backend authentication - Avoid -backend-config with sensitive values
  3. Consider JSON planfiles for sensitive environments - Use atmos terraform generate planfile for reviewable, scannable output
  4. Set appropriate file permissions - Planfiles should be readable only by authorized users
  5. Clean up old planfiles - Remove after applying or when no longer needed

Choosing the Right Approach​

ScenarioRecommended ApproachCommand
Production deployment with approvalBinary planfileatmos terraform plan vpc -s prod
Security scanning requiredJSON planfileatmos terraform generate planfile vpc -s prod --file-template plan.json
Using HTTP backendSkip planfilesatmos terraform plan vpc -s prod --skip-planfile
Rapid development iterationSkip planfilesatmos terraform deploy vpc -s dev
Compliance audit requiredJSON planfile + version controlatmos terraform generate planfile with Git

1. Always Review Plans​

Never apply infrastructure changes without reviewing the plan first:

# Good: Plan, review, then apply
atmos terraform plan vpc -s prod
# ... review the output ...
atmos terraform apply vpc -s prod --from-plan

# Avoid: Direct apply without review (except in dev)
atmos terraform apply vpc -s prod -auto-approve

3. Clean Up Old Planfiles​

Planfiles can contain sensitive information. Clean them up regularly:

# Clean component directory (removes planfiles and varfiles)
atmos terraform clean vpc -s dev

# Or manually remove old planfiles
find components/terraform -name "*.planfile" -mtime +7 -delete

4. Secure Planfile Storage​

When storing planfiles in CI/CD:

  • Encrypt planfiles at rest and in transit
  • Set appropriate retention periods (typically 1-7 days)
  • Restrict access to planfiles containing sensitive data
  • Never commit planfiles to version control

5. Validate Plans Before Applying​

Use tools to validate planfiles before applying:

# Generate planfile in JSON format
atmos terraform generate planfile vpc -s prod --format=json

# Validate with Checkov
checkov --file prod-vpc.planfile.json --framework terraform_plan

# Validate with OPA
opa eval -d policies/ -i prod-vpc.planfile.json "data.terraform.deny[msg]"

Troubleshooting​

Common Issues​

Planfile Not Found​

Error: Planfile not found at expected location

Solution: Ensure you've run atmos terraform plan first, or check the planfile path.

Stale Planfile​

Error: Saved plan is stale; either terraform.tfstate was modified, or the plan file is out of date

Solution: Regenerate the plan with atmos terraform plan.

Terraform Cloud Compatibility​

Error: Run variables are currently not supported

Solution: Use --skip-planfile flag or configure skip_planfile: true in atmos.yaml.

Debugging​

Enable debug logging to see planfile operations:

# Enable debug logs
export ATMOS_LOGS_LEVEL=Debug

# Run commands
atmos terraform plan vpc -s dev
atmos terraform apply vpc -s dev --from-plan