Skip to main content

Using External Stores

The !store YAML function reads data from external key-value stores like AWS SSM Parameter Store, HashiCorp Vault, Artifactory, Redis, and other backends. Use this when you need to share data that isn't stored in Terraform state.

When to Use Stores vs Terraform State
  • Use !terraform.state for reading Terraform outputs (recommended)
  • Use !store for reading from external systems like SSM, Vault, or other key-value stores

You will learn

  • Read from external key-value stores (SSM, Vault, Redis, etc.)
  • Configured in atmos.yaml with named store definitions
  • Supports default values for missing keys
  • Works with YQ expressions for complex data

When to Use Stores​

Use !store when you need to:

  • Read secrets from HashiCorp Vault or AWS Secrets Manager
  • Access configuration stored in SSM Parameter Store
  • Share data that isn't managed by Terraform
  • Integrate with external configuration management systems

For Terraform outputs, use !terraform.state instead.

Configuring Stores​

Define stores in your atmos.yaml:

atmos.yaml

stores:
  # AWS SSM Parameter Store
  ssm/prod:
    backend: aws/ssm
    config:
      region: us-east-1

  # HashiCorp Vault
  vault/secrets:
    backend: vault
    config:
      address: https://vault.example.com
      auth_method: aws

  # Redis
  redis/cache:
    backend: redis
    config:
      address: redis.example.com:6379

Basic Usage​

Read values from a configured store:

stacks/prod.yaml

components:
  terraform:
    app:
      vars:
        # Read from SSM Parameter Store
        api_key: !store ssm/prod app api_key

        # Read from Vault
        db_password: !store vault/secrets database password

Syntax​

# Get key from store for component in current stack
!store <store_name> <component> <key>

# Get key from store for component in a different stack
!store <store_name> <stack> <component> <key>

# With default value
!store <store_name> <component> <key> | default <default-value>

# With YQ query
!store <store_name> <component> <key> | query <yq-expression>

Cross-Stack References​

Read from stores for components in different stacks:

stacks/prod.yaml

components:
  terraform:
    app:
      vars:
        # Read from a component in a different stack
        shared_config: !store ssm/prod shared-services config settings

        # Use template expressions for dynamic stack names
        vpc_id: !store ssm/prod {{ printf "%s-network" .vars.environment }} vpc vpc_id

Default Values​

Provide fallback values for missing keys:

# String default
api_endpoint: !store ssm/prod api endpoint | default "https://api.example.com"

# The default is used if the key doesn't exist in the store
feature_flag: !store ssm/prod config feature_enabled | default false

Working with Complex Data​

Use YQ expressions to extract values from structured data:

# Get a specific field from JSON stored in the store
db_host: !store ssm/prod database config | query .host

# Get first item from a list
primary_endpoint: !store ssm/prod cluster endpoints | query .[0]

# Navigate nested structures
api_key: !store vault/secrets app credentials | query .api.production.key

Supported Store Backends​

aws/ssm
AWS Systems Manager Parameter Store.
aws/secretsmanager
AWS Secrets Manager.
vault
HashiCorp Vault.
redis
Redis key-value store.
artifactory
JFrog Artifactory.

Example: Multi-Store Configuration​

atmos.yaml

stores:
  # Production SSM store
  ssm/prod:
    backend: aws/ssm
    config:
      region: us-east-1
      # Optional: assume role for cross-account access
      assume_role_arn: arn:aws:iam::123456789012:role/ssm-reader

  # Development SSM store
  ssm/dev:
    backend: aws/ssm
    config:
      region: us-west-2

  # Vault for secrets
  vault/secrets:
    backend: vault
    config:
      address: https://vault.example.com
      namespace: infrastructure

stacks/prod.yaml

components:
  terraform:
    app:
      vars:
        # Infrastructure config from SSM
        vpc_id: !store ssm/prod vpc vpc_id
        subnet_ids: !store ssm/prod subnets private_subnet_ids

        # Secrets from Vault
        database_password: !store vault/secrets app db_password
        api_secret: !store vault/secrets app api_secret

Writing to Stores​

Stores are typically populated by Terraform components using provider resources:

components/terraform/vpc/outputs.tf

# Output to Terraform state (for !terraform.state)
output "vpc_id" {
  value = aws_vpc.this.id
}

# Also write to SSM for !store access
resource "aws_ssm_parameter" "vpc_id" {
  name  = "/atmos/${var.stack}/vpc/vpc_id"
  type  = "String"
  value = aws_vpc.this.id
}

Considerations​

  • Secrets exposure: Store values may appear in stdout when describing stacks
  • Permissions: You need read access to all referenced stores
  • Cold starts: Returns error if the key doesn't exist (use | default to handle)
  • Latency: External store calls add latency compared to !terraform.state
  • Consistency: Ensure store values are updated when Terraform state changes

Use !terraform.state for Terraform Outputs

If you're reading Terraform outputs, !terraform.state is faster and doesn't require separate store configuration. Learn More