Skip to main content

!terraform.state

The !terraform.state YAML function allows reading the outputs (remote state) of components in Atmos stack manifests directly from the configured Terraform/OpenTofu backends.

warning

Currently, the !terraform.state YAML function supports the following backend types:

As support for new backend types is added (e.g. azurerm, gcs), this document will be updated accordingly.

Meanwhile, if you are using backends other than local and s3, consider the !store or !terraform.output YAML function to read remote state and share data between components.

Usage​

The !terraform.state function can be called with either two or three parameters:

  # Get the `output` of the `component` in the current stack
  !terraform.state <component> <output>

  # Get the `output` of the `component` in the provided `stack`
  !terraform.state <component> <stack> <output>

  # Get the output of the `component` by evaluating the YQ expression
  !terraform.state <component> <yq-expression>

 # Get the output of the `component` in the provided `stack` by evaluating the YQ expression
  !terraform.state <component> <stack> <yq-expression>

Arguments​

component
Atmos component name
stack
(Optional) Atmos stack name
output or yq-expression
Terraform output or YQ expression to evaluate the output
tip

You can use Atmos Stack Manifest Templating in the !terraform.state YAML function expressions. Atmos processes the templates first, and then executes the !terraform.state function, allowing you to provide the parameters to the function dynamically.

!terraform.state Function Execution Flow​

When processing the !terraform.state YAML function for a component in a stack, Atmos executes the following steps:

  • Stack and Component Context Resolution Atmos resolves the full context for the specified component within the given stack, including all inherited and merged configuration layers (globals, environment and component-level config).

  • Terraform State Backend Lookup and Read Based on the resolved context, Atmos identifies the corresponding backend for the component in the stack and reads the state file directly from the backend. If access to the backend requires a role assumption (e.g. assume_role.role_arn for the s3 backend), Atmos assumes the role before accessing the backend state file.

  • Output Parsing and Interpolation The relevant output variable is extracted from the state file (using a YQ parser). Atmos parses and interpolates the value into the final configuration structure, replacing the !terraform.state directive in the YAML stack manifest with the final value.

tip

The !terraform.state function accepts the same parameters and produces the same result as the !terraform.output function, but has significantly less impact on performance as it reads the state file directly from the configured backend without executing Terraform/OpenTofu commands, generating varfiles and backend config files, and initializing all modules and providers.

To understand the performance implications of the !terraform.output and !terraform.state functions, compare the !terraform.output Execution Flow with the !terraform.state Execution Flow.

Using YQ Expressions to retrieve items from complex output types​

To retrieve items from complex output types such as maps and lists, or do any kind of filtering or querying, you can utilize YQ expressions.

For example:

  • Retrieve the first item from a list
subnet_id1: !terraform.state vpc .private_subnet_ids[0]
  • Read a key from a map
username: !terraform.state config .config_map.username

For more details, review the following docs:

Using YQ Expressions to provide a default value​

If the component for which you are reading the output has not been provisioned yet, the !terraform.state function will return null, unless you specify a default value in the YQ expression, in which case the function will return the default value.

This will allow you to mock outputs when executing atmos terraform plan where there are dependencies between components, and the dependent components are not provisioned yet.

note

To provide a default value, you use the // YQ operator. The whole YQ expression contains spaces, and to make it a single parameter, you need to double-quote it.

YQ requires the strings in the default values to be double-quoted as well. This means that you have to escape the double-quotes in the default values by using two double-quotes.

For example:

  • Specify a string default value. Read the username output from the config component in the current stack. If the config component has not been provisioned yet, return the default value default-user
username: !terraform.state config ".username // ""default-user"""
  • Specify a list default value. Read the private_subnet_ids output from the vpc component in the current stack. If the vpc component has not been provisioned yet, return the default value ["mock-subnet1", "mock-subnet2"]
subnet_ids: !terraform.state vpc ".private_subnet_ids // [""mock-subnet1"", ""mock-subnet2""]"
  • Specify a map default value. Read the config_map output from the config component in the current stack. If the config component has not been provisioned yet, return the default value {"api_endpoint": "localhost:3000", "user": "test"}
config_map: !terraform.state 'config ".config_map // {""api_endpoint"": ""localhost:3000"", ""user"": ""test""}"'

For more details, review the following docs:

Examples​

stack.yaml

components:
  terraform:
    my_lambda_component:
      vars:
        vpc_config:
          # Output of type string
          security_group_id: !terraform.state security-group/lambda id
          security_group_id2: !terraform.state security-group/lambda2 {{ .stack }} id
          security_group_id3: !terraform.state security-group/lambda3 {{ .atmos_stack }} id
          # Output of type list
          subnet_ids: !terraform.state vpc private_subnet_ids
          # Use a YQ expression to get an item from the list
          subnet_id1: !terraform.state vpc .private_subnet_ids[0]
          # Output of type map
          config_map: !terraform.state config {{ .stack }} config_map
          # Use a YQ expression to get a value from the map
          username: !terraform.state config .config_map.username

Specifying Atmos stack​

If you call the !terraform.state function with three parameters, you need to specify the stack as the second argument.

There are multiple ways you can specify the Atmos stack parameter in the !terraform.state function.

Hardcoded Stack Name​

Use it if you want to get an output from a component from a different (well-known and static) stack. For example, you have a tgw component in a stack plat-ue2-dev that requires the vpc_id output from the vpc component from the stack plat-ue2-prod:

plat-ue2-dev
  components:
    terraform:
      tgw:
        vars:
          vpc_id: !terraform.state vpc plat-ue2-prod vpc_id

Reference the Current Stack Name​

Use the .stack (or .atmos_stack) template identifier to specify the same stack as the current component is in (for which the !terraform.state function is executed):

  !terraform.state <component> {{ .stack }} <output>
  !terraform.state <component> {{ .atmos_stack }} <output>

For example, you have a tgw component that requires the vpc_id output from the vpc component in the same stack:

  components:
    terraform:
      tgw:
        vars:
          vpc_id: !terraform.state vpc {{ .stack }} vpc_id
note

Using the .stack or .atmos_stack template identifiers to specify the stack is the same as calling the !terraform.state function with two parameters without specifying the current stack, but without using Go templates. If you need to get an output of a component in the current stack, using the !terraform.state function with two parameters is preferred because it has a simpler syntax and executes faster.

Use a Format Function​

Use the printf template function to construct stack names using static strings and dynamic identifiers. This is convenient when you want to override some identifiers in the stack name:

  !terraform.state <component> {{ printf "%s-%s-%s" .vars.tenant .vars.environment .vars.stage }} <output>

  !terraform.state <component> {{ printf "plat-%s-prod" .vars.environment }} <output>

  !terraform.state <component> {{ printf "%s-%s-%s" .settings.context.tenant .settings.context.region .settings.context.account }} <output>
<component>
Placeholder for an actual component name (e.g. vpc)
<output>
Placeholder for an actual Terraform output (e.g. subnet_ids)

For example, you have a tgw component deployed in the stack plat-ue2-dev. The tgw component requires the vpc_id output from the vpc component from the same environment (ue2) and same stage (dev), but from a different tenant net (instead of plat):

plat-ue2-dev
  components:
    terraform:
      tgw:
        vars:
          vpc_id: !terraform.state vpc {{ printf "net-%s-%s" .vars.environment .vars.stage }} vpc_id
Important

By using the printf "%s-%s-%s" function, you are constructing stack names using the stack context variables/identifiers.

For more information on Atmos stack names and how to define them, refer to stacks.name_pattern and stacks.name_template sections in atmos.yaml CLI config file

Caching the result of !terraform.state function​

Atmos caches (in memory) the results of !terraform.state function.

The cache is per Atmos CLI command execution, e.g., each new execution of a command like atmos terraform plan, atmos terraform apply or atmos describe component will create and use a new memory cache, which involves re-reading the remote state after reinitialisation.

If you define the function in stack manifests for the same component in a stack more than once, the first call will produce the result and cache it, and all the consecutive calls will just use the cached data. This is useful when you use the !terraform.state function for the same component in a stack in multiple places in Atmos stack manifests. It will speed up the function execution and stack processing.

For example:

components:
  terraform:
    test2:
      vars:
        tags:
          test: !terraform.state test id
          test2: !terraform.state test id
          test3: !terraform.state test {{ .stack }} id

In the example, the test2 Atmos component uses the outputs (remote state) of the test Atmos component from the same stack. The YAML function !terraform.state is executed three times (once for each tag).

After the first execution, Atmos caches the result in memory, and reuses it in the next two calls to the function. The caching makes the stack processing much faster. In a production environment where many components are used, the speedup can be significant.

Using !terraform.state with static remote state backend​

Atmos supports brownfield configuration by using the remote state of type static.

For example:

stack.yaml

components:
  terraform:
    # Component `static-backend` is configured with the remote state backend of type `static`
    static-backend:
      remote_state_backend_type: static
      remote_state_backend:
        static:
          region: "us-west-2"
          cluster_name: "production-cluster"
          vpc_cidr: "10.0.0.0/16"
          database:
            type: "postgresql"
            version: "12.7"
            storage_gb: 100
          allowed_ips:
            - "192.168.1.0/24"
            - "10.1.0.0/16"
          tags:
            Environment: "production"
            Owner: "infra-team"

    eks-cluster:
      vars:
        region: !terraform.state static-backend region
        cluster_name: !terraform.state static-backend cluster_name
        vpc_cidr: !terraform.state static-backend vpc_cidr
        db_type: !terraform.state static-backend database.type
        db_storage: !terraform.state static-backend database.storage_gb
        allowed_ips: !terraform.state static-backend allowed_ips
        tags: !terraform.state static-backend tags

When the functions are executed, Atmos detects that the static-backend component has the static remote state configured, and instead of executing terraform output, it just returns the static values from the remote_state_backend.static section.

Executing the command atmos describe component eks-cluster -s <stack> produces the following result:

atmos describe component eks-cluster -s <stack>

vars:
  region: us-west-2
  cluster_name: production-cluster
  vpc_cidr: 10.0.0.0/16
  db_type: postgresql
  db_storage: 100
  allowed_ips:
    - 192.168.1.0/24
    - 10.1.0.0/16
  tags:
    Environment: production
    Owner: infra-team

Considerations​

  • Using !terraform.state with secrets can expose sensitive data to standard output (stdout) in any commands that describe stacks or components.

  • When using !terraform.state with atmos describe affected, Atmos requires access to all referenced remote states. If you operate with limited permissions (e.g., scoped to dev) and reference production stacks, the command will fail.

  • Overusing the function within stacks to reference multiple components can impact performance.

  • Be mindful of disaster recovery (DR) implications when using it across regions.

  • Consider cold-start scenarios: if the dependent component has not yet been provisioned, terraform output will fail.