Skip to main content

Share Data Between Components

Breaking apart your infrastructure into loosely coupled components is a great way to manage complexity and reuse code. However, sometimes, you need to share data between components. In Atmos, you can easily share settings, configurations, and outputs between components and even tap into external data sources.

There are two ways to approach this: using native Terraform support for remote state to read outputs from other components or using template functions in stack configurations. In this chapter, you’ll learn how to share state between components within the same stack or even across different stacks.

You will learn

  • Why you might need to share data between components
  • How to share data between components using Terraform remote state
  • How to use template functions to share data between components in stack configurations

Using Template Functions​

Function: atmos.Component​

The atmos.Component template function can read all configurations of any Atmos component, including its outputs.

For example, we can read the vpc_id output of the vpc component in the current .stack, simply by doing:

components:
  terraform:
    cluster:
      vars:
        vpc_id: '{{ (atmos.Component "vpc" .stack).outputs.vpc_id }}'

The atmos.Component function returns the entire configuration of the component in the stack. The configuration is a map of all the sections of the component, including its outputs. You can access properties using dot (.) notation, and chain any number of attributes with dot (.) notation.

To access the configuration of a component in a different stack, you can specify the stack name as the second argument. For example, here we're reading the vpc_id output of the vpc component in the staging stack:

components:
  terraform:
    cluster:
      vars:
        vpc_id: '{{ (atmos.Component "vpc" "staging").outputs.vpc_id }}'

Ready to learn this topic?

For more advanced examples, check out the atmos.Component function documentation. Learn More

Data Sources​

Data sources are incredibly powerful. They let you glue together components leveraging external data sources without modifying a line of Terraform code. This is great when you want to leave your Terraform codebase untouched, especially if you don't control the source.

Data sources allow you to fetch and use data from external sources in your stack configurations. You can use data sources to fetch data from APIs, various key/value storage systems, or even local files.

They can be fetched from any of the following schemes supported by Gomplate:

  • AWS Systems Manager Parameter Store (aws+smp://)
  • AWS Secrets Manager (aws+sm://)
  • Amazon S3 (s3://)
  • HashiCorp Consul (consul://, consul+http://, consul+https://)
  • Environment Variables (env://)
  • Files (file://)
  • Git Repositories (git://, git+file://, git+http://, git+https://, git+ssh://)
  • Google Cloud Storage (gs://)
  • HTTP/HTTPS Endpoints (http://, https://)
  • Merging Data Sources (merge://)
  • Standard Input (stdin://)
  • HashiCorp Vault (vault://, vault+http://, vault+https://)
On-the-Fly Root Modules

When you combine data sources with vendoring, terraform backends and provider generation, you can leverage any Terraform module as a "root module" and provision it as a component with Atmos.

Configure your data sources in atmos.yaml, then leverage them inside stack configurations.

Here we set up a data source called ip, which will fetch the public IP address by hitting the https://api.ipify.org?format=json endpoint.

atmos.yaml

settings:
  templates:
    settings:
      gomplate:
        timeout: 5
        datasources:
          network_egress:
            url: "https://api.ipify.org?format=json"
            headers:
              accept:
                - "application/json"

Then, you can use the network_egress data source in your stack configurations to fetch the public ip. This is useful for setting a tag indicating the IP address that provisioned the resources.

NOTE:
This assumes the Terraform component accepts a tags variable and appropriately handles tags.

stack.yaml

terraform:
  vars:
    tags:
      provisioned_by_ip: '{{ (datasource "ip").ip }}'

Ready to learn this topic?

Use data sources to fetch data from external sources and use it in your Terraform configurations. Learn More

Using Remote State​

Atmos provides a remote-state Terraform module that makes it easier to look up the remote state of other components in the stack. This module can be used to share data between components provisioned in the same stack or across different stacks, using native HCL.

Our convention is to place all remote-state dependencies in the remote-state.tf file. This file is responsible for fetching the remote state outputs of other components in the stack.

components/terraform/myapp/remote-state.tf

module "vpc" {
  source  = "cloudposse/stack-config/yaml//modules/remote-state"
  version = "1.5.0"

  # Specify the Atmos component name (defined in YAML stack config files) for which to get the remote state outputs
  component = "vpc"

  # `context` input is a way to provide the information about the stack (using the context
  # variables `namespace`, `tenant`, `environment`, `stage` defined in the stack config)
  context = module.this.context
}

Then we can use the module.vpc as easily as if it were provisioned within the myapp component.

This gives us the best of both worlds: the ease of use of Terraform remote state and the reduced blast radius of using smaller components.

components/terraform/myapp/main.tf

resource "aws_network_acl" "default" {

  vpc_id = module.vpc.vpc_id

  ingress {
    protocol   = "tcp"
    rule_no    = 100
    action     = "allow"
    cidr_block = "0.0.0.0/0"
    from_port  = 80
    to_port    = 80
  }
}

Ready to learn this topic?

Use the Terraform-native remote-state module to share data between components. Learn How