Skip to main content

Stack Manifest Templating

Advanced

Use templates as an escape hatch, when standard inheritance as insufficient. Atmos supports Go templates in stack manifests and functions to customize Stack configurations prior to interpreting them as YAML.

Enable Templating​

Templating in Atmos stack manifests is configured in the atmos.yaml CLI config file in the templates.settings section.

templates.settings
In the templates.settings section in atmos.yaml CLI config file
settings.templates.settings

In the settings.templates.settings section in Atmos stack manifests. The settings.templates.settings section can be defined globally per organization, tenant, account, or per component. Atmos deep-merges the configurations from all scopes into the final result using inheritance.

templates.settings.enabled
A boolean flag to enable/disable the processing of Go templates in Atmos stack manifests. If set to false, Atmos will not process Go templates in stack manifests.

Configure Templating​

templates.settings.env
A map of environment variables to use when executing the templates.
templates.settings.evaluations
Number of evaluations/passes to process Go templates. If not defined, evaluations is automatically set to 1. For more details, refer to Template Evaluations and Template Processing Pipelines.
templates.settings.delimiters
A list of left and right delimiters to use to process the templates. If not defined, the default Go template delimiters ["{{", "}}"] will be used.
templates.settings.sprig.enabled
A boolean flag to enable/disable the Sprig Functions in Atmos stack manifests.
templates.settings.gomplate.enabled
A boolean flag to enable/disable the Gomplate Functions and Gomplate Datasources in Atmos stack manifests.
templates.settings.gomplate.timeout
Timeout in seconds to execute Gomplate Datasources.
warning

Some functions are present in both Sprig and Gomplate.

For example, the env function has the same name in Sprig and Gomplate, but has different syntax and accept different number of arguments.

If you use the env function from one templating engine and enable both Sprig and Gomplate, it will be invalid in the other templating engine, and an error will be thrown.

To be able to use the env function from both templating engines, you can do one of the following:

  • Use the env function from one templating engine, and disable the other templating engine by using the templates.settings.sprig.enabled and templates.settings,gomplate.enabled settings

  • Enable both engines and use the Gomplate's env function via its getenv alias

Example Configuration​

atmos.yaml

# https://pkg.go.dev/text/template
templates:
  settings:
    # Enable `Go` templates in Atmos stack manifests
    enabled: true
    # Number of evaluations/passes to process `Go` templates
    # If not defined, `evaluations` is automatically set to `1`
    evaluations: 2
    # Optional template delimiters
    # The `{{ }}` delimiters are the default, no need to specify/redefine them
    delimiters: ["{{", "}}"]
    # Environment variables passed to data sources when evaluating templates
    # https://docs.gomplate.ca/datasources/#using-awssmp-datasources
    # https://docs.gomplate.ca/functions/aws/#configuring-aws
    # https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html
    env:
      AWS_PROFILE: "<AWS profile>"
      AWS_TIMEOUT: 2000
    # https://masterminds.github.io/sprig
    sprig:
      # Enable Sprig functions in `Go` templates in Atmos stack manifests
      enabled: true
    # https://docs.gomplate.ca
    # https://docs.gomplate.ca/functions
    gomplate:
      # Enable Gomplate functions and data sources in `Go` templates in Atmos stack manifests
      enabled: true
      # Timeout in seconds to execute the data sources
      timeout: 5
      datasources: {}

Functions and Data Sources​

Go templates by themselves are pretty basic, supporting concepts like ranges and variable interpolations. But what really makes templating powerful is the library of functions provided by Atmos to the template engine.

In Go templates, you can use the following functions and data sources:

Ready to learn this topic?

Functions are a crucial part of templating in Atmos stack manifests. They allow you to manipulate data and perform operations on the data to customize the stack configurations. Learn About Functions

Configuring Templating in Atmos Stack Manifests​

Templating in Atmos can also be configured in the settings.templates.settings section in stack manifests.

The settings.templates.settings section can be defined globally per organization, tenant, account, or per component. Atmos deep-merges the configurations from all scopes into the final result using inheritance.

The schema is the same as templates.settings in the atmos.yaml CLI config file, except the following settings are not supported in the settings.templates.settings section:

  • settings.templates.settings.enabled
  • settings.templates.settings.sprig.enabled
  • settings.templates.settings.gomplate.enabled
  • settings.templates.settings.evaluations
  • settings.templates.settings.delimiters

These settings are not supported for the following reasons:

  • You can't disable templating in the stack manifests which are being processed by Atmos as Go templates

  • If you define the delimiters in the settings.templates.settings section in stack manifests, the Go templating engine will think that the delimiters specify the beginning and the end of template strings, will try to evaluate it, which will result in an error

As an example, let's define templating configuration for the entire organization in the stacks/orgs/acme/_defaults.yaml stack manifest:

stacks/orgs/acme/_defaults.yaml

settings:
  templates:
    settings:
      # Environment variables passed to data sources when evaluating templates
      # https://docs.gomplate.ca/datasources/#using-awssmp-datasources
      # https://docs.gomplate.ca/functions/aws/#configuring-aws
      # https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html
      env:
        AWS_PROFILE: "<AWS profile>"
        AWS_TIMEOUT: 2000
      gomplate:
        # 7 seconds timeout to execute the data sources
        timeout: 7
        # https://docs.gomplate.ca/datasources
        datasources:
          # 'file' data sources
          # https://docs.gomplate.ca/datasources/#using-file-datasources
          config-1:
            url: "./my-config1.json"
          config-3:
            url: "file:///config3.json"

Atmos deep-merges the configurations from the settings.templates.settings section in Atmos stack manifests with the templates.settings section in atmos.yaml CLI config file using inheritance.

The settings.templates.settings section in Atmos stack manifests takes precedence over the templates.settings section in atmos.yaml CLI config file, allowing you to define the global datasources in atmos.yaml and then add or override datasources in Atmos stack manifests for the entire organization, tenant, account, or per component.

For example, taking into account the configurations described above in atmos.yaml CLI config file and in the stacks/orgs/acme/_defaults.yaml stack manifest, the final datasources map will look like this:

stacks/orgs/acme/_defaults.yaml

gomplate:
  timeout: 7
  datasources:
    ip:
      url: "https://api.ipify.org?format=json"
      headers:
        accept:
          - "application/json"
    random:
      url: "http://www.randomnumberapi.com/api/v1.0/randomstring?min=${ .settings.random.min }&max=${ .settings.random.max }&count=1"
    secret-1:
      url: "aws+smp:///path/to/secret"
    secret-2:
      url: "aws+sm:///path/to/secret"
    s3-config:
      url: "s3://mybucket/config/config.json"
    config-1:
      url: "./my-config1.json"
    config-2:
      url: "file:///config2.json"
    config-3:
      url: "file:///config3.json"

Note that the config-1 datasource from atmos.yaml was overridden with the config-1 datasource from the stacks/orgs/acme/_defaults.yaml stack manifest. The timeout attribute was overridden as well.

You can now use the datasources in Go templates in all Atmos sections that support Go templates.

Atmos sections supporting Go templates​

You can use Go templates in the following Atmos sections to refer to values in the same or other sections:

  • vars
  • settings
  • env
  • providers
  • overrides
  • backend
  • backend_type
  • component
  • metadata.component
  • command
tip

In the template tokens, you can refer to any value in any section that the Atmos command atmos describe component <component> -s <stack> generates

For example, let's say we have the following component configuration using Go templates:

stack.yaml

component:
  terraform:
    vpc:
      settings:
        setting1: 1
        setting2: 2
        setting3: "{{ .vars.var3 }}"
        setting4: "{{ .settings.setting1 }}"
        component: vpc
        backend_type: s3
        region: "us-east-2"
        assume_role: "<role-arn>"
      backend_type: "{{ .settings.backend_type }}"
      metadata:
        component: "{{ .settings.component }}"
      providers:
        aws:
          region: "{{ .settings.region }}"
          assume_role: "{{ .settings.assume_role }}"
      env:
        ENV1: e1
        ENV2: "{{ .settings.setting1 }}-{{ .settings.setting2 }}"
      vars:
        var1: "{{ .settings.setting1 }}"
        var2: "{{ .settings.setting2 }}"
        var3: 3
        # Add the tags to all the resources provisioned by this Atmos component
        tags:
          atmos_component: "{{ .atmos_component }}"
          atmos_stack: "{{ .atmos_stack }}"
          atmos_manifest: "{{ .atmos_stack_file }}"
          region: "{{ .vars.region }}"
          terraform_workspace: "{{ .workspace }}"
          assumed_role: "{{ .providers.aws.assume_role }}"
          description: "{{ .atmos_component }} component provisioned in {{ .atmos_stack }} stack by assuming IAM role {{ .providers.aws.assume_role }}"
          # Examples of using the Sprig and Gomplate functions and datasources
          # https://masterminds.github.io/sprig/os.html
          provisioned_by_user: '{{ env "USER" }}'
          # https://docs.gomplate.ca/functions/strings
          atmos_component_description: "{{ strings.Title .atmos_component }} component {{ .vars.name | strings.Quote }} provisioned in the stack {{ .atmos_stack | strings.Quote }}"
          # https://docs.gomplate.ca/datasources
          provisioned_by_ip: '{{ (datasource "ip").ip }}'
          config1_tag: '{{ (datasource "config-1").tag }}'
          config2_service_name: '{{ (datasource "config-2").service.name }}'
          config3_team_name: '{{ (datasource "config-3").team.name }}'

When executing Atmos commands like atmos describe component and atmos terraform plan/apply, Atmos processes all the template tokens in the manifest and generates the final configuration for the component in the stack:

atmos describe component vpc -s plat-ue2-dev

settings:
  setting1: 1
  setting2: 2
  setting3: 3
  setting4: 1
  component: vpc
  backend_type: s3
  region: us-east-2
  assume_role: <role-arn>
backend_type: s3
metadata:
  component: vpc
providers:
  aws:
    region: us-east-2
    assume_role: <role-arn>
env:
  ENV1: e1
  ENV2: 1-2
vars:
  var1: 1
  var2: 2
  var3: 3
  tags:
    assumed_role: <role-arn>
    atmos_component: vpc
    atmos_component_description: Vpc component "common" provisioned in the stack "plat-ue2-dev"
    atmos_manifest: orgs/acme/plat/dev/us-east-2
    atmos_stack: plat-ue2-dev
    config1_tag: test1
    config2_service_name: service1
    config3_team_name: my-team
    description: vpc component provisioned in plat-ue2-dev stack by assuming IAM role <role-arn>
    provisioned_by_user: <user>
    provisioned_by_ip: 167.38.132.237
    region: us-east-2
    terraform_workspace: plat-ue2-dev

Performance Implications​

There are some performance implications of using Go Templates with Atmos Stack configurations.

Using Go templates and template functions in Atmos stack configurations is generally safe and provides powerful flexibility. However, caution is required when leveraging functions like atmos.Component or others that depend on remote resources or network configurations. These functions can have significant performance implications and potential impacts on availability.

Why the Caution?

Atmos processes stack configuration files in multiple stages: first as Go templates, and then as YAML. During the Go template stage, every template function must be evaluated and resolved before Atmos can load the file. This introduces a critical dependency: Atmos cannot proceed unless all referenced resources are available and accessible.

  1. Performance: Functions like Atmos.Component may require Atmos to retrieve extensive information about other components or outputs that depend on Terraform remote state. This adds latency, especially if used extensively across your stack configurations. In the case of retrieving Terraform outputs, Atmos must initialize the Terraform component which involves downloading all Terraform providers, which is slow. Commands like atmos describe stacks or atmos describe affected, which rely on evaluating all templates, can become noticeably slower as the number of remote calls increases.
  2. Availability Risks: Templated references to remote resources introduce fragility. If a referenced resource becomes unavailable—whether due to downtime, decommissioning, or network issues—Atmos commands that depend on those templates will fail. This can severely impact high availability (HA) scenarios and your ability to reliably deploy or manage infrastructure.

Template Function Best Practices​

Careful management of template dependencies is essential for optimizing the performance of Atmos while ensuring a robust and reliable infrastructure configuration process.

To avoid potential pitfalls and maximize efficiency, follow these best practices:

  1. Minimize Dependency on Remote Sources: Avoid referencing resources in your templates that are not highly available or are prone to downtime. Where possible, use static or locally resolvable values.
  2. Use Atmos.Component Sparingly: While Atmos.Component is powerful, its overuse can significantly degrade performance. Limit its use to scenarios where it is truly necessary, and consider precomputing or caching values to reduce the frequency of evaluations.
  3. Use Terraform Remote State Directly: Instead of relying on template functions to retrieve remote state, use Terraform's native ability to retrieve the remote state of other components.
  4. Test for Resilience: Simulate scenarios where a remote resource becomes unavailable and observe how Atmos behaves. Design your configurations to handle failures gracefully or provide fallbacks where feasible.

Template Evaluations​

Atmos supports many different ways of configuring and using Go templates:

Phases of Template Evaluation​

These templates are processed in different phases and use different context:

  • Go templates in Atmos Custom Commands are processed when the custom commands are executed. The execution context can be specified by using the component_config section. If a custom command defines a component_config section with component and stack, Atmos generates the config for the component in the stack and makes it available in the {{ .ComponentConfig.xxx.yyy.zzz }} template variables, exposing all the component sections that are returned by the atmos describe component <component> -s <stack> CLI command

  • Go templates in Atmos Vendoring and Atmos Component Vendoring are processed when the CLI command atmos vendor pull is executed. The templates in the vendoring manifests support the {{.Version}} variable, and the execution context is provided in the version attribute

  • Go Templates in Imports are used in imported stack manifests to make them DRY and reusable. The context (variables) for the Go templates is provided via the static context section. Atmos processes Go templates in imports as the very first phase of the stack processing pipeline. When executing the CLI commands, Atmos parses and executes the templates using the provided static context, processes all imports, and finds stacks and components

  • Go templates in Atmos stack manifests, on the other hand, are processed as the very last phase of the stack processing pipeline (after all imports are processed, all stack configurations are deep-merged, and the component in the stack is found). For the context (template variables), it uses all the component's attributes returned from the atmos describe component CLI command

These mechanisms, although all using Go templates, serve different purposes, use different contexts, and are executed in different phases of the stack processing pipeline.

For more details, refer to:

Processing Pipelines​

Atmos supports configuring the number of evaluations/passes for template processing in atmos.yaml CLI config file. It effectively allows you to define template processing pipelines.

For example:

atmos.yaml

templates:
  settings:
    # Enable `Go` templates in Atmos stack manifests
    enabled: true
    # Number of evaluations/passes to process `Go` templates
    # If not defined, `evaluations` is automatically set to `1`
    evaluations: 2
  • templates.settings.evaluations - number of evaluations to process Go templates. If not defined, evaluations is automatically set to 1

Template evaluations are useful in the following scenarios:

  • Combining templates from different sections in Atmos stack manifests
  • Using templates in the URLs of datasources

Use-cases​

While Go templates in Atmos stack manifests offer great flexibility for various use-cases, one of the obvious use-cases is to add a standard set of tags to all the resources in the infrastructure.

For example, by adding this configuration to the stacks/orgs/acme/_defaults.yaml Org-level stack manifest:

stacks/orgs/acme/_defaults.yaml
terraform:
  vars:
    tags:
      atmos_component: "{{ .atmos_component }}"
      atmos_stack: "{{ .atmos_stack }}"
      atmos_manifest: "{{ .atmos_stack_file }}"
      terraform_workspace: "{{ .workspace }}"
      # Examples of using the Gomplate and Sprig functions
      # https://docs.gomplate.ca/functions/strings
      atmos_component_description: "{{ strings.Title .atmos_component }} component {{ .vars.name | strings.Quote }} provisioned in the stack {{ .atmos_stack | strings.Quote }}"
      # https://masterminds.github.io/sprig/os.html
      provisioned_by_user: '{{ env "USER" }}'

The tags will be processed and automatically added to all the resources provisioned in the infrastructure.

Excluding Templates in Stack Manifest from Processing by Atmos​

If you need to provide Go templates to external systems (e.g. ArgoCD or Datadog) verbatim and prevent Atmos from processing the templates, use double curly braces + backtick + double curly braces instead of just double curly braces:

{{`{{  instead of  {{

}}`}}  instead of  }}

For example:

components:
  terraform:

    eks/argocd:
      metadata:
        component: "eks/argocd"
      vars:
        enabled: true
        name: "argocd"
        chart_repository: "https://argoproj.github.io/argo-helm"
        chart_version: 5.46.0

        chart_values:
          template-github-commit-status:
            message: |
              Application {{`{{ .app.metadata.name }}`}} is now running new version.
            webhook:
              github-commit-status:
                method: POST
                path: "/repos/{{`{{ call .repo.FullNameByRepoURL .app.metadata.annotations.app_repository }}`}}/statuses/{{`{{ .app.metadata.annotations.app_commit }}`}}"
                body: |
                  {
                    {{`{{ if eq .app.status.operationState.phase "Running" }}`}} "state": "pending"{{`{{end}}`}}
                    {{`{{ if eq .app.status.operationState.phase "Succeeded" }}`}} "state": "success"{{`{{end}}`}}
                    {{`{{ if eq .app.status.operationState.phase "Error" }}`}} "state": "error"{{`{{end}}`}}
                    {{`{{ if eq .app.status.operationState.phase "Failed" }}`}} "state": "error"{{`{{end}}`}},
                    "description": "ArgoCD",
                    "target_url": "{{`{{ .context.argocdUrl }}`}}/applications/{{`{{ .app.metadata.name }}`}}",
                    "context": "continuous-delivery/{{`{{ .app.metadata.name }}`}}"
                  }

When Atmos processes the templates in the manifest shown above, it renders them as raw strings allowing sending the templates to the external system for processing:

chart_values:
  template-github-commit-status:
    message: |
      Application {{ .app.metadata.name }} is now running new version.
    webhook:
      github-commit-status:
        method: POST
        path: "/repos/{{ call .repo.FullNameByRepoURL .app.metadata.annotations.app_repository }}/statuses/{{ .app.metadata.annotations.app_commit }}"
        body: |
          {
            {{ if eq .app.status.operationState.phase "Running" }} "state": "pending"{{end}}
            {{ if eq .app.status.operationState.phase "Succeeded" }} "state": "success"{{end}}
            {{ if eq .app.status.operationState.phase "Error" }} "state": "error"{{end}}
            {{ if eq .app.status.operationState.phase "Failed" }} "state": "error"{{end}},
            "description": "ArgoCD",
            "target_url": "{{ .context.argocdUrl }}/applications/{{ .app.metadata.name }}",
            "context": "continuous-delivery/{{ .app.metadata.name }}"
          }

The printf template function is also supported and can be used instead of double curly braces + backtick + double curly braces.

The following examples produce the same result:

chart_values:
  template-github-commit-status:
    message: >-
      Application {{`{{ .app.metadata.name }}`}} is now running new version.
chart_values:
  template-github-commit-status:
    message: "Application {{`{{ .app.metadata.name }}`}} is now running new version."
chart_values:
  template-github-commit-status:
    message: >-
      {{ printf "Application {{ .app.metadata.name }} is now running new version." }}
chart_values:
  template-github-commit-status:
    message: '{{ printf "Application {{ .app.metadata.name }} is now running new version." }}'

Excluding Templates in Imports​

If you are using Go Templates in Imports and Go templates in stack manifests in the same Atmos manifest, take into account that in this case Atmos will do Go template processing two times (two passes):

  • When importing the manifest and processing the template tokens using the variables from the provided context object
  • After finding the component in the stack as the final step in the processing pipeline

For example, we can define the following configuration in the stacks/catalog/eks/eks_cluster.tmpl template file:

stacks/catalog/eks/eks_cluster.tmpl
components:
  terraform:
    eks/cluster:
      metadata:
        component: eks/cluster
      vars:
        enabled: "{{ .enabled }}"
        name: "{{ .name }}"
        tags:
          atmos_component: "{{ .atmos_component }}"
          atmos_stack: "{{ .atmos_stack }}"
          terraform_workspace: "{{ .workspace }}"

Then we import the template into a top-level stack providing the context variables for the import in the context object:

stacks/orgs/acme/plat/prod/us-east-2.yaml
import:
  - path: "catalog/eks/eks_cluster.tmpl"
    context:
      enabled: true
      name: prod-eks

Atmos will process the import and replace the template tokens using the variables from the context. Since the context does not provide the variables for the template tokens in tags, the following manifest will be generated:

components:
  terraform:
    eks/cluster:
      metadata:
        component: eks/cluster
      vars:
        enabled: true
        name: prod-eks
        tags:
          atmos_component: <no value>
          atmos_stack: <no value>
          terraform_workspace: <no value>

The second pass of template processing will not replace the tokens in tags because they are already processed in the first pass (importing) and the values <no value> are generated.

To deal with this, use double curly braces + backtick + double curly braces instead of just double curly braces in tags to prevent Atmos from processing the templates in the first pass and instead process them in the second pass:

stacks/catalog/eks/eks_cluster.tmpl
components:
  terraform:
    eks/cluster:
      metadata:
        component: eks/cluster
      vars:
        enabled: "{{ .enabled }}"
        name: "{{ .name }}"
        tags:
          atmos_component: "{{`{{ .atmos_component }}`}}"
          atmos_stack: "{{`{{ .atmos_stack }}`}}"
          terraform_workspace: "{{`{{ .workspace }}`}}"

Atmos will first process the import and replace the template tokens using the variables from the context. Then in the second pass the tokens in tags will be replaced with the correct values.

It will generate the following manifest:

components:
  terraform:
    eks/cluster:
      metadata:
        component: eks/cluster
      vars:
        enabled: true
        name: prod-eks
        tags:
          atmos_component: eks/cluster
          atmos_stack: plat-ue2-prod
          terraform_workspace: plat-ue2-prod

Combining templates from different sections in Atmos stack manifests​

You can define more than one step/pass of template processing to use and combine the results from each step.

For example:

atmos.yaml

templates:
  settings:
    enabled: true
    # Number of evaluations to process `Go` templates
    evaluations: 3
settings:
  test: "{{ .atmos_component }}"
  test2: "{{ .settings.test }}"

components:
  terraform:
    vpc:
      vars:
        tags:
          tag1: "{{ .settings.test }}-{{ .settings.test2 }}"
          tag2: "{{\"{{`{{ .atmos_component }}`}}\"}}"

When executing an Atmos command like atmos terraform plan vpc -s <stack>, the above template will be processed in three phases:

  • Evaluation 1

    • settings.test is set to vpc
    • settings.test2 is set to {{ .atmos_component }}
    • vpc.vars.tags.tag1 is set to {{ .atmos_component }}-{{ .settings.test }}
    • vpc.vars.tags.tag2 is set to {{<backtick>{{ .atmos_component }}<backtick>}}

  • Evaluation 2

    • settings.test is vpc
    • settings.test2 is set to vpc
    • vpc.vars.tags.tag1 is set to vpc-vpc
    • vpc.vars.tags.tag2 is set to {{ .atmos_component }}

  • Evaluation 3

    • settings.test is vpc
    • settings.test2 is vpc
    • vpc.vars.tags.tag1 is vpc-vpc
    • vpc.vars.tags.tag2 is set to vpc
warning

The above example shows the supported functionality in Atmos templating. You can use it for some use-cases, but it does not mean that you should use it just for the sake of using, since it's not easy to read and understand what data we have after each evaluation step.

The Using Templates in the URLs of Datasources document describes a practical approach to using evaluation steps in Atmos templates to work with data sources.