Atmos Stacks

When you design cloud architectures with Atmos, you break them apart into pieces called components that you implement with Terraform "root modules". Stacks are how you connect your components with configuration, so that everything comes together.

The power of components comes from their ability to be reused: you can compose stacks with one or more components, even reusing any component multiple times within a stack. But as your stacks grow with more and more components, it often makes sense to start splitting them into different files and that's why you might want to make use of imports. This lets you keep your Stack files easier to scan and reuse their configuration in multiple places.

Stacks define the complete configuration of an environment. Think of Stacks like an architectural "Blueprints" composed of one or more Components configurations and defined using a standardized YAML configuration.

Then by running the atmos command, automate the orchestrate the deployment of loosely coupled components, such as Terraform "root" modules. By doing this, it enables scalable infrastructure-as-code configurations, allowing environments to inherit from one or more common bases (child stacks) by importing configuration that gets deep-merged, thus minimizing config duplication and manual effort. Each stack uses a simple schema that provides a declarative description of your various environments. This approach empowers you to separate your infrastructure’s environment configuration settings from the code it manages (e.g., Terraform components).

By facilitating the infrastructure configurations this way, developers achieve DRY (Don't Repeat Yourself) architectures with minimal configuration. Stacks make infrastructure more streamlined and consistent, significantly enhancing productivity. Best of all, Stacks can deploy vanilla Terraform "root" modules without any code generation, custom vendor extensions, or changes to the HCL code.

Atmos utilizes a custom YAML configuration format for stacks. YAML is ideal because it's portable across multiple toolchains and languages; every developer understands it. The Atmos CLI, the terraform-utils-provider provider, and Spacelift via the terraform-spacelift-cloud-infrastructure-automation module all support stacks. Utilizing the Terraform provider enables native access to the entire infrastructure configuration directly from Terraform.

Ready to learn this topic?

Define your first component configuration using stacks.

Read More

Use-cases

• Rapid Environment Provisioning: Leverage stacks to swiftly set up and replicate development, testing, and production environments, ensuring consistency and reducing manual setup errors. This accelerates the development cycle and enables businesses to respond quickly to market demands or development needs.
• Multi-Tenant Infrastructure Management: Utilize stacks to manage and isolate resources for different clients or projects within a single cloud infrastructure. This approach supports SaaS companies in providing secure, isolated environments for each tenant, optimizing resource utilization and simplifying the management of complex, multi-tenant architectures.
• Compliance and Governance: Implement stacks to enforce compliance and governance policies across all environments systematically. By defining standard configurations that meet regulatory requirements, businesses can ensure that every deployment is compliant, reducing the risk of violations and enhancing security posture.

Conventions

The differentiation between the following two types of stacks is crucial for understanding how to organize stacks and the basis for the various design patterns.

Stack Names (aka "slugs")

Every stack is uniquely identified by a name. The name is used to reference the stack in the Atmos CLI, or with stack dependencies.

These are computed from either the name_pattern (old way) or the more configurable name_template (new way). These are configured in the atmos.yaml configuration file.

For example, using the slug, we can reference a stack like this when applying the vpc stack in the us2-dev environment:

atmos terraform apply vpc -s us2-dev

Components vs Component instances

Components are different from Stacks.

When a component is added to a stack, we call that a "Component Instance"

Parent Stacks vs Child Stacks

Parent Stacks

These are the top-level stacks that are responsible for importing Child stacks. Components inside of Parent stacks are deployable, unlike in Child stacks.

Child Stacks

These are any stacks whose components cannot be deployed independently without being imported by a Parent Stack. Catalogs are typically where we keep our Child stacks.

Logical Stacks vs. Physical Stack Manifests

Logical Stacks

Represent the entire environment defined by context variables and global settings in atmos.yaml. Logical stacks are the in-memory representation of the deep-merged configuration.

Physical Stacks

Are the raw YAML files where the specific configurations of components are defined.

Atmos processes each physical stack file, first evaluating any templates and then processing it as YAML. After loading the YAML, it proceeds to deep-merge the configuration with the current in-memory logical representation of the Stack, then apply any overrides. This is done iteratively for each physical stack file in the order they are defined in the import section of the Stack file.

Note, the logical repersentation is never influenced by file paths or directories. It's only influenced by the configuration itself.

Schema

A Stack file contains a manifest defined in YAML that follows a simple, extensible schema. In fact, every Stack file follows exactly the same schema, and every setting in the configuration is optional. Enforcing a consistent schema ensures we can easily import and deep-merge configurations and use inheritance to achieve DRY configuration.

stack.yaml

# Configurations that should get deep-merged into this one
import:
  # each import is a "Stack" file. The `.yaml` extension is optional, and we do not recommend using it.
  - ue2-globals

# Top-level variables that are inherited by every single component.
# Use these wisely. Too many global vars will pollute the variable namespace.
vars:
  # Variables can be anything you want. They can be scalars, lists, and maps. Whatever is supported by YAML.
  stage: dev

# There can then be global variables for each type of component.
# Here we set global variables for any "terraform" component.
terraform:
  vars: { }

# Here we set global variables for any "helmfile" component.
helmfile:
  vars: { }

# Components are the building blocks of reusable infrastructure.
# They can be anything. Atmos natively supports "terraform" and "helmfile".
components:
  terraform:
    vpc:
      command: "/usr/bin/terraform-0.15"
      backend:
        s3:
          workspace_key_prefix: "vpc"
      vars:
        cidr_block: "10.102.0.0/18"
    eks:
      backend:
        s3:
          workspace_key_prefix: "eks"
      vars: { }

  helmfile:
    nginx-ingress:
      vars:
        installed: true

Stack Attributes

components

The components is the list of all the building blocks.

Example:

components:
  sometool: # "sometool" can be any tool
    somecomponent: # "somecomponent" can be the name of any "sometool" component
      vars: # etc...

So for terraform, it might look something like this:

components:
  terraform:
    vpc:
      vars: # etc...

Stack Files

Stack files can be very numerous in large cloud environments (think many dozens to hundreds of stack files). To enable the proper organization of stack files, SweetOps has established some conventions that are good to follow. However, these are just conventions, and there are no limits enforced by the tool.

By convention, we recommend storing all Stacks in a stacks/ folder at the root of your infrastructure repository. This way it's clear where they live and helps keep the configuration separate from your code (e.g. HCL).

The filename of individual environment stacks can follow any convention, and the best one will depend on how you model environments at your organization.

Basic Layout

A basic form of organization is to follow the naming pattern where each $environment-$stage.yaml is a file. This works well until you have so many environments and stages.

For example, $environment might be ue2 (for us-east-2) and $stage might be prod which would result in stacks/ue2-prod.yaml

Some resources, however, are global in scope. For example, Route53 and IAM might not make sense to tie to a region. These are what we call "global resources". You might want to put these into a file like stacks/global-region.yaml to connote that they are not tied to any particular region.

Hierarchical Layout

We recommend using a hierarchical layout that follows the way AWS thinks about infrastructure. This works very well when you may have dozens or hundreds of accounts and regions that you operate in. Use Catalogs to organize your Stack configurations.