Skip to main content

Atmos Stacks

Stacks are a way to express the complete infrastructure needed for an environment. Think of a Stack like an architectural "Blueprint" composed of one or more Components and defined using a standardized YAML configuration.

This abstraction layer helps to automate the orchestration and deployment of loosely coupled components, such as Terraform "root" modules. They enable scalable infrastructure-as-code configurations, allowing environments to inherit from one or more common bases (child stacks) by importing configuration that gets deep-merged, thus minimizing config duplication and manual effort. Each stack uses a simple schema that provides a declarative description of your various environments. This approach empowers you to separate your infrastructure’s environment configuration settings from the code it manages (e.g., Terraform components).

By facilitating the infrastructure configurations this way, developers achieve DRY (Don't Repeat Yourself) architectures with minimal configuration. Stacks make infrastructure more streamlined and consistent, significantly enhancing productivity. Best of all, Stacks can deploy vanilla Terraform "root" modules without any code generation, custom vendor extensions, or changes to the HCL code.

Atmos utilizes a custom YAML configuration format for stacks. YAML is ideal because it's portable across multiple toolchains and languages; every developer understands it. The Atmos CLI, the terraform-utils-provider provider, and Spacelift via the terraform-spacelift-cloud-infrastructure-automation module all support stacks. Utilizing the Terraform provider enables native access to the entire infrastructure configuration directly from Terraform.

Use-cases

  • Rapid Environment Provisioning: Leverage stacks to swiftly set up and replicate development, testing, and production environments, ensuring consistency and reducing manual setup errors. This accelerates the development cycle and enables businesses to respond quickly to market demands or development needs.
  • Multi-Tenant Infrastructure Management: Utilize stacks to manage and isolate resources for different clients or projects within a single cloud infrastructure. This approach supports SaaS companies in providing secure, isolated environments for each tenant, optimizing resource utilization and simplifying the management of complex, multi-tenant architectures.
  • Compliance and Governance: Implement stacks to enforce compliance and governance policies across all environments systematically. By defining standard configurations that meet regulatory requirements, businesses can ensure that every deployment is compliant, reducing the risk of violations and enhancing security posture.

Best Practices

tip

Remember to adhere to the laws physics. All other laws are meant broken.

  • Treat Stack Templates like an Escape Hatch Apply it carefully and only when necessary. Using templates instead of inheritance can make stack configurations complex and hard to manage. Be careful using stack templates together with the component factory pattern.
  • Avoid Too Many Levels of Imports It's very difficult for others to follow relationships when there are too many nested levels and overrides.
  • Balance DRY Principles with Configuration Clarity Avoid overly DRY configuration as it leads to complexity rashes. Sometimes repeating configuration is beneficial for maintenance and clarity.
  • Reserve Code Generation for Stack Configuration While we generally advise against using code generation for application logic (components), it's beneficial for creating configurations where appropriate, such as developer environments and SaaS tenants. These configurations ought to be committed.
  • Use Mixin Pattern for Snippets of Stack Configuration Employ the mixin pattern for clarity when there there is brief configuration snippets that are reusable. Steer clear of minimal stack configurations simply for the sake of DRYness as it frequently leads to too many levels of imports.

Conventions

The differentiation between the following two types of stacks is crucial for understanding how to organize stacks and the basis for the various design patterns.

Parent Stacks

Parent Stacks are the top-level stacks that are responsible for importing Child stacks. Components inside of Parent stacks are deployable, unlike in Child stacks.

Child Stacks

Child Stacks are any stacks whose components cannot be deployed independently without being imported by a Parent Stack. Catalogs are typically where we keep our Child stacks.

Schema

A Stack file contains a manifest defined in YAML that follows a simple, extensible schema. In fact, every Stack file follows exactly the same schema, and every setting in the configuration is optional. Enforcing a consistent schema ensures we can easily import and deep-merge configurations and use inheritance to achieve DRY configuration.

# Configurations that should get deep-merged into this one
import:
# each import is a "Stack" file. The `.yaml` extension is optional, and we do not recommend using it.
- ue2-globals

# Top-level variables that are inherited by every single component.
# Use these wisely. Too many global vars will pollute the variable namespace.
vars:
# Variables can be anything you want. They can be scalars, lists, and maps. Whatever is supported by YAML.
stage: dev

# There can then be global variables for each type of component.
# Here we set global variables for any "terraform" component.
terraform:
vars: { }

# Here we set global variables for any "helmfile" component.
helmfile:
vars: { }

# Components are the building blocks of reusable infrastructure.
# They can be anything. Atmos natively supports "terraform" and "helmfile".
components:
terraform:
vpc:
command: "/usr/bin/terraform-0.15"
backend:
s3:
workspace_key_prefix: "vpc"
vars:
cidr_block: "10.102.0.0/18"
eks:
backend:
s3:
workspace_key_prefix: "eks"
vars: { }

helmfile:
nginx-ingress:
vars:
installed: true

Stack Attributes

components

The components is the list of all the building blocks.

Example:

components:
sometool: # "sometool" can be any tool
somecomponent: # "somecomponent" can be the name of any "sometool" component
vars: # etc...

So for terraform, it might look something like this:

components:
terraform:
vpc:
vars: # etc...

Stack Files

Stack files can be very numerous in large cloud environments (think many dozens to hundreds of stack files). To enable the proper organization of stack files, SweetOps has established some conventions that are good to follow. However, these are just conventions, and there are no limits enforced by the tool.

By convention, we recommend to store all Stacks in a stacks/ folder at the root of your infrastructure repository. This way it's clear where they live and helps keep the configuration separate from your code (e.g. HCL).

The filename of individual environment stacks can follow any convention, and the best one will depend on how you model environments at your organization.

Basic Layout

A basic form of organization is to follow the pattern of naming where each $environment-$stage.yaml is a file. This works well until you have so many environments and stages.

For example, $environment might be ue2 (for us-east-2) and $stage might be prod which would result in stacks/ue2-prod.yaml

Some resources, however, are global in scope. For example, Route53 and IAM might not make sense to tie to a region. These are what we call "global resources". You might want to put these into a file like stacks/global-region.yaml to connote that they are not tied to any particular region.

Hierarchical Layout

We recommend using a hierarchical layout that follows the way AWS thinks about infrastructure. This works very well when you may have dozens or hundreds of accounts and regions that you operate in. Use Catalogs to organize your Stack configurations.