Atlantis Integration
Atmos natively supports Atlantis for Terraform Pull Request Automation.
How it Works​
With Atmos, all your configurations are neatly defined in YAML. This makes transformations of that data very easy.
Atmos supports three commands that, when combined, make it easy to use Atlantis:
-
Generate the
atlantis.yaml
repo-level configuration:atmos atlantis generate repo-config
-
Generate the backend configuration for all components:
atmos terraform generate backends --format=backend-config|hcl
-
Generate the full deep-merged configurations of all stacks for each component:
atmos terraform generate varfiles
Configuration​
Atlantis Integration can be configured in two different ways (or a combination of them):
- In the
integrations.atlantis
section inatmos.yaml
- In the
settings.atlantis
sections in the stack config files
Configure Atlantis Integration in integrations.atlantis
section in atmos.yaml
​
To configure Atmos to generate the Atlantis repo configurations, update the integrations.atlantis
section in atmos.yaml
.
Here's an example to get you started. As with everything in Atmos, it supports deep-merging at all levels. Anything under
the integrations.atlantis
section in atmos.yaml
can be overridden in the stack config sections settings.atlantis
at any level of the inheritance
chain.
# atmos.yaml CLI config
# Integrations
integrations:
# Atlantis integration
# https://www.runatlantis.io/docs/repo-level-atlantis-yaml.html
atlantis:
# Path and name of the Atlantis config file `atlantis.yaml`
# Supports absolute and relative paths
# All the intermediate folders will be created automatically (e.g. `path: /config/atlantis/atlantis.yaml`)
# Can be overridden on the command line by using `--output-path` command-line argument in `atmos atlantis generate repo-config` command
# If not specified (set to an empty string/omitted here, and set to an empty string on the command line), the content of the file will be dumped to `stdout`
# On Linux/macOS, you can also use `--output-path=/dev/stdout` to dump the content to `stdout` without setting it to an empty string in `atlantis.path`
path: "atlantis.yaml"
# Config templates
# Select a template by using the `--config-template <config_template>` command-line argument in `atmos atlantis generate repo-config` command
config_templates:
config-1:
version: 3
automerge: true
delete_source_branch_on_merge: true
parallel_plan: true
parallel_apply: true
allowed_regexp_prefixes:
- dev/
- staging/
- prod/
# Project templates
# Select a template by using the `--project-template <project_template>` command-line argument in `atmos atlantis generate repo-config` command
project_templates:
project-1:
# generate a project entry for each component in every stack
name: "{tenant}-{environment}-{stage}-{component}"
workspace: "{workspace}"
dir: "{component-path}"
terraform_version: v1.8
delete_source_branch_on_merge: true
autoplan:
enabled: true
when_modified:
- "**/*.tf"
- "varfiles/$PROJECT_NAME.tfvars"
apply_requirements:
- "approved"
# Workflow templates
# https://www.runatlantis.io/docs/custom-workflows.html#custom-init-plan-apply-commands
# https://www.runatlantis.io/docs/custom-workflows.html#custom-run-command
workflow_templates:
workflow-1:
plan:
steps:
- run: terraform init -input=false
# When using workspaces, you need to select the workspace using the $WORKSPACE environment variable
- run: terraform workspace select $WORKSPACE
# You must output the plan using `-out $PLANFILE` because Atlantis expects plans to be in a specific location
- run: terraform plan -input=false -refresh -out $PLANFILE -var-file varfiles/$PROJECT_NAME.tfvars
apply:
steps:
- run: terraform apply $PLANFILE
Using the config and project templates, Atmos generates a separate atlantis project for each Atmos component in every stack.
For example, by running this command:
atmos atlantis generate repo-config --config-template config-1 --project-template project-1
the following Atlantis repo-config would be generated:
version: 3
automerge: true
delete_source_branch_on_merge: true
parallel_plan: true
parallel_apply: true
allowed_regexp_prefixes:
- dev/
- staging/
- prod/
projects:
- name: tenant1-ue2-staging-test-test-component-override-3
workspace: test-component-override-3-workspace
workflow: workflow-1
dir: tests/fixtures/scenarios/complete/components/terraform/test/test-component
terraform_version: v1.8
delete_source_branch_on_merge: true
autoplan:
enabled: true
when_modified:
- '**/*.tf'
- varfiles/$PROJECT_NAME.tfvars
apply_requirements:
- approved
- name: tenant1-ue2-staging-infra-vpc
workspace: tenant1-ue2-staging
workflow: workflow-1
dir: tests/fixtures/scenarios/complete/components/terraform/infra/vpc
terraform_version: v1.8
delete_source_branch_on_merge: true
autoplan:
enabled: true
when_modified:
- '**/*.tf'
- varfiles/$PROJECT_NAME.tfvars
apply_requirements:
- approved
workflows:
workflow-1:
apply:
steps:
- run: terraform apply $PLANFILE
plan:
steps:
- run: terraform init -input=false
- run: terraform workspace select $WORKSPACE
- run: terraform plan -input=false -refresh -out $PLANFILE -var-file varfiles/$PROJECT_NAME.tfvars
NOTE: If Atlantis Integration is configured only in the integrations.atlantis
section in atmos.yaml
, the command-line
flags --config-template
and --project-template
are required to specify a config template and a project template from the collection of
templates defined in the integrations.atlantis.config_templates
and integrations.atlantis.project_templates
sections in atmos.yaml
. You can
change this behavior by using the settings.atlantis
sections in stack config files.
Configure Atlantis Integration in settings.atlantis
sections in stack configs​
The integrations.atlantis.config_templates
, integrations.atlantis.config_templates
and integrations.atlantis.config_templates
sections
in atmos.yaml
can be overridden in the settings.atlantis
sections in stack config files. In fact, you don't have to define the sections
in atmos.yaml
at all and instead use only the settings.atlantis
sections in stack configs to configure work with the Atlantis Integration.
Configuring the Atlantis Integration in the settings.atlantis
sections in the stack configs has the following advantages:
-
The
settings
section is a first class section in Atmos (similar tovars
). It participates in deep-merging and in the inheritance chain. It can be defined and overridden at any level (organization/namespace, OU/tenant, region/environment, account/stage, base component, component). You can define the base settings at the org, tenant or account level, and then override some settings at the component level, making the whole configuration DRY -
When executing the
atmos atlantis generate repo-config
command, you don't need to pass the--config-template
and--project-template
flags to specify which config and project templates to use. Instead, Atmos will get this information from thesettings.atlantis
section -
When executing the
atmos describe component <component> -s <stack>
command, you will see the configured Atlantis Integration in the outputs. For example:
- If you configure the Atlantis Integration in the
settings.atlantis
sections in the stack configs, then the commandatmos describe affected
will be able to use it and output the affected Atlantis projects in theatlantis_project
field. For example:
Configure settings.atlantis.workflow_templates
section in stack configs​
If you are using the Atlantis Repo Level workflows, you can configure the workflows
in the settings.atlantis.workflow_templates
section.
If the settings.atlantis.workflow_templates
section is configured in stack configs, it's copied to the generated atlantis.yaml
file verbatim.
For example, add the workflow_templates
section at the org level in the config file stacks/orgs/cp/_defaults.yaml
:
settings:
atlantis:
workflow_templates:
workflow-1:
apply:
steps:
- run: terraform apply $PLANFILE
plan:
steps:
- run: terraform init
- run: terraform workspace select $WORKSPACE || terraform workspace new $WORKSPACE
- run: terraform plan -out $PLANFILE -var-file varfiles/$PROJECT_NAME.tfvars.json
then execute the atmos atlantis generate repo-config
command:
version: 3
workflows:
workflow-1:
apply:
steps:
- run: terraform apply $PLANFILE
plan:
steps:
- run: terraform init
- run: terraform workspace select $WORKSPACE || terraform workspace new $WORKSPACE
- run: terraform plan -out $PLANFILE -var-file varfiles/$PROJECT_NAME.tfvars
The settings.atlantis.workflow_templates
section in stack configs has higher priority then the integration.atlantis.workflow_templates
section in atmos.yaml
. If both are defined, Atmos will select the workflows from the settings.atlantis.workflow_templates
section and copy them
into the generated atlantis.yaml
file. On the other hand, if the settings.atlantis.workflow_templates
section is not defined in stack configs,
Atmos will use the workflows from the integration.atlantis.workflow_templates
section from atmos.yaml
.
Define config template and project template in settings.atlantis
section in stack configs​
The Atlantis config template and project template can be defined in the settings.atlantis
section in two different ways:
-
Define
config_template_name
andproject_template_name
in thesettings.atlantis
section. These attributes tell Atmos to select a config template and a project template from theintegration.atlantis
section inatmos.yaml
. For example:atmos.yamlintegrations:
atlantis:
path: "atlantis.yaml"
# Config templates
config_templates:
config-1:
version: 3
automerge: true
delete_source_branch_on_merge: true
parallel_plan: true
parallel_apply: true
allowed_regexp_prefixes:
- dev/
- staging/
- prod/
# Project templates
project_templates:
project-1:
# generate a project entry for each component in every stack
name: "{tenant}-{environment}-{stage}-{component}"
workspace: "{workspace}"
dir: "{component-path}"
terraform_version: v1.8
delete_source_branch_on_merge: true
autoplan:
enabled: true
when_modified:
- "**/*.tf"
- "varfiles/$PROJECT_NAME.tfvars.json"
apply_requirements:
- "approved"stacks/orgs/cp/_defaults.yamlsettings:
atlantis:
# Select a config template defined in `atmos.yaml` in
# the `integrations.atlantis.config_templates` section
config_template_name: "config-1"
# Select a project template defined in `atmos.yaml` in
# the `integrations.atlantis.project_templates` section
project_template_name: "project-1"In this case, the
config_template_name
andproject_template_name
attributes are used instead of specifying the--config-template
and--project-template
flags on the command line when executing the commandatmos atlantis generate repo-config
. And the attributes can be defined at any level in the stack configs and they participate in deep-merging and inheritance (meaning they can be overridden per tenant, environment, stage and component). -
Define
config_template
andproject_template
in thesettings.atlantis
section. These attributes tell Atmos to use the templates instead of searching for them in theintegration.atlantis
section inatmos.yaml
. For example:stacks/orgs/cp/tenant1/dev/us-east-2.yamlsettings:
atlantis:
# For this `tenant1-ue2-dev` stack, override the org-wide config template
# specified in `stacks/orgs/cp/_defaults.yaml`
# in the `settings.atlantis.config_template_name` section
config_template:
version: 3
automerge: false
delete_source_branch_on_merge: false
parallel_plan: true
parallel_apply: false
allowed_regexp_prefixes:
- dev/
# For this `tenant1-ue2-dev` stack, override the org-wide project template
# specified in `stacks/orgs/cp/_defaults.yaml`
# in the `settings.atlantis.project_template_name` section
project_template:
# generate a project entry for each component in every stack
name: "{tenant}-{environment}-{stage}-{component}"
workspace: "{workspace}"
workflow: "workflow-1"
dir: "{component-path}"
terraform_version: v1.8
delete_source_branch_on_merge: false
autoplan:
enabled: true
when_modified:
- "**/*.tf"
- "varfiles/$PROJECT_NAME.tfvars.json"
apply_requirements:
- "approved"
-
Atlantis integration can be configured in the
integrations.atlantis
section inatmos.yaml
. If this is the only place where it's configured, then you need to pass the--config-template
and--project-template
flags to theatmos atlantis generate repo-config
command -
Atlantis integration can also be configured in the
settings.atlantis
section in the stack configs. Theconfig_template_name
andproject_template_name
attributes can be used to select the config and project templates from theintegrations.atlantis
section inatmos.yaml
instead of specifying the--config-template
and--project-template
flags on the command line -
The
config_template
andproject_template
sections insettings.atlantis
can be used to define the config and project template for the particular stack or component. If defined, the sections will override all the configurations in theintegrations.atlantis
section inatmos.yaml
, and will override theconfig_template_name
andproject_template_name
attributes insettings.atlantis
. These sections have the highest priority.
Atlantis Workflows​
Atlantis workflows can be defined in two different ways:
-
In the Server Side Config using the
workflows
section andworkflow
attributerepos:
- id: /.*/
branch: /.*/
# 'workflow' sets the workflow for all repos that match.
# This workflow must be defined in the workflows section.
workflow: custom
# allowed_overrides specifies which keys can be overridden by this repo in
# its atlantis.yaml file.
allowed_overrides: [apply_requirements, workflow, delete_source_branch_on_merge, repo_locking]
# allowed_workflows specifies which workflows the repos that match
# are allowed to select.
allowed_workflows: [custom]
# allow_custom_workflows defines whether this repo can define its own
# workflows. If false (default), the repo can only use server-side defined
# workflows.
allow_custom_workflows: true
# workflows lists server-side custom workflows
workflows:
custom:
plan:
steps:
- init
- plan
apply:
steps:
- run: echo applying
- apply -
In the Repo Level atlantis.yaml Config using the
workflows
section and theworkflow
attribute in each Atlantis project inatlantis.yaml
version: 3
projects:
- name: my-project-name
branch: /main/
dir: .
workspace: default
workflow: myworkflow
workflows:
myworkflow:
plan:
steps:
- init
- plan
apply:
steps:
- run: echo applying
- apply
If you use the Server Side Config to define the Atlantis workflows, you don't need to
define workflows in the CLI Config Atlantis Integration section in atmos.yaml
or in
the settings.atlantis.workflow_templates
section in the stack configurations. When you defined the workflows in the server config workflows
section, you can reference a workflow to be used for each generated Atlantis project in the project templates.
On the other hand, if you use Repo Level workflows,
you need to provide at least one workflow template in the integrations.atlantis.workflow_templates
section in
the Atlantis Integration in atmos.yaml
, or in the settings.atlantis.workflow_templates
section in the stack
configurations.
For example, after executing the following command:
atmos atlantis generate repo-config --config-template config-1 --project-template project-1
the generated atlantis.yaml
file would look like this:
version: 3
projects:
- name: tenant1-ue2-dev-infra-vpc
workspace: tenant1-ue2-dev
workflow: workflow-1
workflows:
workflow-1:
apply:
steps:
- run: terraform apply $PLANFILE
plan:
steps:
- run: terraform init -input=false
- run: terraform workspace select $WORKSPACE || terraform workspace new $WORKSPACE
- run: terraform plan -input=false -refresh -out $PLANFILE -var-file varfiles/$PROJECT_NAME.tfvars.json
Dynamic Repo Config Generation​
If you want to generate the atlantis.yaml
file before Atlantis can parse it, you can use
the Dynamic Repo Config Generation feature of Atlantis. You
can add a run
command to pre_workflow_hooks
. The repo config will be generated right before Atlantis can parse it.
repos:
- id: /.*/
pre_workflow_hooks:
- run: "./repo-config-generator.sh"
description: "Generating configs"
See also Pre Workflow Hooks and Post Workflow Hooks for more information.
To help with dynamic repo config generation, the atmos atlantis generate repo-config
command accepts the --affected-only
flag.
If set to true
, Atmos will generate Atlantis projects only for the Atmos components changed between two Git commits.
repos:
- id: /.*/
pre_workflow_hooks:
- run: "atmos atlantis generate repo-config --affected-only=true"
description: "Generating configs"
If the --affected-only=true
flag is passed, Atmos uses two different Git commits to produce a list of affected Atmos components and stacks and then
generate the atlantis.yaml
file for the affected Atlantis projects only.
For the first commit, the command assumes that the current repo root is a Git checkout. An error will be thrown if the current repo is not a Git
repository (the .git
folder does not exist or is configured incorrectly).
The second commit can be specified on the command line by using
the --ref
(Git References) or --sha
(commit SHA) flags.
Either --ref
or --sha
should be used. If both flags are provided at the same time, the command will first clone the remote branch pointed to by
the --ref
flag and then checkout the Git commit pointed to by the --sha
flag (--sha
flag overrides --ref
flag).
NOTE: If the flags are not provided, the ref
will be set automatically to the reference to the default branch (e.g. main
) and the commit SHA
will point to the HEAD
of the branch.
If you specify the --repo-path
flag with the path to the already cloned repository, the command will not clone the target
repository, but instead will use the already cloned one to compare the current branch with. In this case, the --ref
, --sha
, --ssh-key
and --ssh-key-password
flags are not used, and an error will be thrown if the --repo-path
flag and any of the --ref
, --sha
, --ssh-key
or --ssh-key-password
flags are provided at the same time.
The command works by:
-
Cloning the target branch (
--ref
) or checking out the commit (--sha
) of the remote target branch, or using the already cloned target repository specified by the--repo-path
flag -
Deep merging all stack configurations for both the current working branch and the remote target branch
-
Looking for changes in the component directories
-
Comparing each section of the stack configuration looking for differences
-
Generating the
atlantis.yaml
file with theprojects
sections consisting of a list of the affected Atlantis projects
Since Atmos first checks the component folders for changes, if it finds any affected files, it will mark all related components and stacks as affected. Atmos will then skip evaluating those stacks for differences since we already know that they are affected.
Refer to atmos atlantis generate repo-config
for the description of the command and all flags.
Working with Private Repositories​
If the flag --affected-only=true
is passed on the command line (e.g. atmos atlantis generate repo-config --affected-only=true
), the command
will clone and checkout the remote target repo (which can be the default refs/heads<default_branch>
reference, or specified by the command-line
flags --ref
, --sha
or --repo-path
). If the remote target repo is private, special attention needs to be given to how to work with private
repositories.
There are a few ways to work with private repositories with which the current local branch is compared to detect the changed files and affected Atmos stacks and components:
-
Using the
--ssh-key
flag to specify the filesystem path to a PEM-encoded private key to clone private repos using SSH, and the--ssh-key-password
flag to provide the encryption password for the PEM-encoded private key if the key contains a password-encrypted PEM block -
Execute the
atmos atlantis generate repo-config --affected-only=true --repo-path <path_to_cloned_target_repo>
command in a GitHub Action. For this to work, clone the remote target repository using the checkout GitHub action. Then use the--repo-path
flag to specify the path to the already cloned target repository with which to compare the current branch -
It should just also work with whatever SSH config/context has been already set up, for example, when using SSH agents. In this case, you don't need to use the
--ssh-key
,--ssh-key-password
and--repo-path
flags to clone private repositories
Using with GitHub Actions​
If the atmos atlantis generate repo-config --affected-only=true
command is executed in a GitHub Action, and
you don't want to store or generate a long-lived SSH private key on the server, you can do the following (NOTE: This is only required if the
action is attempting to clone a private repo which is not itself):
-
Create a GitHub Personal Access Token (PAT) with scope permissions to clone private repos
-
Add the created PAT as a repository or GitHub organization secret
-
In your GitHub Action, clone the remote repository using the checkout GitHub Action
-
Execute
atmos atlantis generate repo-config --affected-only=true --repo-path <path_to_cloned_target_repo>
command with the--repo-path
flag set to the cloned repository path using theGITHUB_WORKSPACE
ENV
variable ( which points to the default working directory on the GitHub runner for steps, and the default location of the repository when using the checkout action). For example:atmos atlantis generate repo-config --affected-only=true --repo-path $GITHUB_WORKSPACE
Example GitHub Action​
Here's an example GitHub Action to use Atlantis with Atmos.
The action executes the atmos generate varfiles/backends
commands to generate Terraform varfiles and backend config files for all Atmos stacks,
then executes the atmos atlantis generate repo-config
command to generate the Atlantis repo config file (atlantis.yaml
) for all Atlantis projects,
then commits all the generated files and calls Atlantis via a webhook.
You can adopt and modify it to your own needs.
name: atmos
on:
workflow_dispatch:
issue_comment:
types:
- created
pull_request:
types:
- opened
- edited
- synchronize
- closed
branches: [ main ]
env:
ATMOS_VERSION: 1.156.0
ATMOS_CLI_CONFIG_PATH: ./
jobs:
generate-atlantis-yaml:
name: Generate varfiles, backend config and atlantis.yaml
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
if: github.event.pull_request.state == 'open' || ${{ github.event.issue.pull_request }}
with:
ref: ${{ github.event.pull_request.head.ref }}
fetch-depth: 2
# Install Atmos and generate tfvars and backend config files
- name: Generate TF var files and backend configs
if: github.event.pull_request.state == 'open' || ${{ github.event.issue.pull_request }}
shell: bash
run: |
wget -q https://github.com/cloudposse/atmos/releases/download/v${ATMOS_VERSION}/atmos_${ATMOS_VERSION}_linux_amd64 && \
mv atmos_${ATMOS_VERSION}_linux_amd64 /usr/local/bin/atmos && \
chmod +x /usr/local/bin/atmos
atmos terraform generate varfiles --file-template={component-path}/varfiles/{namespace}-{environment}-{component}.tfvars.json
atmos terraform generate backends --format=backend-config --file-template={component-path}/backends/{namespace}-{environment}-{component}.backend
# Commit changes (if any) to the PR branch
- name: Commit changes to the PR branch
if: github.event.pull_request.state == 'open' || ${{ github.event.issue.pull_request }}
shell: bash
run: |
untracked=$(git ls-files --others --exclude-standard)
changes_detected=$(git diff --name-only)
if [ -n "$untracked" ] || [ -n "$changes_detected" ]; then
git config --global user.name github-actions
git config --global user.email github-actions@github.com
git add -A *
git commit -m "Committing generated autogenerated var files"
git push
fi
# Generate atlantis.yaml with atmos
- name: Generate Dynamic atlantis.yaml file
if: github.event.pull_request.state == 'open' || ${{ github.event.issue.pull_request }}
shell: bash
run: |
atmos atlantis generate repo-config --config-template config-1 --project-template project-1
# Commit changes (if any) to the PR branch
- name: Commit changes to the PR branch
if: github.event.pull_request.state == 'open' || ${{ github.event.issue.pull_request }}
shell: bash
run: |
yaml_changes=$(git diff --name-only)
untracked=$(git ls-files --others --exclude-standard atlantis.yaml)
if [ -n "$yaml_changes" ] || [ -n "$untracked" ]; then
git config --global user.name github-actions
git config --global user.email github-actions@github.com
git add -A *
git commit -m "Committing generated atlantis.yaml"
git push
fi
call-atlantis:
if: ${{ always() }}
needs: generate-atlantis-yaml
name: Sending data to Atlantis
runs-on: ubuntu-latest
steps:
- name: Invoke deployment hook
uses: distributhor/workflow-webhook@v2
env:
webhook_type: 'json-extended'
webhook_url: ${{ secrets.WEBHOOK_URL }}
webhook_secret: ${{ secrets.WEBHOOK_SECRET }}
verbose: false
Next Steps​
Generating the Atlantis repo-config
is only part of what's needed to use Atmos with Atlantis. The rest will depend on your organization's
preferences for generating the Terraform .tfvars
files and backends.
You can use pre-commit hooks and/or GitHub Actions (or similar) to generate the .tfvars
files and state backend configurations, which are derived
from the Atmos stack configurations.
The following commands will generate those files.
You can commit the resulting files back to VCS (e.g. git add -A
) and push upstream. That way Atlantis will trigger on the "affected
files" and propose a plan.
Or you can use the Dynamic Repo Config Generation in the Atlantis pre-workflow hooks and
the atmos atlantis generate repo-config
command with the --affected-only=true
flag to dynamically generate the atlantis.yaml
file with the
affected (changed) Atlantis projects to avoid the need of committing those files to VCS.
References​
For more information, refer to: