atmos vendor pull
This command implements Atmos Vendoring. Use this command to download sources from local and remote repositories for Terraform and Helmfile components and stacks.
With Atmos vendoring, you can copy components and other artifacts from the following sources:
- Copy all files from an OCI Registry into a local folder
- Copy all files from Git, Mercurial, Amazon S3, Google GCP into a local folder
- Copy all files from an HTTP/HTTPS endpoint into a local folder
- Copy a single file from an HTTP/HTTPS endpoint to a local file
- Copy a local file into a local folder (keeping the same file name)
- Copy a local file to a local file with a different file name
- Copy a local folder (all files) into a local folder
Usage​
Execute the vendor pull
command like this:
atmos vendor pull
atmos vendor pull --everything
atmos vendor pull --component <component> [options]
atmos vendor pull -c <component> [options]
atmos vendor pull --tags <tag1>,<tag2> [options]
Description​
Atmos supports two different ways of vendoring components, stacks and other artifacts:
- Using
component.yaml
vendoring manifest - Using
vendor.yaml
vendoring manifest
The component.yaml
vendoring manifest can be used to vendor components from remote repositories.
A component.yaml
file placed into a component's directory is used to describe the vendoring config for one component only.
Using component.yaml
is not recommended, and it's maintained for backwards compatibility.
The vendor.yaml
vendoring manifest provides more functionality than using component.yaml
files.
It's used to describe vendoring config for all components, stacks and other artifacts for the entire infrastructure.
The file is placed into the directory from which the atmos vendor pull
command is executed. It's the recommended way to describe vendoring
configurations.
Vendoring using vendor.yaml
manifest​
-
The
vendor.yaml
vendoring manifest supports Kubernetes-style YAML config to describe vendoring configuration for components, stacks, and other artifacts. -
The
source
attribute supports all protocols (local files, Git, Mercurial, HTTP, HTTPS, Amazon S3, Google GCP), and all URL and archive formats as described in go-getter, and also theoci://
scheme to download artifacts from OCI registries. -
The
targets
in thesources
support absolute paths and relative paths (relative to thevendor.yaml
file). Note: if thetargets
paths are set as relative, and if thevendor.yaml
file is detected by Atmos using thebase_path
setting inatmos.yaml
, thetargets
paths will be considered relative to thebase_path
. Multiple targets can be specified. -
included_paths
andexcluded_paths
support POSIX-style greedy Globs for filenames/paths (double-star/globstar**
is supported as well). -
The
tags
in each source specifies a list of tags to apply to the component. This allows you to only vendor the components that have the specified tags by executing a commandatmos vendor pull --tags <tag1>,<tag2>
Refer to Atmos Vendoring
for more details
Vendoring using component.yaml
manifest​
-
The
component.yaml
vendoring manifest supports Kubernetes-style YAML config to describe component vendoring configuration. The file is placed into the component's folder. -
The URIs (
uri
) incomponent.yaml
support all protocols (local files, Git, Mercurial, HTTP, HTTPS, Amazon S3, Google GCP), and all URL and archive formats as described in go-getter, and also theoci://
scheme to download artifacts from OCI registries. -
included_paths
andexcluded_paths
incomponent.yaml
support POSIX-style greedy Globs for file names/paths (double-star/globstar**
is supported as well).
Refer to Atmos Component Vendoring
for more details
Vendoring from OCI Registries​
The following config can be used to download the vpc
component from an AWS public ECR registry:
apiVersion: atmos/v1
kind: ComponentVendorConfig
metadata:
name: vpc-vendor-config
description: Config for vendoring of 'vpc' component
spec:
source:
# Download the component from the AWS public ECR registry (https://docs.aws.amazon.com/AmazonECR/latest/public/public-registries.html)
uri: "oci://public.ecr.aws/cloudposse/components/terraform/stable/aws/vpc:{{.Version}}"
version: "latest"
Vendoring from SSH​
Atmos supports SSH for accessing non-public Git repositories, which is convenient for local development. Atmos will use any installed SSH keys automatically.
In automated systems like GitHub Actions, we recommend sticking with the https://
scheme for vendoring. Atmos will automatically inject the GITHUB_TOKEN
.
There are two primary ways to specify an SSH source.
SCP-style Sources​
Atmos supports traditional SCP-style sources, which use a colon to separate the host from the repository, like this:
git::git@github.com:cloudposse/terraform-null-label.git?ref={{.Version}}
Atmos rewrites this URL to the following format:
git::ssh://git@github.com/cloudposse/terraform-null-label.git?depth=1&ref={{.Version}}
If no username is supplied and the host is github.com
, Atmos automatically injects the default username git
.
Explicit SSH Sources​
When the ssh://
scheme is explicitly specified, the URL is used as provided, and no rewriting occurs.
For example:
git::ssh://git@github.com/cloudposse/terraform-null-label.git?ref={{ .Version }}
Important Notes​
-
The following URL is invalid because
go-getter
misinterpretsgithub.com:
as a URL scheme (likehttp:
orgit:
), causing a parsing error:github.com:cloudposse/terraform-null-label.git?ref={{ .Version }}
-
When a URL has no scheme, Atmos defaults to HTTPS and injects credentials if available.
github.com/cloudposse/terraform-null-label.git?ref={{ .Version }}
Git over HTTPS Vendoring​
Atmos supports vendoring components using Git over HTTPS.
For example:
github.com/cloudposse/terraform-null-label.git?ref={{ .Version }}
is automatically resolved as:
git::https://github.com/cloudposse/terraform-null-label.git?depth=1&ref={{ .Version }}
Authentication & Token Usage for HTTPS​
Atmos prioritizes authentication credentials based on predefined environment variables. The priority order for each provider is:
GitHub​
ATMOS_GITHUB_TOKEN
- Bearer token for GitHub API requests, enabling authentication for private repositories and higher rate limits.
GITHUB_TOKEN
- Used as a fallback if
ATMOS_GITHUB_TOKEN
is not set.
Default Username for HTTPS: x-access-token
Bitbucket​
ATMOS_BITBUCKET_TOKEN
- Bitbucket app password for API requests; used to avoid rate limits. When both
ATMOS_BITBUCKET_TOKEN
andBITBUCKET_TOKEN
are defined, the former prevails. BITBUCKET_TOKEN
- Used as a fallback when
ATMOS_BITBUCKET_TOKEN
is not set. ATMOS_BITBUCKET_USERNAME
- Bitbucket username for authentication. Takes precedence over
BITBUCKET_USERNAME
. BITBUCKET_USERNAME
- Used as a fallback when
ATMOS_BITBUCKET_USERNAME
is not set. Bitbucket requires a valid username and does not accept dummy values likex-access-token
.
GitLab​
ATMOS_GITLAB_TOKEN
- Personal Access Token (PAT) for GitLab authentication. Takes precedence over
GITLAB_TOKEN
. GITLAB_TOKEN
- Used as a fallback if
ATMOS_GITLAB_TOKEN
is not set.
Default Username for HTTPS: "oauth2"
How HTTPS URLs Are Resolved​
When resolving Git sources, Atmos follows these rules:
- If a full HTTPS URL is provided (
git::https://github.com/...
), it is used as-is. No token data is injected, even if environment variables are set. - If a repository name is provided without a scheme (
github.com/org/repo.git
), it defaults tohttps://
, and if a token is set, it is injected into the URL. - If a username and repository name are provided in SCP format (
git@github.com:org/repo.git
), it is rewritten as an SSH URL.
For more details on configuration, refer to Atmos Configuration.
Run atmos vendor pull --help
to see all the available options
Examples​
atmos vendor pull
atmos vendor pull --everything
atmos vendor pull --component vpc
atmos vendor pull -c vpc-flow-logs-bucket
atmos vendor pull -c echo-server --type helmfile
atmos vendor pull --tags dev,test
atmos vendor pull --tags networking --dry-run
When executing the atmos vendor pull
command, Atmos performs the following steps to decide which vendoring manifest to use:
-
If
vendor.yaml
manifest is found (in the directory from which the command is executed), Atmos will parse the file and execute the command against it. If the flag--component
is not specified, Atmos will vendor all the artifacts defined in thevendor.yaml
manifest. If the flag--component
is passed in, Atmos will vendor only that component -
If
vendor.yaml
is not found, Atmos will look for thecomponent.yaml
manifest in the component's folder. Ifcomponent.yaml
is not found, an error will be thrown. The flag--component
is required in this case
Flags​
Flag | Description | Alias | Required |
---|---|---|---|
--component | Atmos component to pull | -c | no |
--tags | Only vendor the components that have the specified tags.tags is a comma-separated values (CSV) string | no | |
--type | Component type: terraform or helmfile (terraform is default) | -t | no |
--dry-run | Dry run | no |