atmos vendor pull

This command implements Atmos Vendoring. Use this command to download sources from local and remote repositories for Terraform and Helmfile components and stacks.

atmos vendor pull

00:00.0 / 00:00.0

atmos vendor pull --help

With Atmos vendoring, you can copy components and other artifacts from the following sources:

• Copy all files from an OCI Registry into a local folder
• Copy all files from Git, Mercurial, Amazon S3, Google GCP into a local folder
• Copy all files from an HTTP/HTTPS endpoint into a local folder
• Copy a single file from an HTTP/HTTPS endpoint to a local file
• Copy a local file into a local folder (keeping the same file name)
• Copy a local file to a local file with a different file name
• Copy a local folder (all files) into a local folder

Usage

Execute the vendor pull command like this:

atmos vendor pull
atmos vendor pull --everything
atmos vendor pull --component <component> [options]
atmos vendor pull -c <component> [options]
atmos vendor pull --tags <tag1>,<tag2> [options]
atmos vendor pull --stack <stack> [options]
atmos vendor pull --labels <key1>=<value1>,<key2>=<value2> [options]

Description

Atmos supports two different ways of vendoring components, stacks and other artifacts:

• Using component.yaml vendoring manifest
• Using vendor.yaml vendoring manifest

The component.yaml vendoring manifest can be used to vendor components from remote repositories. A component.yaml file placed into a component's directory is used to describe the vendoring config for one component only. Using component.yaml is not recommended, and it's maintained for backwards compatibility.

The vendor.yaml vendoring manifest provides more functionality than using component.yaml files. It's used to describe vendoring config for all components, stacks and other artifacts for the entire infrastructure. The file is placed into the directory from which the atmos vendor pull command is executed. It's the recommended way to describe vendoring configurations.

Vendoring using vendor.yaml manifest

• The vendor.yaml vendoring manifest supports Kubernetes-style YAML config to describe vendoring configuration for components, stacks, and other artifacts.

• The source attribute supports all protocols (local files, Git, Mercurial, HTTP, HTTPS, Amazon S3, Google GCP), and all URL and archive formats as described in go-getter, and also the oci:// scheme to download artifacts from OCI registries. See Vendor URL Syntax for complete documentation on supported URL formats and authentication.

• The targets in the sources support absolute paths and relative paths (relative to the vendor.yaml file). Note: if the targets paths are set as relative, and if the vendor.yaml file is detected by Atmos using the base_path setting in atmos.yaml, the targets paths will be considered relative to the base_path. Multiple targets can be specified.

• included_paths and excluded_paths support POSIX-style greedy Globs for filenames/paths (double-star/globstar ** is supported as well).

• The tags in each source specifies a list of tags to apply to the component. This allows you to only vendor the components that have the specified tags by executing a command atmos vendor pull --tags <tag1>,<tag2>

tip

Refer to Atmos Vendoring for more details

Vendoring using component.yaml manifest

• The component.yaml vendoring manifest supports Kubernetes-style YAML config to describe component vendoring configuration. The file is placed into the component's folder.

• The URIs (uri) in component.yaml support all protocols (local files, Git, Mercurial, HTTP, HTTPS, Amazon S3, Google GCP), and all URL and archive formats as described in go-getter, and also the oci:// scheme to download artifacts from OCI registries.

• included_paths and excluded_paths in component.yaml support POSIX-style greedy Globs for file names/paths (double-star/globstar ** is supported as well).

tip

Refer to Atmos Component Vendoring for more details

Vendoring from OCI Registries

The following config can be used to download the vpc component from an AWS public ECR registry:

apiVersion: atmos/v1
kind: ComponentVendorConfig
metadata:
  name: vpc-vendor-config
  description: Config for vendoring of 'vpc' component
spec:
  source:
    # Download the component from the AWS public ECR registry (https://docs.aws.amazon.com/AmazonECR/latest/public/public-registries.html)
    uri: "oci://public.ecr.aws/cloudposse/components/terraform/stable/aws/vpc:{{.Version}}"
    version: "latest"

Vendoring from SSH

Atmos supports SSH for accessing non-public Git repositories, which is convenient for local development. Atmos will use any installed SSH keys automatically.

tip

In automated systems like GitHub Actions, we recommend sticking with the https:// scheme for vendoring. Atmos will automatically inject the GITHUB_TOKEN.

There are two primary ways to specify an SSH source.

SCP-style Sources

Atmos supports traditional SCP-style sources, which use a colon to separate the host from the repository, like this:

git::git@github.com:cloudposse/terraform-null-label.git?ref={{.Version}}

Atmos rewrites this URL to the following format:

git::ssh://git@github.com/cloudposse/terraform-null-label.git?depth=1&ref={{.Version}}

If no username is supplied and the host is github.com, Atmos automatically injects the default username git.

Explicit SSH Sources

When the ssh:// scheme is explicitly specified, the URL is used as provided, and no rewriting occurs.

For example:

git::ssh://git@github.com/cloudposse/terraform-null-label.git?ref={{ .Version }}

Important Notes

• The following URL is invalid because go-getter misinterprets github.com: as a URL scheme (like http: or git:), causing a parsing error:

  github.com:cloudposse/terraform-null-label.git?ref={{ .Version }}

• When a URL has no scheme, Atmos defaults to HTTPS and injects credentials if available.

  github.com/cloudposse/terraform-null-label.git?ref={{ .Version }}

Git over HTTPS Vendoring

Atmos supports vendoring components using Git over HTTPS.

For example:

github.com/cloudposse/terraform-null-label.git?ref={{ .Version }}

is automatically resolved as:

git::https://github.com/cloudposse/terraform-null-label.git?depth=1&ref={{ .Version }}

Authentication & Token Usage for HTTPS

Atmos prioritizes authentication credentials based on predefined environment variables. The priority order for each provider is:

GitHub

ATMOS_GITHUB_TOKEN

Bearer token for GitHub API requests, enabling authentication for private repositories and higher rate limits.

GITHUB_TOKEN

Used as a fallback if ATMOS_GITHUB_TOKEN is not set.

Default Username for HTTPS: x-access-token

Bitbucket

ATMOS_BITBUCKET_TOKEN

Bitbucket app password for API requests; used to avoid rate limits. When both ATMOS_BITBUCKET_TOKEN and BITBUCKET_TOKEN are defined, the former prevails.

BITBUCKET_TOKEN

Used as a fallback when ATMOS_BITBUCKET_TOKEN is not set.

BITBUCKET_USERNAME

Bitbucket username for authentication. Bitbucket requires a valid username and does not accept dummy values like x-access-token.

GitLab

ATMOS_GITLAB_TOKEN

Personal Access Token (PAT) for GitLab authentication. Takes precedence over GITLAB_TOKEN.

GITLAB_TOKEN

Used as a fallback if ATMOS_GITLAB_TOKEN is not set.

Default Username for HTTPS: "oauth2"

How HTTPS URLs Are Resolved

When resolving Git sources, Atmos follows these rules:

1. If a full HTTPS URL is provided (git::https://github.com/...), it is used as-is. No token data is injected, even if environment variables are set.
2. If a repository name is provided without a scheme (github.com/org/repo.git), it defaults to https://, and if a token is set, it is injected into the URL.
3. If a username and repository name are provided in SCP format (git@github.com:org/repo.git), it is rewritten as an SSH URL.

note

For more details on configuration, refer to Atmos Configuration.

tip

Run atmos vendor pull --help to see all the available options

Examples

Pull Everything

# Pull all vendored components
atmos vendor pull

# Explicit flag form
atmos vendor pull --everything

Pull by Tags

Use tags to selectively vendor groups of components. Tags are defined in your vendor.yaml and allow you to organize components by purpose, team, or environment:

# Pull only networking-related components
atmos vendor pull --tags networking

# Pull components for development environment
atmos vendor pull --tags dev

# Pull multiple tag groups
atmos vendor pull --tags dev,test

# Preview what would be pulled (dry-run)
atmos vendor pull --tags networking --dry-run

Pull Specific Components

# Pull a specific component by name
atmos vendor pull --component vpc
atmos vendor pull -c vpc

# Pull multiple components
atmos vendor pull -c vpc-flow-logs-bucket

# Pull a Helmfile component
atmos vendor pull -c echo-server --type helmfile

Pull by Stack

Use --stack to vendor every component in a stack that declares its own component.yaml, without vendoring the rest of the repository:

# Vendor all component.yaml-declared components used by a stack
atmos vendor pull --stack dev-us-west-2
atmos vendor pull -s dev-us-west-2

# Preview what a stack pull would do (dry-run)
atmos vendor pull --stack dev-us-west-2 --dry-run

Components in the stack without their own component.yaml are silently skipped -- not every component in a stack has to vendor this way.

Pull by Labels

--tags and --labels filter two different things, but they compose. --tags filters the tags declared directly on each vendor.yaml source (a manifest concept). --labels filters by each component's stack metadata.labels (a component concept) -- the same resolution --stack performs, and it composes with --stack to narrow further:

# Vendor every component (across all stacks) whose metadata.labels match
atmos vendor pull --labels tier=1

# Combine with --stack to narrow to one stack's matching components
atmos vendor pull --stack dev-us-west-2 --labels tier=1

# Either "=" or ":" works as the key/value separator
atmos vendor pull --labels tier:1,cost-center:platform

# Narrow a stack/labels selection further by declared vendor.yaml tags
atmos vendor pull --stack dev-us-west-2 --labels tier=1 --tags networking

--labels cannot be combined with --component (a stack-resolved set doesn't compose with a single explicit target). --tags composes with --stack/--labels as an independent filter: it narrows the stack-resolved set to only components whose vendor.yaml source also declares a matching tag. A component with no vendor.yaml entry at all (the common case for --stack, which installs via component.yaml regardless) has no tags to match and is excluded by a non-empty --tags filter -- the same way any filter excludes an entity missing the filtered attribute.

note

When executing the atmos vendor pull command, Atmos performs the following steps to decide which vendoring manifest to use:

• If --stack and/or --labels is specified, Atmos vendors every resolved component (declared in the given stack, and/or matching the given metadata.labels across all stacks) that has its own component.yaml, regardless of whether a vendor.yaml also exists. Neither flag can be combined with --component. --tags composes with --stack/--labels to narrow the resolved set further by declared vendor.yaml tags -- see Pull by Labels.

• Otherwise, if vendor.yaml manifest is found (in the directory from which the command is executed), Atmos will parse the file and execute the command against it. If the flag --component is not specified, Atmos will vendor all the artifacts defined in the vendor.yaml manifest. If the flag --component is passed in, Atmos will vendor only that component. --tags composes with --component too (narrows to that component only if its declared tags also match) or filters the whole manifest on its own.

• If vendor.yaml is not found, Atmos will look for the component.yaml manifest in the component's folder. If component.yaml is not found, an error will be thrown. The flag --component is required in this case. component.yaml has no tags concept, so --component combined with --tags in this path never matches.

Flags

--component / -c (optional)

Atmos component to pull. Mutually exclusive with --stack/--labels. Composes with --tags (narrows to this component only if its declared vendor.yaml tags also match, when a vendor.yaml exists).

--stack / -s (optional)

Only vendor components belonging to the specified stack. Vendors every resolved component that has its own component.yaml, bypassing vendor.yaml for installation. Cannot be combined with --component. Composes with --labels to narrow further, and with --tags to filter by declared vendor.yaml source tags. Considers every component type (terraform, helmfile, packer) unless you pass --type explicitly.

--everything (optional)

Vendor all components.

--tags (optional)

Only vendor the components whose vendor.yaml source declares any of the specified tags. An independent filter: composes with --component or --stack/--labels to narrow further, or stands on its own.
tags is a comma-separated values (CSV) string.

--labels (optional)

Only vendor components whose stack metadata.labels match ALL of the specified key/value pairs (a component concept, resolved the same way as --stack). Comma-separated key=value or key:value pairs, e.g. --labels=tier=1,cost-center:platform. Cannot be combined with --component; composes with --stack and --tags. Considers every component type unless you pass --type explicitly, the same as --stack.

--type / -t (optional)

Component type: terraform, helmfile, or packer (terraform is default). Pass this flag explicitly to scope a --stack/--labels selection to one component type. When you omit it, every component type is considered.

--dry-run (optional)

Dry run.

--refresh-lock (optional)

Refresh immutable vendor.lock.yaml entries from declared sources instead of failing on drift.

--lock-enforcement (optional)

Override vendor.lock.enforcement for this run: strict, warn, or silent.