Vendor Manifest
The vendoring configuration is defined in the vendor.yaml
manifest (vendor config file). The vendoring manifest is used to make copies of 3rd-party components, stacks, and other artifacts in your own repository.
It functions a little bit like the packages.json
file in Node.js or the go.mod
file in Go, but for infrastructure code.
How it works
Atmos searches for the vendoring manifest in the following locations and uses the first one found:
-
In the directory from which the
atmos vendor pull
command is executed, usually in the root of the infrastructure repo -
In the directory pointed to by the
base_path
setting in theatmos.yaml
CLI config file
After defining the vendor.yaml
manifest, all the remote artifacts can be downloaded by running the following command:
atmos vendor pull
To vendor a particular component or other artifact, execute the following command:
atmos vendor pull -c <component>
To vendor components and artifacts tagged with specific tags, execute the following command:
atmos vendor pull --tags <tag1>,<tag2>
Refer to atmos vendor pull
CLI command for more details
Vendoring Manifest
To vendor remote artifacts, create a vendor.yaml
file similar to the example below:
vendor.yaml
With this configuration, it would be possible to run the following commands:
The vendor.yaml
vendoring manifest supports Kubernetes-style YAML config to describe vendoring configuration for components, stacks,
and other artifacts. The file is placed into the directory from which the atmos vendor pull
command is executed (usually the root of the repo).
source
The
source
attribute supports all protocols (local files, Git, Mercurial, HTTP, HTTPS, Amazon S3, Google GCP), and all the URL and archive formats as described in go-getter, and also theoci://
scheme to download artifacts from OCI registries.IMPORTANT: Include the
{{ .Version }}
parameter in yoursource
URI to ensure the correct version of the artifact is downloaded. For example:source: "github.com/cloudposse/terraform-aws-components.git//modules/vpc-flow-logs-bucket?ref={{.Version}}"
targets
The
targets
in each source supports absolute paths and relative paths (relative to thevendor.yaml
file). Note: if thetargets
paths are set as relative, and if thevendor.yaml
file is detected by Atmos using thebase_path
setting inatmos.yaml
, thetargets
paths will be considered relative to thebase_path
. Multiple targets can be specified.included_paths
andexcluded_paths
included_paths
andexcluded_paths
support POSIX-style greedy Globs for filenames/paths (double-star/globstar**
is supported as well).component
The
component
attribute in each source is optional. It's used in theatmos vendor pull -- component <component>
command if the component is passed in. In this case, Atmos will vendor only the specified component instead of vendoring all the artifacts configured in thevendor.yaml
manifest.source
andtargets
templatesThe
source
andtargets
attributes support Go templates and Sprig Functions. This can be used to templatise thesource
andtargets
paths with the artifact versions specified in theversion
attribute.Here's an advanced example showcasing how templates and Sprig functions can be used together with
targets
:targets:
# Vendor a component into a major-minor versioned folder like 1.2
- "components/terraform/infra/vpc-flow-logs-bucket/{{ (first 2 (splitList \".\" .Version)) | join \".\" }}"tags
The
tags
in each source specifies a list of tags to apply to the component. This allows you to only vendor the components that have the specified tags by executing a commandatmos vendor pull --tags <tag1>,<tag2>
imports
The
imports
section defines the additional vendoring manifests that are merged into the main manifest. Hierarchical imports are supported at many levels (one vendoring manifest can import another, which in turn can import other manifests, etc.). Atmos processes all imports and all sources in the imported manifests in the order they are defined.noteThe imported file extensions are optional. Imports that do not include file extensions will default to the
.yaml
extension.vendor.yaml
warningThe
glob
library that Atmos uses to download remote artifacts does not treat the double-star**
as including sub-folders. If the component's folder has sub-folders, and you need to vendor them, they have to be explicitly defined as in the following example.
Hierarchical Imports in Vendoring Manifests
Use imports
to split the main vendor.yaml
manifest into smaller files for maintainability, or by their roles in the infrastructure.
For example, import separate manifests for networking, security, data management, CI/CD, and other layers:
imports:
- "layers/networking"
- "layers/security"
- "layers/data"
- "layers/analytics"
- "layers/firewalls"
- "layers/cicd"
Hierarchical imports are supported at many levels. For example, consider the following vendoring configurations:
vendor.yaml
vendor/vendor2.yaml
vendor/vendor4.yaml
When you execute the atmos vendor pull
command, Atmos processes the import chain and the sources in the imported manifests in the order they
are defined:
- First, the main
vendor.yaml
file is read based on search paths - The
vendor/vendor2
andvendor/vendor3
manifests (defined in the mainvendor.yaml
file) are imported - The
vendor/vendor2
file is processed, and thevendor/vendor4
manifest is imported - The
vendor/vendor4
file is processed, and thevendor/vendor5
manifest is imported - Etc.
- Then all the sources from all the imported manifests are processed and the artifacts are downloaded into the paths defined by the
targets
Vendoring from OCI Registries
Atmos supports vendoring from OCI registries.
To specify a repository in an OCI registry, use the oci://<registry>/<repository>:tag
scheme.
Artifacts from OCI repositories are downloaded as Docker image tarballs, then all the layers are processed, un-tarred and un-compressed,
and the files are written into the directories specified by the targets
attribute of each source
.
For example, to vendor the vpc
component from the public.ecr.aws/cloudposse/components/terraform/stable/aws/vpc
AWS public ECR registry, use the following source
:
source: "oci://public.ecr.aws/cloudposse/components/terraform/stable/aws/vpc:latest"
The schema of a vendor.yaml
manifest is as follows:
vendor.yaml
To vendor the vpc
component, execute the following command: