Just-in-Time Component Vendoring with source

Atmos now supports just-in-time (JIT) vendoring of components directly from stack configuration using the top-level source field. This works for Terraform, Helmfile, and Packer components. Declare component sources inline without requiring separate component.yaml files—components are automatically downloaded on first use.

What Changed

We've added automatic JIT source provisioning and new source command groups for terraform, helmfile, and packer. When you run any command, Atmos automatically provisions the component source if the directory doesn't exist:

components:
  terraform:
    vpc:
      source:
        uri: github.com/cloudposse/terraform-aws-components//modules/vpc
        version: 1.450.0
        included_paths:
          - "*.tf"
        excluded_paths:
          - "*.md"
          - "tests/**"

  helmfile:
    nginx:
      source:
        uri: github.com/cloudposse-archives/helmfiles//releases/nginx-ingress
        version: 0.126.0
        included_paths:
          - "*.yaml"

  packer:
    eks-ami:
      source:
        uri: github.com/aws-samples/amazon-eks-custom-amis
        version: main
        included_paths:
          - "*.pkr.hcl"

The new commands are available for each component type:

Terraform:

• atmos terraform source pull - Vendor a component from source configuration (use --force to re-vendor)
• atmos terraform source list - List components with source configured
• atmos terraform source describe - View source configuration
• atmos terraform source delete - Remove vendored source directory

Helmfile:

• atmos helmfile source pull
• atmos helmfile source list
• atmos helmfile source describe
• atmos helmfile source delete

Packer:

• atmos packer source pull
• atmos packer source list
• atmos packer source describe
• atmos packer source delete

Why This Matters

Previously, vendoring components required maintaining separate component.yaml files alongside your stack configuration. This created maintenance overhead and made it harder to:

1. Version per environment - Different environments often need different component versions
2. Keep configuration colocated - Source information was separated from component configuration
3. Vendor on demand - Components had to be pre-vendored before use

With source, you can now:

• Declare sources inline in your stack manifests
• Override versions per environment using stack inheritance
• Vendor just-in-time when you need the component
• Filter files with included_paths and excluded_paths

No Vendored Components Committed to Git

With source-based versioning, you no longer need to commit vendored component code to your Git repository if that's not adding value for your workflow. Components are downloaded just-in-time during execution instead of being pre-vendored and checked in. This means:

Benefits:

• Smaller repositories - No vendored Terraform code bloating your git history
• Faster clones - Particularly helpful for large component libraries
• Simplified PRs - Version changes are single-line config updates, not large diffs

Tradeoffs:

• No local immutable copy - Components aren't in your repo, so you lose the audit trail in Git
• No pre-deploy diff review - Can't review component code changes before deployment
• AI coding assistants lack context - Tools like Claude Code, Cursor, and GitHub Copilot work significantly better with vendored code—they have full context of your components for navigation, understanding dependencies, and accurate suggestions
• Network dependency - Requires network access during execution

For teams that value having vendored code committed for audit trails, code review, AI-assisted development, or offline deployments, traditional vendoring remains the right choice. Source-based versioning is ideal when the operational overhead of vendoring outweighs its benefits.

How to Use It

Automatic Provisioning (Default)

Add source to your component configuration:

# stacks/dev.yaml
components:
  terraform:
    vpc:
      source:
        uri: github.com/cloudposse/terraform-aws-components//modules/vpc
        version: 1.450.0
      vars:
        cidr_block: "10.0.0.0/16"

Then just run terraform—the source is provisioned automatically:

atmos terraform plan vpc --stack dev
# → Auto-provisions source for component 'vpc'
# → Successfully auto-provisioned source to components/terraform/vpc
# → Terraform runs

Explicit CLI Commands

For fine-grained control, use the source CLI commands:

atmos terraform source pull vpc --stack dev

Version Per Environment

Use stack inheritance to set different versions per environment:

# stacks/catalog/vpc/defaults.yaml
components:
  terraform:
    vpc/defaults:
      source:
        uri: github.com/cloudposse/terraform-aws-components//modules/vpc
        version: 1.450.0

# stacks/dev.yaml - use latest
components:
  terraform:
    vpc:
      metadata:
        inherits: [vpc/defaults]
      source:
        version: 1.451.0  # Override for dev

# stacks/prod.yaml - pin to stable
components:
  terraform:
    vpc:
      metadata:
        inherits: [vpc/defaults]
      # Uses inherited version 1.450.0

Supported Source Types

The source provisioner uses go-getter and supports:

• Git repositories (GitHub, GitLab, Bitbucket)
• S3 buckets
• GCS buckets
• HTTP/HTTPS URLs
• OCI registries

Combining with Workdir Isolation

The source provisioner works seamlessly with the workdir provisioner for concurrent operations across all component types (terraform, helmfile, and packer). When both are enabled, you get the best of both worlds:

1. Source provisioner vendors the component from a remote location
2. Workdir provisioner creates an isolated execution directory

Why Combine Them?

When running terraform plan on the same component across multiple stacks concurrently, you need isolation to prevent conflicts in .terraform/ directories and state files. The workdir provisioner provides this isolation.

With remote sources, different stacks might use different versions of the same component. The combination ensures:

• Each stack gets the correct component version.
• Each execution runs in its own isolated directory.
• No conflicts between concurrent operations.

Configuration Example

Enable both source and provision.workdir for a component:

components:
  terraform:
    vpc:
      source:
        uri: github.com/cloudposse/terraform-aws-components//modules/vpc
        version: 1.450.0
      provision:
        workdir:
          enabled: true
      vars:
        cidr_block: "10.0.0.0/16"

Workflow

With both source and workdir enabled, everything happens automatically:

# Just run terraform - both source and workdir are provisioned automatically
atmos terraform plan vpc -s dev
# → Source provisioner: downloads to .workdir/terraform/dev-vpc/ (directly to workdir)
# → Terraform runs in isolated workdir

When source and workdir are both configured, the source provisioner downloads directly to the workdir location (.workdir/terraform/<stack>-<component>/), skipping the intermediate component directory.

For explicit control, you can still use the CLI:

# Explicit source pull (without workdir, goes to components/terraform/vpc/)
atmos terraform source pull vpc --stack dev

This means you can safely run multiple stacks in parallel:

# These can run concurrently without conflicts
atmos terraform plan vpc -s dev &
atmos terraform plan vpc -s staging &
atmos terraform plan vpc -s prod &
wait

Directory Structure

After running the above commands, your project looks like:

project/
├── components/
│   ├── terraform/
│   │   └── vpc/                    # Vendored by source pull
│   ├── helmfile/
│   │   └── nginx/                  # Vendored by source pull
│   └── packer/
│       └── eks-ami/                # Vendored by source pull
├── .workdir/                       # Created when workdir enabled
│   ├── terraform/
│   │   ├── dev-vpc/               # Isolated for dev stack
│   │   ├── staging-vpc/           # Isolated for staging stack
│   │   └── prod-vpc/              # Isolated for prod stack
│   ├── helmfile/
│   │   └── dev-nginx/             # Isolated for dev stack
│   └── packer/
│       └── dev-eks-ami/           # Isolated for dev stack
└── stacks/
    ├── dev.yaml
    ├── staging.yaml
    └── prod.yaml

Get Involved

We'd love to hear your feedback on this feature! Please open an issue if you have questions or suggestions.

For more details, see the source documentation for terraform, helmfile, and packer, as well as the Source-Based Version Pinning design pattern.