Skip to main content

Terraform Providers

Terraform utilizes plugins known as providers for communication with cloud providers, SaaS providers, and various APIs.

In order for Terraform to install these providers, the corresponding Terraform configurations need to explicitly state what providers are required. Furthermore, certain providers require additional configuration, such as specifying endpoint URLs or cloud regions, before they can be used.

Provider Configuration in Terraform​

When working with Terraform, you specify provider configurations in your Terraform code. This involves declaring which providers your infrastructure requires and providing any necessary configuration parameters. These parameters may include endpoint URLs, cloud regions, access credentials, or any other provider-specific configuration parameters.

To declare a provider in Terraform, use a provider block within your Terraform configuration files, usually in a providers.tf file in the component (a.k.a. root module) directory. The provider block specifies the provider type and all the necessary configuration parameters.

Here's an AWS provider configuration example for a vpc component. The provider config is defined in the components/terraform/vpc/providers.tf file:

components/terraform/vpc/providers.tf

  provider "aws" {
    region = "us-east-2"
    assume_role {
      role_arn: "IAM Role ARN"
    }
  }

In this example, the aws provider block includes the region and IAM role required for Terraform to communicate with the AWS services.

By correctly defining provider configurations in your Terraform code, you ensure that Terraform can seamlessly install, configure, and use the necessary plugins to manage your infrastructure across various cloud and services.

Provider Configuration and Overrides in Atmos Manifests​

Atmos allows you to define and override provider configurations using the providers section in Atmos stack manifests. The section can be defined globally for the entire organization, OU/tenant, account, region, or per component.

For example, the providers section at the global scope can look like this:

stacks/orgs/acme/_defaults.yaml

terraform:
  providers:
    aws:
      region: "us-east-2"
      assume_role:
        role_arn: "IAM Role ARN"

Similarly, it can be defined (or overridden) at the OU/tenant, account and region scopes in the corresponding _defaults.yaml stack manifests.

If you want to override a provider configuration for a specific component, use the component.terraform.<component>.providers section. For example, the following config can be used to override the assume_role parameter just for the vpc component:

stacks/catalog/vpc/defaults.yaml

components:
  terraform:
    vpc:
      providers:
        aws:
          assume_role:
            role_arn: "IAM Role ARN for VPC"

You can include the providers sections in any Atmos stack manifest at any level of inheritance. Atmos will process, deep-merge and override all the providers configurations for a component in the following order:

  • Global scopes (terraform.providers sections for the Org, OUs, accounts and regions)
  • Base component scope (component.terraform.<base_component>.providers section)
  • Current component scope (component.terraform.<component>.providers section)
tip

Refer to Atmos Component Inheritance for more information on all types of component inheritance supported by Atmos

When you define the providers sections, Atmos processes the inheritance chain for a component and generates a file providers_override.tf.json in the component's folder with the final values for all the defined providers.

For example:

atmos terraform plan vpc -s plat-ue2-prod --logs-level=Trace

> atmos terraform plan vpc -s plat-ue2-prod --logs-level=Trace

Variables for the component 'vpc' in the stack 'plat-ue2-prod':
  environment: ue2
  max_subnet_count: 3
  name: common
  namespace: cp
  region: us-east-2
  stage: prod
  tenant: plat

Writing the variables to file:
components/terraform/vpc/plat-ue2-prod.terraform.tfvars.json

Writing the provider overrides to file:
components/terraform/vpc/providers_override.tf.json

The generated providers_override.tf.json file would look like this:

providers_override.tf.json

{
    "provider": {
      "aws": {
        "assume_role": {
          "role_arn": "IAM Role ARN for VPC"
        }
      }
    }
}

Terraform then uses the values in the generated providers_override.tf.json to override the parameters for all the providers in the file.

alias: Multiple Provider Configuration in Atmos Manifests​

Atmos allows you to define multiple configurations for the same provider using a list of provider blocks and the alias meta-argument.

The generated providers_override.tf.json file will have a list of provider configurations, and Terraform/OpenTofu will use and override the providers as long as the aliased providers are defined in the Terraform component.

For example:

stacks/catalog/vpc/defaults.yaml

components:
  terraform:
    vpc:
      providers:
        aws:
          - region: us-west-2
            assume_role:
              role_arn: "role-1"
          - region: us-west-2
            alias: "account-2"
            assume_role:
              role_arn: "role-2"
warning

The above example uses a list of configuration blocks for the aws provider.

Since it's a list, by default it doesn't work with deep-merging of stacks in the inheritance chain since list are not deep-merged, they are replaced.

If you want to use the above configuration in the inheritance chain and allow appending or merging of lists, consider configuring the settings.list_merge_strategy in the atmos.yaml CLI config file.

For more details, refer to Atmos CLI Settings.

Local Provider Development with Dev Overrides​

When developing custom Terraform providers locally, you need a way to test them without publishing to a registry. Terraform supports this through development overrides, which allows you to point Terraform to locally-built provider binaries.

Important Distinction

This section covers development overrides for locally-built provider binaries, which is separate from the provider configuration described in the sections above:

  • Provider Configuration (providers section in Atmos stacks) → Controls provider behavior (credentials, regions, endpoints)
  • Development Overrides (.terraformrc file) → Points to local provider binaries during development

Understanding Provider Configuration vs. Development Overrides​

The providers section in Atmos stack manifests (covered above) configures how providers behave - things like credentials, regions, and endpoints. This configuration is serialized to providers_override.tf.json and used by Terraform to override provider settings.

Development overrides work at a different level: they tell Terraform where to find the provider binary itself. This is configured in Terraform's CLI configuration file (.terraformrc or terraform.rc), not in your stack manifests.

Setting Up Development Overrides with Atmos​

To use locally-built providers with Atmos, follow these steps:

1. Create a Terraform CLI configuration file with dev_overrides in your component folder.

Create a .terraformrc file in your component directory (e.g., components/terraform/mycomponent/.terraformrc):

components/terraform/mycomponent/.terraformrc

provider_installation {
  dev_overrides {
    "registry.terraform.io/myorg/myprovider" = "/absolute/path/to/provider/binary/directory"
  }

  # For all other providers, install them directly as normal.
  direct {}
}

Replace myorg/myprovider with your provider's registry address and /absolute/path/to/provider/binary/directory with the absolute path to the directory containing your locally-built provider binary (not the binary itself).

Component-Specific Configuration

The .terraformrc file should be placed in each component folder that needs to use the local provider, not in your infrastructure repository root. This is because Terraform's CLI configuration is loaded relative to where Terraform runs, and Atmos executes Terraform from within the component directory.

Do not commit .terraformrc to version control - it contains developer-specific local paths. Add it to your component's .gitignore or your global .gitignore.

CLI Configuration Requirements

The provider_installation block with dev_overrides must be in a Terraform CLI configuration file (.terraformrc or terraform.rc). It cannot be defined in .tf files like providers.tf because it's a CLI-level setting, not infrastructure configuration.

2. Point Terraform to your CLI configuration.

Set the TF_CLI_CONFIG_FILE environment variable in your component's env section to tell Terraform where to find this configuration:

stacks/catalog/mycomponent/defaults.yaml

components:
  terraform:
    mycomponent:
      env:
        # Points to .terraformrc in the component directory
        # Atmos runs Terraform from the component folder, so this resolves correctly
        TF_CLI_CONFIG_FILE: ".terraformrc"
Relative Paths Work

Since Atmos executes Terraform from within the component directory, you can use a relative path like .terraformrc. Atmos will resolve it correctly to components/terraform/mycomponent/.terraformrc.

Alternatively, you can use an absolute path or ${PWD}/.terraformrc if you prefer explicit paths.

3. Build your provider.

Build your provider and place the binary in the directory you specified:

bash

cd /path/to/your/provider
go build -o /path/to/provider/binary/terraform-provider-myprovider

4. Run Atmos commands as usual.

Now when you run Atmos commands, Terraform will use your locally-built provider:

atmos terraform plan

atmos terraform plan mycomponent -s dev
Development Workflow

During active development, you can:

  1. Make changes to your provider code
  2. Rebuild the provider binary
  3. Run atmos terraform plan/apply immediately without publishing a release
  4. Iterate quickly on provider functionality

This eliminates the need to publish development versions to a registry during active development.

Using settings Section for Environment Variables​

You can also use the settings section to manage environment variables across your stack hierarchy:

stacks/orgs/acme/_defaults.yaml

terraform:
  settings:
    provider_dev_mode: true
    provider_dev_path: "/Users/developer/providers/bin"

components:
  terraform:
    mycomponent:
      env:
        TF_CLI_CONFIG_FILE: "${PWD}/.terraformrc"

Then reference these settings in your components as needed. However, note that the actual .terraformrc file configuration cannot be templated through Atmos - it must exist as a file that Terraform can read directly.

Dev Overrides Warning

When dev_overrides are active, Terraform will show a warning on every command:

Warning: Provider development overrides are in effect

This is expected behavior and confirms that your local provider is being used. Remove or comment out the dev_overrides block in .terraformrc when you're ready to use the published provider version.

Example: Complete Setup​

Here's a complete example for developing a custom provider:

Directory Structure:

infrastructure/
├── components/
│   └── terraform/
│       └── myapp/
│           ├── .terraformrc          # CLI config (not committed)
│           ├── .gitignore            # Ignore .terraformrc
│           ├── main.tf
│           ├── providers.tf
│           └── versions.tf
└── stacks/
    └── catalog/
        └── myapp/
            └── defaults.yaml

components/terraform/myapp/.terraformrc

provider_installation {
  dev_overrides {
    "registry.terraform.io/acme/cloud" = "/Users/developer/providers/bin"
  }
  direct {}
}

components/terraform/myapp/.gitignore

# Don't commit developer-specific CLI config
.terraformrc

stacks/catalog/myapp/defaults.yaml

components:
  terraform:
    myapp:
      # Configure the provider's behavior (credentials, endpoints, etc.)
      providers:
        acme:
          endpoint: "https://api.acme.com"
          api_token: "{{ .settings.acme_api_token }}"

      # Point to the CLI config in the component directory
      env:
        TF_CLI_CONFIG_FILE: ".terraformrc"

      # Regular component configuration
      vars:
        name: "my-application"

With this setup:

  • .terraformrc lives in the component folder where Terraform executes
  • Terraform finds your local provider binary via .terraformrc
  • Atmos configures the provider's behavior via the providers section
  • Your component works seamlessly with the local provider during development
  • The file is not committed, so each developer can have their own local paths

References​