Skip to main content

Organizational Structure Configuration

Atmos Design Pattern

The Organizational Structure Configuration pattern describes core concepts and best practices to structure and organize components and stacks to design for organizational complexity and provision multi-account enterprise-grade environments.

The pattern is frequently used to model multi-region infrastructures for organizations with multiple organizational units/departments/tenants and multiple accounts.

Use-cases

Use the Organizational Structure Configuration pattern when:

  • You have one or more organizations with multiple organizational units/departments/tenants

  • Each OU/department/tenant has multiple accounts

  • You want to provision the infrastructure into many regions

Benefits

The Organizational Structure Configuration pattern provides the following benefits:

  • The defaults for the components, organizations, tenants/OUs, regions, account/stages are defined in just one place (and then imported) making the entire configuration extremely DRY, reusable, and easy to understand and manage

  • New Organizations can be easily added and configured without affecting the configurations for the existing organizations

  • New tenants/OUs can be easily added to an organization without affecting the configurations for the existing tenants/OUs

  • New stages/accounts can be easily added to a tenant/OU without affecting the configurations for the existing stages/accounts

  • New regions can be added to the infrastructure, configured with Atmos, and components provisioned into the regions. New regions can be used for te general infrastructure, or for disaster recovery (DR), or for compliance and audit

  • After adding new organizations, tenants, accounts or regions, the entire configuration will still remain DRY, reusable and easy to manage thanks to using the described folder structure, catalogs, mixins, and hierarchical imports

Example

The following example shows the Atmos stack and component configurations to provision the vpc and vpc-flow-logs-bucket components into a multi-org, multi-tenant, multi-account, multi-region environment. There are two organizations (org1 and org2) with two OUs/tenants (core and plat), multiple accounts in each OU/tenant, and two regions (us-east-2 and us-west-2).

   │   # Centralized stacks configuration (stack manifests)
   ├── stacks
   │   ├── catalog # component-specific defaults
   │   │   ├── vpc-flow-logs-bucket
   │   │   │   └── defaults.yaml
   │   │   └── vpc
   │   │       └── defaults.yaml
   │   ├── mixins
   │   │   ├── tenant # tenant-specific defaults
   │   │   │   ├── core.yaml
   │   │   │   └── plat.yaml
   │   │   ├── region # region-specific defaults
   │   │   │   ├── global-region.yaml
   │   │   │   ├── us-east-2.yaml
   │   │   │   └── us-west-2.yaml
   │   │   └── stage # stage-specific defaults
   │   │       ├── audit.yaml
   │   │       ├── automation.yaml
   │   │       ├── identity.yaml
   │   │       ├── root.yaml
   │   │       ├── dev.yaml
   │   │       ├── staging.yaml
   │   │       └── prod.yaml
   │   └── orgs # Organizations
   │       ├── org1
   │       │   ├── _defaults.yaml
   │       │   ├── core # 'core' represents the "Core" OU (a.k.a tenant)
   │       │   │   ├── _defaults.yaml
   │       │   │   ├── audit
   │       │   │   │   ├── _defaults.yaml
   │       │   │   │   ├── global-region.yaml
   │       │   │   │   ├── us-east-2.yaml
   │       │   │   │   └── us-west-2.yaml
   │       │   │   ├── automation
   │       │   │   │   ├── _defaults.yaml
   │       │   │   │   ├── global-region.yaml
   │       │   │   │   ├── us-east-2.yaml
   │       │   │   │   └── us-west-2.yaml
   │       │   │   ├── identity
   │       │   │   │   ├── _defaults.yaml
   │       │   │   │   ├── global-region.yaml
   │       │   │   │   ├── us-east-2.yaml
   │       │   │   │   └── us-west-2.yaml
   │       │   │   └── root
   │       │   │       ├── _defaults.yaml
   │       │   │       ├── global-region.yaml
   │       │   │       ├── us-east-2.yaml
   │       │   │       └── us-west-2.yaml
   │       │   └── plat # 'plat' represents the "Platform" OU (a.k.a tenant)
   │       │       ├── _defaults.yaml
   │       │       ├── dev
   │       │       │   ├── _defaults.yaml
   │       │       │   ├── global-region.yaml
   │       │       │   ├── us-east-2.yaml
   │       │       │   └── us-west-2.yaml
   │       │       ├── staging
   │       │       │   ├── _defaults.yaml
   │       │       │   ├── global-region.yaml
   │       │       │   ├── us-east-2.yaml
   │       │       │   └── us-west-2.yaml
   │       │       └── prod
   │       │           ├── _defaults.yaml
   │       │           ├── global-region.yaml
   │       │           ├── us-east-2.yaml
   │       │           └── us-west-2.yaml
   │       └── org2
   │           ├── _defaults.yaml
   │           ├── core # 'core' represents the "Core" OU (a.k.a tenant)
   │           │   ├── _defaults.yaml
   │           │   ├── audit
   │           │   ├── automation
   │           │   ├── identity
   │           │   └── root
   │           └── plat # 'plat' represents the "Platform" OU (a.k.a tenant)
   │               ├── _defaults.yaml
   │               ├── dev
   │               ├── staging
   │               └── prod
   │  
   │   # Centralized components configuration
   └── components
       └── terraform # Terraform components (a.k.a Terraform "root" modules)
           ├── vpc
           ├── vpc-flow-logs-bucket
           ├── < other components >

Add the following minimal configuration to atmos.yaml CLI config file :

atmos.yaml
components:
terraform:
base_path: "components/terraform"

stacks:
base_path: "stacks"
included_paths:
# Tell Atmos to search for the top-level stack manifests in the `orgs` folder and its sub-folders
- "orgs/**/*"
excluded_paths:
# Tell Atmos that all `_defaults.yaml` files are not top-level stack manifests
- "**/_defaults.yaml"
# If you are using multiple organizations (namespaces), use the following `name_pattern`:
name_pattern: "{namespace}-{tenant}-{environment}-{stage}"
# If you are using a single organization (namespace), use the following `name_pattern`:
# name_pattern: "{tenant}-{environment}-{stage}"

schemas:
jsonschema:
base_path: "stacks/schemas/jsonschema"
opa:
base_path: "stacks/schemas/opa"
atmos:
manifest: "stacks/schemas/atmos/atmos-manifest/1.0/atmos-manifest.json"

Configure Component Catalogs

Add the following default configuration to the stacks/catalog/vpc-flow-logs-bucket/defaults.yaml manifest:

stacks/catalog/vpc-flow-logs-bucket/defaults.yaml
components:
terraform:
vpc-flow-logs-bucket:
metadata:
# Point to the Terraform component
component: vpc-flow-logs-bucket
vars:
enabled: true
name: "vpc-flow-logs"
traffic_type: "ALL"
force_destroy: true
lifecycle_rule_enabled: false

Add the following default configuration to the stacks/catalog/vpc/defaults.yaml manifest:

stacks/catalog/vpc/defaults.yaml
components:
terraform:
vpc:
metadata:
# Point to the Terraform component
component: vpc
settings:
# All validation steps must succeed to allow the component to be provisioned
validation:
validate-vpc-component-with-jsonschema:
schema_type: jsonschema
schema_path: "vpc/validate-vpc-component.json"
description: Validate 'vpc' component variables using JSON Schema
check-vpc-component-config-with-opa-policy:
schema_type: opa
schema_path: "vpc/validate-vpc-component.rego"
module_paths:
- "catalog/constants"
description: Check 'vpc' component configuration using OPA policy
vars:
enabled: true
name: "common"
max_subnet_count: 3
map_public_ip_on_launch: true
assign_generated_ipv6_cidr_block: false
nat_gateway_enabled: true
nat_instance_enabled: false
vpc_flow_logs_enabled: true
vpc_flow_logs_traffic_type: "ALL"
vpc_flow_logs_log_destination_type: "s3"
nat_eip_aws_shield_protection_enabled: false
subnet_type_tag_key: "acme/subnet/type"
ipv4_primary_cidr_block: 10.9.0.0/18

Configure OU/Tenant Manifests

In stacks/mixins/tenant/core.yaml, add the following config:

stacks/mixins/tenant/core.yaml
vars:
tenant: core

# Other defaults for the `core` tenant/OU

In stacks/mixins/tenant/plat.yaml, add the following config:

stacks/mixins/tenant/plat.yaml
vars:
tenant: plat

# Other defaults for the `plat` tenant/OU

Configure Region Manifests

In stacks/mixins/region/us-east-2.yaml, add the following config:

stacks/mixins/region/us-east-2.yaml
import:
# All accounts (stages) in `us-east-2` region will have the `vpc-flow-logs-bucket` component
- catalog/vpc-flow-logs-bucket/defaults
# All accounts (stages) in `us-east-2` region will have the `vpc` component
- catalog/vpc/defaults

vars:
region: us-east-2
environment: ue2

# Other defaults for the `us-east-2` region

In stacks/mixins/region/us-west-2.yaml, add the following config:

stacks/mixins/region/us-west-2.yaml
import:
# All accounts (stages) in `us-west-2` region will have the `vpc-flow-logs-bucket` component
- catalog/vpc-flow-logs-bucket/defaults
# All accounts (stages) in `us-east-2` region will have the `vpc` component
- catalog/vpc/defaults

vars:
region: us-west-2
environment: uw2

# Other defaults for the `us-west-2` region

Configure Stage/Account Manifests

In stacks/mixins/stage/dev.yaml, add the following config:

stacks/mixins/stage/dev.yaml
vars:
stage: dev

# Other defaults for the `dev` stage/account

In stacks/mixins/stage/prod.yaml, add the following config:

stacks/mixins/stage/prod.yaml
vars:
stage: prod

# Other defaults for the `prod` stage/account

In stacks/mixins/stage/staging.yaml, add the following config:

stacks/mixins/stage/staging.yaml
vars:
stage: staging

# Other defaults for the `staging` stage/account

Configure Organization Defaults

note

The _defaults.yaml files in the organization, tenant, account and region folders is the recommended way to define the stack manifests with the default configurations for organizations, OUs/tenants, accounts and regions. The _defaults.yaml files themselves are not top-level Atmos stacks, they just contain the default values for the organizations, OUs/tenants, accounts and regions (to make the entire configuration reusable and DRY)

In stacks/orgs/org1/_defaults.yaml, add the following config:

stacks/orgs/org1/_defaults.yaml
vars:
namespace: org1

The file defines the context variable namespace for the entire org1 organization.

In stacks/orgs/org2/_defaults.yaml, add the following config:

stacks/orgs/org2/_defaults.yaml
vars:
namespace: org2

The file defines the context variable namespace for the entire org2 organization.

Configure Tenant/OU Defaults

In stacks/orgs/org1/core/_defaults.yaml, add the following config for the org1 organization and core OU (tenant):

stacks/orgs/org1/core/_defaults.yaml
import:
- orgs/org1/_defaults
- mixins/tenant/core

In stacks/orgs/org1/plat/_defaults.yaml, add the following config for the org1 organization and plat OU (tenant):

stacks/orgs/org1/plat/_defaults.yaml
import:
- orgs/org1/_defaults
- mixins/tenant/plat

In stacks/orgs/org2/core/_defaults.yaml, add the following config for the org2 organization and core OU (tenant):

stacks/orgs/org2/core/_defaults.yaml
import:
- orgs/org2/_defaults
- mixins/tenant/core

In stacks/orgs/org2/plat/_defaults.yaml, add the following config for the org2 organization and plat OU (tenant):

stacks/orgs/org2/plat/_defaults.yaml
import:
- orgs/org2/_defaults
- mixins/tenant/plat

Configure Stage/Account Defaults

In stacks/orgs/org1/plat/dev/_defaults.yaml, add the following config for the org1 organization, plat tenant, dev account:

stacks/orgs/org1/plat/dev/_defaults.yaml
import:
- orgs/org1/plat/_defaults
- mixins/stage/dev

In stacks/orgs/org2/plat/dev/_defaults.yaml, add the following config for the org2 organization, plat tenant, dev account:

stacks/orgs/org2/plat/dev/_defaults.yaml
import:
- orgs/org2/plat/_defaults
- mixins/stage/dev

In stacks/orgs/org1/plat/prod/_defaults.yaml, add the following config for the org1 organization, plat tenant, prod account:

stacks/orgs/org1/plat/prod/_defaults.yaml
import:
- orgs/org1/plat/_defaults
- mixins/stage/prod

In stacks/orgs/org2/plat/prod/_defaults.yaml, add the following config for the org2 organization, plat tenant, prod account:

stacks/orgs/org2/plat/prod/_defaults.yaml
import:
- orgs/org2/plat/_defaults
- mixins/stage/prod

Similarly, configure the defaults for the other accounts in the core and plat tenants in the org1 and org2 organizations.

Configure Top-Level Stack Manifests

After we've configured the catalog for the components, the mixins for the tenants, regions and stages, and the defaults for the organizations, OUs and accounts, the final step is to configure the Atmos root (top-level) stacks and the Atmos components in the stacks.

In stacks/orgs/org1/plat/dev/us-east-2.yaml, add the following config:

stacks/orgs/org1/plat/dev/us-east-2.yaml
import:
- orgs/org1/plat/dev/_defaults
- mixins/region/us-east-2

In stacks/orgs/org1/plat/dev/us-west-2.yaml, add the following config:

stacks/orgs/org1/plat/dev/us-west-2.yaml
import:
- orgs/org1/plat/dev/_defaults
- mixins/region/us-west-2

In stacks/orgs/org1/plat/prod/us-east-2.yaml, add the following config:

stacks/orgs/org1/plat/prod/us-east-2.yaml
import:
- orgs/org1/plat/prod/_defaults
- mixins/region/us-east-2

In stacks/orgs/org1/plat/prod/us-west-2.yaml, add the following config:

stacks/orgs/org1/plat/prod/us-west-2.yaml
import:
- orgs/org1/plat/prod/_defaults
- mixins/region/us-west-2

In stacks/orgs/org1/plat/staging/us-east-2.yaml, add the following config:

stacks/orgs/org1/plat/staging/us-east-2.yaml
import:
- orgs/org1/plat/staging/_defaults
- mixins/region/us-east-2

In stacks/orgs/org1/plat/staging/us-west-2.yaml, add the following config:

stacks/orgs/org1/plat/staging/us-west-2.yaml
import:
- orgs/org1/plat/staging/_defaults
- mixins/region/us-west-2

Similarly, configure the top-level stack manifests for the org2 organization.

Provision the Atmos Components in the Stacks

To provision the components in the org1 organization, execute the following commands:

# `dev` account, `us-east-2` region
atmos terraform apply vpc-flow-logs-bucket -s org1-plat-ue2-dev
atmos terraform apply vpc -s org1-plat-ue2-dev

# `dev` account, `us-west-2` region
atmos terraform apply vpc-flow-logs-bucket -s org1-plat-uw2-dev
atmos terraform apply vpc -s org1-plat-uw2-dev

# `staging` account, `us-east-2` region
atmos terraform apply vpc-flow-logs-bucket -s org1-plat-ue2-staging
atmos terraform apply vpc -s org1-plat-ue2-staging

# `staging` account, `us-west-2` region
atmos terraform apply vpc-flow-logs-bucket -s org1-plat-uw2-staging
atmos terraform apply vpc -s org1-plat-uw2-staging

# `prod` account, `us-east-2` region
atmos terraform apply vpc-flow-logs-bucket -s org1-plat-ue2-prod
atmos terraform apply vpc -s org1-plat-ue2-prod

# `prod` account, `us-west-2` region
atmos terraform apply vpc-flow-logs-bucket -s org1-plat-uw2-prod
atmos terraform apply vpc -s org1-plat-uw2-prod

To provision the components in the org2 organization, execute the following commands:

# `dev` account, `us-east-2` region
atmos terraform apply vpc-flow-logs-bucket -s org2-plat-ue2-dev
atmos terraform apply vpc -s org2-plat-ue2-dev

# `dev` account, `us-west-2` region
atmos terraform apply vpc-flow-logs-bucket -s org2-plat-uw2-dev
atmos terraform apply vpc -s org2-plat-uw2-dev

# `staging` account, `us-east-2` region
atmos terraform apply vpc-flow-logs-bucket -s org2-plat-ue2-staging
atmos terraform apply vpc -s org2-plat-ue2-staging

# `staging` account, `us-west-2` region
atmos terraform apply vpc-flow-logs-bucket -s org2-plat-uw2-staging
atmos terraform apply vpc -s org2-plat-uw2-staging

# `prod` account, `us-east-2` region
atmos terraform apply vpc-flow-logs-bucket -s org2-plat-ue2-prod
atmos terraform apply vpc -s org2-plat-ue2-prod

# `prod` account, `us-west-2` region
atmos terraform apply vpc-flow-logs-bucket -s org2-plat-uw2-prod
atmos terraform apply vpc -s org2-plat-uw2-prod

Limitations

The Organizational Structure Configuration pattern has the following limitations and drawbacks:

  • The configuration described by the pattern can be complex for very simple infrastructures (e.g. just one organization, one organizational unit, a few accounts and regions)
note

Even if you are just starting with a very simple infrastructure (e.g. just one organization, one organizational unit, a few accounts, one or a few regions), it's still recommended that you start with the configuration described by the Organizational Structure Configuration Design Pattern.

In the future, when you modify your infrastructure to provision multi-organization, multi-account, multi-region environments at scale, it will be easy to extend the configuration without changing anything for the exiting environments.

References