Organizational Structure Configuration
The Organizational Structure Configuration pattern describes core concepts and best practices to structure and organize components and stacks to design for organizational complexity and provision multi-account enterprise-grade environments.
The pattern is frequently used to model multi-region infrastructures for organizations with multiple organizational units/departments/tenants and multiple accounts.
Use-cases
Use the Organizational Structure Configuration pattern when:
-
You have one or more organizations with multiple organizational units/departments/tenants
-
Each OU/department/tenant has multiple accounts
-
You want to provision the infrastructure into many regions
Benefits
The Organizational Structure Configuration pattern provides the following benefits:
-
The defaults for the components, organizations, tenants/OUs, regions, account/stages are defined in just one place (and then imported) making the entire configuration extremely DRY, reusable, and easy to understand and manage
-
New Organizations can be easily added and configured without affecting the configurations for the existing organizations
-
New tenants/OUs can be easily added to an organization without affecting the configurations for the existing tenants/OUs
-
New stages/accounts can be easily added to a tenant/OU without affecting the configurations for the existing stages/accounts
-
New regions can be added to the infrastructure, configured with Atmos, and components provisioned into the regions. New regions can be used for te general infrastructure, or for disaster recovery (DR), or for compliance and audit
-
After adding new organizations, tenants, accounts or regions, the entire configuration will still remain DRY, reusable and easy to manage thanks to using the described folder structure, catalogs, mixins, and hierarchical imports
Example
The following example shows the Atmos stack and component configurations to provision the vpc and vpc-flow-logs-bucket components into
a multi-org, multi-tenant, multi-account, multi-region environment. There are two organizations (org1 and org2) with two
OUs/tenants (core and plat), multiple accounts in each OU/tenant, and two regions (us-east-2 and us-west-2).
│ # Centralized stacks configuration (stack manifests)
├── stacks
│ ├── catalog # component-specific defaults
│ │ ├── vpc-flow-logs-bucket
│ │ │ └── defaults.yaml
│ │ └── vpc
│ │ └── defaults.yaml
│ ├── mixins
│ │ ├── tenant # tenant-specific defaults
│ │ │ ├── core.yaml
│ │ │ └── plat.yaml
│ │ ├── region # region-specific defaults
│ │ │ ├── global-region.yaml
│ │ │ ├── us-east-2.yaml
│ │ │ └── us-west-2.yaml
│ │ └── stage # stage-specific defaults
│ │ ├── audit.yaml
│ │ ├── automation.yaml
│ │ ├── identity.yaml
│ │ ├── root.yaml
│ │ ├── dev.yaml
│ │ ├── staging.yaml
│ │ └── prod.yaml
│ └── orgs # Organizations
│ ├── org1
│ │ ├── _defaults.yaml
│ │ ├── core # 'core' represents the "Core" OU (a.k.a tenant)
│ │ │ ├── _defaults.yaml
│ │ │ ├── audit
│ │ │ │ ├── _defaults.yaml
│ │ │ │ ├── global-region.yaml
│ │ │ │ ├── us-east-2.yaml
│ │ │ │ └── us-west-2.yaml
│ │ │ ├── automation
│ │ │ │ ├── _defaults.yaml
│ │ │ │ ├── global-region.yaml
│ │ │ │ ├── us-east-2.yaml
│ │ │ │ └── us-west-2.yaml
│ │ │ ├── identity
│ │ │ │ ├── _defaults.yaml
│ │ │ │ ├── global-region.yaml
│ │ │ │ ├── us-east-2.yaml
│ │ │ │ └── us-west-2.yaml
│ │ │ └── root
│ │ │ ├── _defaults.yaml
│ │ │ ├── global-region.yaml
│ │ │ ├── us-east-2.yaml
│ │ │ └── us-west-2.yaml
│ │ └── plat # 'plat' represents the "Platform" OU (a.k.a tenant)
│ │ ├── _defaults.yaml
│ │ ├── dev
│ │ │ ├── _defaults.yaml
│ │ │ ├── global-region.yaml
│ │ │ ├── us-east-2.yaml
│ │ │ └── us-west-2.yaml
│ │ ├── staging
│ │ │ ├── _defaults.yaml
│ │ │ ├── global-region.yaml
│ │ │ ├── us-east-2.yaml
│ │ │ └── us-west-2.yaml
│ │ └── prod
│ │ ├── _defaults.yaml
│ │ ├── global-region.yaml
│ │ ├── us-east-2.yaml
│ │ └── us-west-2.yaml
│ └── org2
│ ├── _defaults.yaml
│ ├── core # 'core' represents the "Core" OU (a.k.a tenant)
│ │ ├── _defaults.yaml
│ │ ├── audit
│ │ ├── automation
│ │ ├── identity
│ │ └── root
│ └── plat # 'plat' represents the "Platform" OU (a.k.a tenant)
│ ├── _defaults.yaml
│ ├── dev
│ ├── staging
│ └── prod
│
│ # Centralized components configuration
└── components
└── terraform # Terraform components (a.k.a Terraform "root" modules)
├── vpc
├── vpc-flow-logs-bucket
├── < other components >
Add the following minimal configuration to atmos.yaml CLI config file :
components:
terraform:
base_path: "components/terraform"
stacks:
base_path: "stacks"
included_paths:
# Tell Atmos to search for the top-level stack manifests in the `orgs` folder and its sub-folders
- "orgs/**/*"
excluded_paths:
# Tell Atmos that all `_defaults.yaml` files are not top-level stack manifests
- "**/_defaults.yaml"
# If you are using multiple organizations (namespaces), use the following `name_pattern`:
name_pattern: "{namespace}-{tenant}-{environment}-{stage}"
# If you are using a single organization (namespace), use the following `name_pattern`:
# name_pattern: "{tenant}-{environment}-{stage}"
schemas:
jsonschema:
base_path: "stacks/schemas/jsonschema"
opa:
base_path: "stacks/schemas/opa"
atmos:
manifest: "stacks/schemas/atmos/atmos-manifest/1.0/atmos-manifest.json"
Configure Component Catalogs
Add the following default configuration to the stacks/catalog/vpc-flow-logs-bucket/defaults.yaml manifest:
components:
terraform:
vpc-flow-logs-bucket:
metadata:
# Point to the Terraform component
component: vpc-flow-logs-bucket
vars:
enabled: true
name: "vpc-flow-logs"
traffic_type: "ALL"
force_destroy: true
lifecycle_rule_enabled: false
Add the following default configuration to the stacks/catalog/vpc/defaults.yaml manifest:
components:
terraform:
vpc:
metadata:
# Point to the Terraform component
component: vpc
settings:
# All validation steps must succeed to allow the component to be provisioned
validation:
validate-vpc-component-with-jsonschema:
schema_type: jsonschema
schema_path: "vpc/validate-vpc-component.json"
description: Validate 'vpc' component variables using JSON Schema
check-vpc-component-config-with-opa-policy:
schema_type: opa
schema_path: "vpc/validate-vpc-component.rego"
module_paths:
- "catalog/constants"
description: Check 'vpc' component configuration using OPA policy
vars:
enabled: true
name: "common"
max_subnet_count: 3
map_public_ip_on_launch: true
assign_generated_ipv6_cidr_block: false
nat_gateway_enabled: true
nat_instance_enabled: false
vpc_flow_logs_enabled: true
vpc_flow_logs_traffic_type: "ALL"
vpc_flow_logs_log_destination_type: "s3"
nat_eip_aws_shield_protection_enabled: false
subnet_type_tag_key: "acme/subnet/type"
ipv4_primary_cidr_block: 10.9.0.0/18
Configure OU/Tenant Manifests
In stacks/mixins/tenant/core.yaml, add the following config:
vars:
tenant: core
# Other defaults for the `core` tenant/OU
In stacks/mixins/tenant/plat.yaml, add the following config:
vars:
tenant: plat
# Other defaults for the `plat` tenant/OU
Configure Region Manifests
In stacks/mixins/region/us-east-2.yaml, add the following config:
import:
# All accounts (stages) in `us-east-2` region will have the `vpc-flow-logs-bucket` component
- catalog/vpc-flow-logs-bucket/defaults
# All accounts (stages) in `us-east-2` region will have the `vpc` component
- catalog/vpc/defaults
vars:
region: us-east-2
environment: ue2
# Other defaults for the `us-east-2` region
In stacks/mixins/region/us-west-2.yaml, add the following config:
import:
# All accounts (stages) in `us-west-2` region will have the `vpc-flow-logs-bucket` component
- catalog/vpc-flow-logs-bucket/defaults
# All accounts (stages) in `us-east-2` region will have the `vpc` component
- catalog/vpc/defaults
vars:
region: us-west-2
environment: uw2
# Other defaults for the `us-west-2` region
Configure Stage/Account Manifests
In stacks/mixins/stage/dev.yaml, add the following config:
vars:
stage: dev
# Other defaults for the `dev` stage/account
In stacks/mixins/stage/prod.yaml, add the following config:
vars:
stage: prod
# Other defaults for the `prod` stage/account
In stacks/mixins/stage/staging.yaml, add the following config:
vars:
stage: staging
# Other defaults for the `staging` stage/account
Configure Organization Defaults
The _defaults.yaml files in the organization, tenant, account and region folders is the recommended way to define the stack manifests with the
default configurations for organizations, OUs/tenants, accounts and regions. The _defaults.yaml files themselves are not top-level Atmos stacks,
they just contain the default values for the organizations, OUs/tenants, accounts and regions (to make the entire configuration reusable and DRY)
In stacks/orgs/org1/_defaults.yaml, add the following config:
vars:
namespace: org1
The file defines the context variable namespace for the entire org1 organization.
In stacks/orgs/org2/_defaults.yaml, add the following config:
vars:
namespace: org2
The file defines the context variable namespace for the entire org2 organization.
Configure Tenant/OU Defaults
In stacks/orgs/org1/core/_defaults.yaml, add the following config for the org1 organization and core OU (tenant):
import:
- orgs/org1/_defaults
- mixins/tenant/core
In stacks/orgs/org1/plat/_defaults.yaml, add the following config for the org1 organization and plat OU (tenant):
import:
- orgs/org1/_defaults
- mixins/tenant/plat
In stacks/orgs/org2/core/_defaults.yaml, add the following config for the org2 organization and core OU (tenant):
import:
- orgs/org2/_defaults
- mixins/tenant/core
In stacks/orgs/org2/plat/_defaults.yaml, add the following config for the org2 organization and plat OU (tenant):
import:
- orgs/org2/_defaults
- mixins/tenant/plat
Configure Stage/Account Defaults
In stacks/orgs/org1/plat/dev/_defaults.yaml, add the following config for the org1 organization, plat tenant, dev account:
import:
- orgs/org1/plat/_defaults
- mixins/stage/dev
In stacks/orgs/org2/plat/dev/_defaults.yaml, add the following config for the org2 organization, plat tenant, dev account:
import:
- orgs/org2/plat/_defaults
- mixins/stage/dev
In stacks/orgs/org1/plat/prod/_defaults.yaml, add the following config for the org1 organization, plat tenant, prod account:
import:
- orgs/org1/plat/_defaults
- mixins/stage/prod
In stacks/orgs/org2/plat/prod/_defaults.yaml, add the following config for the org2 organization, plat tenant, prod account:
import:
- orgs/org2/plat/_defaults
- mixins/stage/prod
Similarly, configure the defaults for the other accounts in the core and plat tenants in the org1 and org2 organizations.
Configure Top-Level Stack Manifests
After we've configured the catalog for the components, the mixins for the tenants, regions and stages, and the defaults for the organizations, OUs and accounts, the final step is to configure the Atmos root (top-level) stacks and the Atmos components in the stacks.
In stacks/orgs/org1/plat/dev/us-east-2.yaml, add the following config:
import:
- orgs/org1/plat/dev/_defaults
- mixins/region/us-east-2
In stacks/orgs/org1/plat/dev/us-west-2.yaml, add the following config:
import:
- orgs/org1/plat/dev/_defaults
- mixins/region/us-west-2
In stacks/orgs/org1/plat/prod/us-east-2.yaml, add the following config:
import:
- orgs/org1/plat/prod/_defaults
- mixins/region/us-east-2
In stacks/orgs/org1/plat/prod/us-west-2.yaml, add the following config:
import:
- orgs/org1/plat/prod/_defaults
- mixins/region/us-west-2
In stacks/orgs/org1/plat/staging/us-east-2.yaml, add the following config:
import:
- orgs/org1/plat/staging/_defaults
- mixins/region/us-east-2
In stacks/orgs/org1/plat/staging/us-west-2.yaml, add the following config:
import:
- orgs/org1/plat/staging/_defaults
- mixins/region/us-west-2
Similarly, configure the top-level stack manifests for the org2 organization.
Provision the Atmos Components in the Stacks
To provision the components in the org1 organization, execute the following commands:
# `dev` account, `us-east-2` region
atmos terraform apply vpc-flow-logs-bucket -s org1-plat-ue2-dev
atmos terraform apply vpc -s org1-plat-ue2-dev
# `dev` account, `us-west-2` region
atmos terraform apply vpc-flow-logs-bucket -s org1-plat-uw2-dev
atmos terraform apply vpc -s org1-plat-uw2-dev
# `staging` account, `us-east-2` region
atmos terraform apply vpc-flow-logs-bucket -s org1-plat-ue2-staging
atmos terraform apply vpc -s org1-plat-ue2-staging
# `staging` account, `us-west-2` region
atmos terraform apply vpc-flow-logs-bucket -s org1-plat-uw2-staging
atmos terraform apply vpc -s org1-plat-uw2-staging
# `prod` account, `us-east-2` region
atmos terraform apply vpc-flow-logs-bucket -s org1-plat-ue2-prod
atmos terraform apply vpc -s org1-plat-ue2-prod
# `prod` account, `us-west-2` region
atmos terraform apply vpc-flow-logs-bucket -s org1-plat-uw2-prod
atmos terraform apply vpc -s org1-plat-uw2-prod
To provision the components in the org2 organization, execute the following commands:
# `dev` account, `us-east-2` region
atmos terraform apply vpc-flow-logs-bucket -s org2-plat-ue2-dev
atmos terraform apply vpc -s org2-plat-ue2-dev
# `dev` account, `us-west-2` region
atmos terraform apply vpc-flow-logs-bucket -s org2-plat-uw2-dev
atmos terraform apply vpc -s org2-plat-uw2-dev
# `staging` account, `us-east-2` region
atmos terraform apply vpc-flow-logs-bucket -s org2-plat-ue2-staging
atmos terraform apply vpc -s org2-plat-ue2-staging
# `staging` account, `us-west-2` region
atmos terraform apply vpc-flow-logs-bucket -s org2-plat-uw2-staging
atmos terraform apply vpc -s org2-plat-uw2-staging
# `prod` account, `us-east-2` region
atmos terraform apply vpc-flow-logs-bucket -s org2-plat-ue2-prod
atmos terraform apply vpc -s org2-plat-ue2-prod
# `prod` account, `us-west-2` region
atmos terraform apply vpc-flow-logs-bucket -s org2-plat-uw2-prod
atmos terraform apply vpc -s org2-plat-uw2-prod
Limitations
The Organizational Structure Configuration pattern has the following limitations and drawbacks:
- The configuration described by the pattern can be complex for very simple infrastructures (e.g. just one organization, one organizational unit, a few accounts and regions)
Even if you are just starting with a very simple infrastructure (e.g. just one organization, one organizational unit, a few accounts, one or a few regions), it's still recommended that you start with the configuration described by the Organizational Structure Configuration Design Pattern.
In the future, when you modify your infrastructure to provision multi-organization, multi-account, multi-region environments at scale, it will be easy to extend the configuration without changing anything for the exiting environments.
Related Patterns
- Inline Component Configuration
- Inline Component Customization
- Component Catalog
- Component Catalog with Mixins
- Component Catalog Template
- Component Inheritance
- Partial Component Configuration
- Partial Stack Configuration
- Layered Stack Configuration