Main-module approach for handling multiple environments in Terraform

Handling multiple environments in Terraform might be challenging and you can find many approaches and best practices on the Internet. Ths "main-module" approach is one of them using just built-in Terraform concepts (as an alternative to Terraform workspaces with Terraform Cloud/Enterprise or using wrapper tools like Terragrunt).

But why not just use “tfvars” files?
Because you cannot use variables in backend and provider configuration blocks.

Goals:

• Have separate backends for Terraform remote state-files per environment
• Be able to use separate system accounts for different environments
• Be able to use different versions of providers and Terraform itself per environment (and upgrade one by one)
• Ensure that all required properties are provided per environment (Terraform validate won't pass if an environmental property is missing)
• Ensure that all resources/modules are always added to all environments. It is not possible to "forget" about a whole module.

Here is a basic structure of a Terraform project with the main-module approach that worked for me and few other teams:

- terraform_project
  - env
     - dev01        <-- Terraform home, run from here 
        - .terraform    <-- git ignored of course
        - dev01.tf  <-- backend, env config, includes always _only_ the main module
     - dev02
        - .terraform
        - dev02.tf 
     - stg01
        - .terraform
        - stg01.tf
     - prd01
        - .terraform
        - prd01.tf 
  - main        <-- main umbrella module
     - main.tf
     - variables.tf         
  - modules         <-- submodules of the main module
     - module_a
     - module_b
     - module_c

And a real-life example screenshot:

env
Each env subdir is a Terraform "home".
Run terraform init/plan/apply in these directories. Change the environment by changing the directory. This prevents an error of merging state files of multiple environments.

Sample dev01.tf file (part):

provider "azurerm" {
  version = "~>1.42.0"
}

terraform {
  backend "azurerm" {
    resource_group_name  = "tradelens-host-rg"
    storage_account_name = "stterraformstate001"
    container_name       = "terraformstate"
    key                  = "dev.terraform.terraformstate"
  }
}

module "main" {
  source               = "../../main"
  subscription_id      = "000000000-0000-0000-0000-00000000000"
  project_name         = "tlens"
  environment_name     = "dev"
  resource_group_name  = "tradelens-main-dev"
  tenant_id            = "790fd69f-41a3-4b51-8a42-685767c7d8zz"
  location             = "West Europe"
  developers_object_id = "58968a05-dc52-4b69-a7df-ff99f01e12zz"
  terraform_sp_app_id  = "8afb2166-9168-4919-ba27-6f3f9dfad3ff"

  kubernetes_version      = "1.14.8"
  kuberenetes_vm_size     = "Standard_B2ms"
  kuberenetes_nodes_count = 4

  enable_ddos_protection = false
  enable_waf             = false
}

This gives a possibility to configure backend and provider per environment as well as provide all required parameters to the main module (i.e. the environment)

main

This is the main module that is an umbrella module for the whole Terraform project.

Having this module ensures that each environment will be as similar as possible to other environments. Adding a new resource or module to the main module will add this to all environments.

In order to implement differences between environments (pricing tiers etc.) use flags and parameters.

The variables.tf file lists the required parameters for an environment. Environment-specific settings.

Sample variables.tf file (part)

variable "subscription_id" {
  description = "Azure Subscription ID"
  type        = string
}

variable "project_name" {
  description = "Project name"
  type        = string
}

variable "environment_name" {
  description = "Environment name"
  type        = string
}

variable "resource_group_name" {
  description = "Target resource group name"
  type        = string
}

...

The main module can include submodules and resources. However, it is a good practice to divide it into a few submodules to introduce some encapsulation, increase readability and limit dependencies between resources.

Any shared remote resources will be included here.

A sample main.tf file (part):

module "eventhub" {
  source              = "../modules/eventhub"
  environment_name    = var.environment_name
  resource_group_name = var.resource_group_name
  project_name        = var.project_name
}

module "dremio" {
  source                   = "../modules/dremio"
  environment_name         = var.environment_name
  resource_group_name      = var.resource_group_name
  project_name             = var.project_name
  linux_admin              = "dremio"
  linux_admin_password     = "1b4d7c79-a143-45e2-8ff1-ee63c0823090"
  dremio_data_name         = module.dremio_storage.dremio_data_name
  dremio_data_id           = module.dremio_storage.dremio_data_id
  dremio_data_disk_size_gb = module.dremio_storage.dremio_data_disk_size_gb
}

module "dremio_storage" {
  source              = "../modules/dremio-storage"
  resource_group_name = var.resource_group_name
  project_name        = var.project_name
}


module "datalake" {
  source              = "../modules/datalake"
  resource_group_name = var.resource_group_name
  project_name        = var.project_name
  environment_name    = var.environment_name
}

This should be enough to visualize the concept.

Comments

[rob]

Nice article. In our projects it looks really very similar and we're using the same/comparable approach. It works fine for almost every situation.
The only problem we have is if we have bigger differences between environments. I know that there shouldn't be much differences except for example scaling and number of machines in a cluster etc. But sometimes we have one or the other resource which should only be deployed to prod for example. We're more or less fighting with count = 0 but that's a nasty workaround. Hope that there will be conditional modules sooner or later. Maybe i should take a look at Pulumi.

[thgsiddhimorajkar]

Hello, I am following this method to manage multiple environments. But when i give the source to the module the way it is done here, i get an error - "Error: Unreadable module directory

Unable to evaluate directory symlink: CreateFile modules: The system cannot
find the file specified.

Error: Failed to read module directory

Module directory does not exist or cannot be read."

What might be the issue?

[GSA]

Same here, some instructions are missing. You shall ru your code with terraform -chdir="./clients/client1" init, by passing directory while execution from root folder, not actually switching to individual client directory

[Eliise]

I'm using something similar, but the biggest issue I've got is managing the outputs. I've got about 30, which leads to lots of duplication. Do you have ideas?

[Piotr Gwiazda]

@eliises for some reason this did not notify me about comments. I just don't use outputs. Most of the time one terraform layer 'leaves' some information in Kay Vault etc, and another layer picks it using "data".