Unveiling the Magic: How Terraform Works Under the Hood to Manage Your Infrastructure as Code

I talked about Infrastructure as Code (IaC) in my last piece. Since I am the project manager, team lead, and junior developer on my portfolio project, I thoroughly examined it and chose to use the IaC approach to build each side project. Since I do the majority of my deployments in Azure, this led me to explore some of the infrastructure as code tools like Azure Resource Manager and Azure Biceps. Yes, given how common AWS is, some people utilize Azure like me.

Biceps became my go-to IaC tool until I discovered Terraform. In this post, I will explain how Terraform works under the hood to provision infrastructure.

What is Terraform

Terraform is an open-source infrastructure as code tool developed by HashiCorp for customizing and deploying your infrastructure. Terraform allows you to describe both cloud and on-premises resources in human-readable configuration files that you can version, reuse, and share.

Why Terraform

I loved Azure Biceps, but the major reasons I chose Terraform were multi-cloud capabilities, declarative configuration, and a large community.

• Multi-Cloud Support
  Terraform works with AWS, Azure, Google Cloud, and many other providers.
  Facilitates hybrid and multi-cloud deployments.

• Declarative Configuration
  Uses a declarative approach where you define the desired state, and Terraform ensures infrastructure matches it. This reduces complexity to imperative provisioning tools.

• Large Community & Ecosystem
  Terraform has Extensive community support and a growing ecosystem of providers and modules, with an
  Active development and contributions from HashiCorp.

How Terraform Works

Under the hood, Terraform works by following a structured process to deploy infrastructure based on the configuration files you write. These files describe the desired state of your infrastructure, and Terraform ensures that your actual infrastructure matches this state.

Let’s break down how Terraform works under the hood to deploy infrastructure:

To interact with different cloud platforms or services, Terraform uses providers. Providers are plugins that act as a bridge between Terraform and the APIs of cloud platforms or services.

Providers are responsible for understanding API interactions and exposing resources for a specific cloud or service. Each provider corresponds to a cloud platform (e.g., AWS, Azure, GCP) or a service (e.g., Kubernetes, GitHub).
When you write your configuration file, the Terraform form provider communicates with the cloud providers to which you deploy your resources by translating it into a language the Cloud API understands to create, update, delete, and manage infrastructure resources.

The Terraform community have already written thousands of providers to manage many different types of resources and services. You can find all publicly available providers on the Terraform Registry, including Amazon Web Services (AWS), Azure, Google Cloud Platform (GCP), Kubernetes, Helm, GitHub, Splunk, DataDog, and many more.

Examples of Terraform Providers:

1. Azure:
   • Proivder: azurerm
   • Used to manage Azure resources like virtual machines, storage accounts, and networks.
   • Example:

provider "azurerm" {
  features {}
}

resource "azurerm_resource_group" "example" {
  name     = "example-resources"
  location = "East US"
}

1. AWS:
   • Provider: aws
   • Used to manage AWS resources like EC2 instances, S3 buckets, and IAM roles.
   • Example:

provider "aws" {
  region = "us-east-1"
}

resource "aws_instance" "example" {
  ami           = "ami-0c55b159cbfafe1f0"
  instance_type = "t2.micro"
}

1. Google Cloud Provider(GCP):
   • Provider: google
   • Used to manage GCP resources like Compute Engine, Cloud Storage, and BigQuery.
   • Example:

provider "google" {
  project = "my-gcp-project"
  region  = "us-central1"
}

resource "google_storage_bucket" "example" {
  name     = "example-bucket"
  location = "US"
}

• Terraform providers are plugins that enable communication with cloud platforms or services.

• Each provider is configured with credentials and settings specific to the platform.

• Examples include azurerm for Azure, aws for AWS, google for GCP, and
  others like Kubernetes and GitHub.

By using providers, Terraform abstracts the complexity of interacting with different cloud APIs, making it easier to manage multi-cloud or hybrid infrastructure.

Key Terraform Actions Under the Hood:

1.Initialization (terraform init)
Providers and Modules: When you first initialize your Terraform project using terraform init, Terraform Core downloads the necessary provider plugins for the specific services you want to use (e.g., AWS, Azure, GCP); it downloads and installs any modules referenced in your configuration. This sets up the working directory to prepare for execution.

2.Plan (terraform plan)
State File: When you run terraform plan, Terraform compares the current state of your infrastructure (stored in the terraform.tfstate file) to the desired state defined in your configuration files.
Dependency Graph: Terraform builds a dependency graph of all resources defined in your configuration. This helps Terraform understand the correct order in which resources should be created, modified, or destroyed based on dependencies (e.g., a virtual machine needs a network to be created first).
Execution Plan: Terraform then generates an execution plan. The plan details what resources will be created, modified, or deleted, and in which order. The plan is generated by comparing your configuration against the current state of your infrastructure. The execution plan is displayed for you to review and approve.

3.Apply (terraform apply)
Resource Creation/Modification: When you run terraform apply, Terraform executes the changes outlined in the plan: Terraform communicates with the cloud providers (e.g., AWS, Azure, GCP) using the appropriate provider plugins. Each provider has its own API that Terraform interacts with. Terraform sends API requests to the cloud provider’s API to create, modify, or delete resources according to your configuration. For example, if you're creating an EC2 instance on AWS, Terraform sends a request to the AWS API to launch the instance with the desired specifications (e.g., size, region, tags).
State File Update: As resources are created or modified, Terraform updates the state file (terraform.tfstate) to reflect the actual state of the infrastructure. Terraform relies on this state file to track provisioning, ensuring consistency for future runs.
Providers’ Role: Each provider is responsible for interacting with a specific infrastructure or service, using its own API. For example, the AWS provider interacts with the AWS API to provision resources like EC2 instances, VPCs, S3 buckets, etc. The Azure provider interacts with the Azure Resource Manager (ARM) API to provision resources like virtual machines, networks, etc.
Resource Dependencies: Terraform ensures that resources are created or modified in the correct order based on their dependencies. For instance, a virtual machine depends on a network to exist before it can be created.

4.Resource Change and Idempotency
Terraform ensures idempotency, meaning that running terraform apply multiple times with the same configuration should yield the same result. If nothing has changed in your .tf files, Terraform won’t re-create the resources. It will simply confirm that the infrastructure is already in the desired state. If the configuration has changed, Terraform will calculate the minimal changes required to bring the infrastructure in sync with the new configuration and apply those changes.

Terraform Workflow

1.Write:
Write Configuration: You define your desired infrastructure state in .tf files using HCL.
Initialize Terraform (terraform init): Initializes the project and downloads necessary providers and modules.

2.Plan (terraform plan):
Terraform compares the desired state (in your configuration) with the actual state (from the state file). It creates an execution plan to show which changes will be made (create, update, or delete resources).

3.Apply (terraform apply):
Terraform sends API requests to the cloud providers (AWS, Azure, etc.) to create, update, or delete resources as defined in the plan. Terraform updates the state file to reflect the changes made to the infrastructure.

Summary

Under the hood, Terraform is a tool that manages your infrastructure by defining the desired state in configuration files, comparing it to the current state (through the state file), and then applying changes by communicating with cloud provider APIs. The process involves initialization, creating execution plans, applying changes, and updating the state file to keep infrastructure consistent across different runs. The core logic ensures that Terraform operates in a consistent, idempotent way, always aiming to bring your infrastructure to the desired state.