Terraform manages the infrastructure resources and deployment using the state file. By running the refresh command, you can update the state file with the actual infrastructure configuration.
As of Terraform version v0.15.4, the terraform refresh command was deprecated because its default behavior could be deemed unsafe if you have misconfigured credentials for any of your providers.
In this blog, we will explore the terraform refresh command and how it works, and also discuss its limitations and alternatives through the use of practical hands-on examples.
Disclaimer : All use cases of the Terraform
refreshdiscussed here work similarly in OpenTofu, the open-source Terraform alternative. However, to keep it simple and familiar for DevOps engineers, we will use “Terraform refresh” as a catch-all term throughout this blog post.
What is Terraform Refresh
The terraform refresh command ensures that your state file reflects the current state of your infrastructure resources.
Let’s say that the resources managed by the Terraform code are sometimes modified using a console, CLI, third-party software APIs, or scripts. This means that the current infrastructure configuration will not match your Terraform code, creating drift because configuration changes were made outside of the regular code-to-cloud CI/CD pipeline.
One way to resolve these drifts is by running refresh, which updates the state file with the actual infrastructure configuration. However, this approach comes with several downsides and is generally not considered a best practice due to some of the reasons we’ll describe below.
Before getting to that, here are some scenarios in which you might need to run the terraform refresh command.
- Syncing the state: As mentioned, when the state file is out of sync with the current infrastructure, you can run the
refreshcommand to reconcile the differences between the state file and its actual state (drifts). - Before running Terraform plan: Running
refreshbeforeterraform planensures that the plan is based on the most recent resource configuration changes made in your cloud environment.
The syntax for the terraform refresh command is:
terraform refresh [options]
And the options could be any of the following:
-
-state=<path>: Specifies the path to the state file. Unless specified, it is the default path to the terraform.tfstate file. -
-state-out=<path>: Using this, you can define where you want to store your refreshed state file. If the value is not passed, it overwrites the existing state file. -
-lock=<true|false>: When the state is refreshed, it helps define whether the state lock should be acquired. The default value is ‘true’. -
-lock-timeout=<duration>: Determines the wait time for a state lock to be acquired. The default is ‘0s’ (no timeout). -
-backup=<path>: Used to pass the location path for a backup of the state file before it's overwritten. The default value is ‘.tfstate’backup'.
How does Terraform Refresh State Work
In Terraform CLI, when the actual configuration of your resources on cloud providers (such as AWS, GCP, or Azure) no longer matches the configuration defined in your Terraform configuration, it causes a drift.
In such a scenario, when you run terraform refresh command, it reconciles this difference between the desired infrastructure (your state file) and the current infrastructure (actual cloud configuration) to match the difference between them (a.k.a drifts), by doing the following:
- First, Terraform inspects the existing state file (terraform.tfstate), which contains the desired state of your infrastructure. While reading the state file, Terraform identifies the resources that need to be refreshed or updated as defined in your .tf files.
- Next, Terraform makes API calls to the providers, to retrieve the current state of the infrastructure resources, as they were defined in the Terraform code.
- Once the configuration is fetched, Terraform compares the configuration in the terraform.tfstate file with the current state of your infrastructure. If there are any changes in the resource arguments, the file is refreshed and updated with the current state for all those resources.
Two important things to note:
- After running
refreshit’s always a good idea to verify that the state file is updated using theterraform showcommand. - It’s important to keep in mind that the
refreshcommand doesn’t actually make any changes to the current infrastructure, and only updates the state file.
Example Scenario
To better demonstrate how terraform refresh works, let’s go into a quick hands-on example of how it can be used to update your state file.
First, let’s define an AWS S3 bucket using the Terraform main.tf file.
provider "aws" {
region = "us-east-1"
}
resource "aws_s3_bucket" "terraform_state" {
bucket = "env0-terraform-state-bucket"
lifecycle {
prevent_destroy = false
}
tags = {
Name = "Terraform State Bucket"
}
}
Next, let’s run the terraform init and terraform apply commands to deploy the S3 bucket on AWS.
➜ env0 git:(main) ✗ terraform init
Initializing the backend...
Initializing provider plugins...
- Finding latest version of hashicorp/aws...
- Installing hashicorp/aws v5.77.0...
- Installed hashicorp/aws v5.77.0 (signed by HashiCorp)
…
Terraform has been successfully initialized!
…
➜ env0 git:(main) ✗ terraform apply --auto-approve
+ create
Terraform will perform the following actions:
# aws_s3_bucket.terraform_state will be created
+ resource "aws_s3_bucket" "terraform_state" {
+ bucket = "env0-terraform-state-bucket"
+ force_destroy = false
+ tags = {
+ "Name" = "Terraform State Bucket"
}
+ tags_all = {
+ "Name" = "Terraform State Bucket"
}
…
}
Plan: 1 to add, 0 to change, 0 to destroy.
aws_s3_bucket.terraform_state: Creating...
aws_s3_bucket.terraform_state: Creation complete after 5s [id=env0-terraform-state-bucket]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
To close the loop, you can verify that the bucket was created by visiting the AWS console. Navigate to ‘S3’ and search with the bucket name. Here, you’ll see an ‘env0-terraform-state-bucket’.
With the bucket created, run the terraform show commands to view the current local state file with the resources managed using Terraform.
➜ env0 git:(main) ✗ terraform show
# aws_s3_bucket.terraform_state:
resource "aws_s3_bucket" "terraform_state" {
arn = "arn:aws:s3:::env0-terraform-state-bucket"
bucket = "env0-terraform-state-bucket"
force_destroy = false
tags = {
"Name" = "Terraform State Bucket"
}
tags_all = {
"Name" = "Terraform State Bucket"
}
…
Now, in the AWS console, edit the bucket tags under ‘Properties.’ You can add an owner tag, which helps you identify the owner of this bucket. Here, it is ‘Saksham’.
This manual change creates a drift in the infrastructure, which means that the current Terraform state file is out of sync with the current AWS infrastructure.
Now it’s time to run the terraform plan command to detect the drift changes in your infrastructure.
➜ env git:(main) ✗ expoterraform plan
aws_s3_bucket.terraform_state: Refreshing state... [id=env0-terraform-state-bucket]
~ update in-place
# aws_s3_bucket.terraform_state will be updated in-place
~ resource "aws_s3_bucket" "terraform_state" {
~ tags = {
"Name" = "Terraform State Bucket"
- "Owner" = "Saksham" -> null
}
~ tags_all = {
- "Owner" = "Saksham" -> null
…
Plan: 0 to add, 1 to change, 0 to destroy.
The above plan output displays that the new tag – ‘Owner’ – was added to the S3 bucket outside of your Terraform configuration. If you apply your Terraform code, the tag will be removed from the bucket.
This means that there is a drift and the Terraform configuration in your state file does not contain any ‘Owner’ tag.
You can update your state file by running the terraform refresh command.
➜ env0 git:(main) ✗ terraform refresh
aws_s3_bucket.terraform_state: Refreshing state... [id=env0-terraform-state-bucket]
Once the Terraform state file is refreshed, you can review the updated state file output using terraform show.
➜ env0 git:(main) ✗ expoterraform show
# aws_s3_bucket.terraform_state:
resource "aws_s3_bucket" "terraform_state" {
arn = "arn:aws:s3:::env0-terraform-state-bucket"
bucket = "env0-terraform-state-bucket"
id = "env0-terraform-state-bucket"
tags = {
"Name" = "Terraform State Bucket"
"Owner" = "Saksham"
}
tags_all = {
"Name" = "Terraform State Bucket"
"Owner" = "Saksham"
}
…
In the above output, you can see that the S3 bucket configuration has been updated with the latest tags from the AWS in the state file.
If you want to overwrite this change with your Terraform code configuration, you need to run the apply command. Otherwise, if you want to accept this change, add the tags in your Terraform code and run apply to deploy the changes to the cloud.
Concerns with Terraform Refresh
As mentioned above, the terrafrom refresh command was deprecated because of unsafe behavior in the case of misconfigured providers.
For instance, if you have misconfigured the provider credentials for one AWS account (A) with another AWS account (B), the command could trick Terraform into updating the state file with changes from account (B) rather than account (A), which could lead to various issue such as the deletion of all the resources in the state file without any confirmation.
In addition to the above concerns, the refresh command has other inherent limitations, which include:
- Running
refreshdoes not modify the configuration in the .tf files. In case there was a configuration change, you will need to manually update your Terraform management code. - The
refreshcommand helps you detect drift by comparing the current and desired states. However, you still need to fix this drift to avoid resource misconfigurations or security issues. And fixing the drift doesn’t always mean reverting to the configuration in the state file, as some drift could be a result of an intended change (e.g., manual error fix or the result of other software behavior). - When multiple team members work on a large infrastructure, continuously running
terraform refreshjust to fetch any manual changes made to the infrastructure is not really feasible, running the risk of creating merge conflicts and plenty of overheard.
Apply refresh-only: A (Slightly) Better Alternative:
Terraform version 0.15.4 introduced the -refresh-only flag to provide more control over the functionality of the refresh command. When you run this with the plan and apply commands, you will be greeted with an interactive prompt where you can review and confirm the detected changes.
For example, run the following command:
terraform apply -refresh-only
This will allow you to review the changes in your current infrastructure before updating your state file with them. Once you have reviewed them, simply approve or reject these changes.
Unlike the refresh command, using apply -refresh-only offers a better approach to handling drift. However, it is still far from being a comprehensive solution as it doesn’t solve the challenges of drift detection, nor does it help provide context for the drift.
As a result, some drifts go unnoticed, and their reconciliation could end up causing more infrastructure chaos than remediation. Also, rolling back some of the drifts could cause a domino effect of issues, effectively removing necessary – albeit manual or third-party – changes.
Drift Detection with env0
With env0’s drift detection and cause analysis features, you do not need to worry about scheduling runs for plan or refresh to continuously monitor your infrastructure or identify potential drifts. Moreover, you will also have additional context to ensure that the drifts are reconciled without causing any unwanted cascading issues across your cloud infrastructure.
Let’s look at how you can identify and investigate drifts using env0:
Automated Drift Detection
Using env0*,* you can automate the process of drift detection in your infrastructure code by enabling the ‘Drift Detection’ in ‘Settings’ with a cron job.
For example, schedule a drift detection run once every hour in your environment. This will run the scheduled plan command and analyze its output for any changes done outside your Terraform configuration or drifts. When any drift is detected, you can see the result as ‘drifted’ in the plan stage of your deployment
This capability saves plenty of time and prevents any incidents due to neglected drift.
Using some of the platform's more advanced capabilities, you can even be preemptively aware of future drifts that might occur after applying the new plan.
These, for example, could be a result of using dynamic variables, and they will appear as line items in the env0 dashboard, enabling you to further investigate by going back to reviewing the plan.
Drift Cause
In addition to the above, env0 platform also offers a unique Drift Cause feature which connects the dots between the state of the codified infrastructure and out-of-code audit logs to offer additional context about drifts, and enable team to:
- Identify who made the change, when, and how
- Understand the specific event or action responsible for the drift (e.g., automated or scripted procedure via CLI or API, or a human being via cloud provider interface)
Check out the video below to see it in action.
Drift Monitoring and Alerting
env0 also provides a centralized dashboard that displays the ‘Drift Status’ for each infrastructure environment deployed using Terraform. You can also see more information on what percentage of environments are drifted or if these environments have active, inactivate, or failed deployments, like so:
Using this dashboard, teams can easily monitor multiple environments for drift simultaneously under one unified dashboard, improving team efficiency. Additionally, team members can quickly identify which environments are drifting from their desired Terraform configuration.
Approval Policies
Related to the topic of drifts, env0 allows you to enable approval policies (e.g., using Open Policy Agent). These can be tailored to preemptively address common causes of drifts, which could be identified through the use of a dashboard and insights provided by Drift Cause.
Leveraging these insights on what could typically go wrong, platform and infrastructure teams can enhance scheduled deployments with relevant approval policies together, to create a framework for a smart auto-remediation.
Final Thoughts
By now, you should understand what Terraform refresh does and when you can use it for your use case. We did a deep dive into what happens when you run the refresh command and shared some practical examples.
Even though terraform refresh is really helpful for larger teams, we learned about its limitations and how you can overcome them with env0’s drift detection and remediation capabilities.
Frequently Asked Questions
Q1. Does Terraform plan refresh the state?
Yes, Terraform refreshes the state and updates the state file with the most recent resource configuration of the current infrastructure. It ensures that the plan reflects the current state rather than the desired state before any changes are made.
Q2. How do I apply Terraform without refresh state?
You can apply the Terraform refresh by running the terraform apply -refresh=false command. It skips the refresh and applies the plan based on the existing state.
Q3. How does Terraform refresh state work?
Terraform refreshes its state by querying the infrastructure to detect any changes made outside of Terraform (manual changes, updates by other tools, etc.). This process ensures that the Terraform state file reflects the current state of the infrastructure.
Q4. What is the difference between Terraform refresh and Terraform import?
Terraform’s refresh command updates the state file by checking the actual infrastructure to reflect any changes made outside Terraform (e.g., manually via the AWS Console). It doesn’t create or modify resources, only syncs the state with the real world.
Terraform’s import command is used to bring existing resources into Terraform’s management. It associates an existing infrastructure resource with a Terraform resource block, allowing Terraform to manage it going forward. It updates the state file but doesn’t modify the resource itself.
Top comments (0)