Migrating CloudFormation to TF

#terraform #aws #infrastructureascode #devops

One day you might find yourself in the unfortunate position of wanting to migrate away from CloudFormation (CFN). While some may say that CFN is bad and should never be used. I can confirm that it is still better than:

CloudFormation CDK
AWS SAM
Serverless - Not "serverless", but the company that is abusing this name.
SST
And many others

The truth is: CloudFormation isn't bad, however like most things, it is bad when you find out your current solution doesn't support the thing that you want it to support.

So back to the problem...You want to migrate from CloudFormation to OpenTofu (since no one uses Terraform anymore after their legal scandal), and part of that problem involves a migration.

The Migration

Migrations are technically easy. Monolith to microservices, event buses to REST, MSSQL to NoSQL DynamoDB. The hard part is always the non-technical part. The part where you figure out what you want, now that's the problem. Unless of course you have a monolith, because you should just give up now. No one successfully converts from a Monolith to microservices. They write some code, complain a lot, then apply for a new job at a new company telling their would-be manager "Look how I helped this company migrate to microservices. I'm Great!"

But this isn't a story about how monoliths are bad, it is about how to migrate your Infrastructure as Code (IaC) solution.

Realistically, you have to painstakingly generate the new IaC HCL files for OpenTofu. You have existing CloudFormation as well as the real live version of your infrastructure currently supporting an massive business. And if you are like us at Authress, you might also have a 99.999% uptime SLA you need to account for.

If you have 100+ CFN stacks, you probably don't want to import these resources in OpenTofu by hand. Instead, you'll want some sort of tool to do this, and there are a bunch:

Former2 - Export from AWS to HCL.
Firefly.ai - AI in the company name, yuck
CF2TF - Open source converter
Doing it by hand to verify you have everything you need.

And there are still more... You could even try one of the LLMs out there.

Running the migration

At this point you only need to run the imports into OpenTofu or into Terraform for each of your resources:

import {
  to = aws_instance.example
  id = "i-abcd1234"
}

resource "aws_instance" "example" {
  name = "hashi"
  # (other resource arguments...)
}

Once we have all of those generated we just need to run tf plan, tf apply, and then delete the import statements.

And you are done!

Cleanup

The one thing that no one tells you at this point is that you aren't done. Importing the resources and having the committed IaC HCL does not mean you are done. If you are like me, then you care that you still have 100s of CFN stacks deployed in your AWS accounts. Maybe these stacks all have CFN Drift and don't even represent the current state of the world anymore.

However, even if they do represent the current state, you probably don't want someone going into your account and accidentally updating or deleting those. Or your desire to have a pristine account compels you to delete these stacks. You probably wouldn't be someone working on this problem in the first place if you didn't care that these old stacks are still here.

The problem is that there is no way to delete a stack without also deleting the resources in that stack. And of course, you want to keep the resources in those stacks, so that's a conundrum. Thankfully, I've figured out a hack to get around this.

The involves utilizing three features:

the delete_failed status
FORCE_DELETION action flag
CloudFormation execution Role ARN

The delete_failed status occurs whenever CFN tries to delete a resource that it believes is no longer necessary, but the resource is either in use OR CFN doesn't have access to delete the resource. Take note of this second one.

Second, when a stack is in the delete_failed status, you are allowed to force delete the stack and retain explicit resources that you might still be using.

So all we need to do is get the stack into the delete_failed state, and then ask CFN to retain all the resources.

CloudFormation allows, for "security reasons", you to specify a role ARN to execute CFN with. When you do that the CFN stack changes will only be executed with that role. So we'll define a new role that does not have access to anything. We'll abuse the Role ARN property to force CFN to fail to delete any resources and thus fail to delete stack.

Cleanup Execution

Create the Role:

CfnDeleteStackRole:
     Type: AWS::IAM::Role
     Properties:
       RoleName: cloudformation-service-role-delete-stack
       AssumeRolePolicyDocument:
         Version: '2012-10-17'
         Statement:
           - Effect: Allow
             Principal:
               Service: cloudformation.amazonaws.com
             Action: sts:AssumeRole

With that role, we'll call the Delete Stack:

aws cloudformation delete-stack \
    --stack-name my-stack
    --role-arn arn:aws:iam::account:role/cloudformation-service-role-delete-stack

This execution call will fail, but we knew that was going to happen. Now, it will put the stack in the status delete_failed.

Finally, we can execute the delete again, utilizing the force deletion parameters:

aws cloudformation delete-stack \
    --stack-name my-stack
    --role-arn arn:aws:iam::account:role/cloudformation-service-role-delete-stack
    --deletion_mode FORCE_DELETE_STACK

Depending on the resources you have in your stack you or if you want extra security to prevent deleting your precious resources, you can add the flag --retain-resources to the CLI command:

aws cloudformation delete-stack \
    --stack-name my-stack
    --role-arn arn:aws:iam::account:role/cloudformation-service-role-delete-stack
    --deletion_mode FORCE_DELETE_STACK
    --retain-resources $LOGICAL_RESOURCES_LIST

With $LOGICAL_RESOURCES_LIST value set as the string list of CFN resources.

const cfnTemplateFile = await fs.readFile('./cfn-template.json');
const cfnTemplate = JSON.parse(cfnTemplateFile);
const resourceKeys = Object.keys(cfnTemplate.Resources).join(',')
return resourceKeys;

// Use resourceKeys as $LOGICAL_RESOURCES_LIST