A brief introduction to AWS Cloudformation

AWS CloudFormation

Originally published on https://milapneupane.com.np/2019/10/07/a-brief-introduction-to-aws-cloudformation

AWS CloudFormation is a tool to write the Infrastructure as a Code(IaC). Any kind of AWS resources such as VPCs, EC2, S3, RDS or any other kind of resources can be created using AWS CloudFormation.

This has a lot of advantages such as:

1. Automation of AWS resource creation
2. Disaster recovery
3. Copying a resource from one location to another
4. Code review of the infrastructure and many more.

AWS CloudFormation Template

Cloudformation uses stacks to create resources. A stack might contain one or more resources. Let us look into a simple DynamoDB creation with AWS CloudFormation template:

// dynamodb-cloudformation.yaml

AWSTemplateFormatVersion: "2010-09-09"
Resources:
  userDynamoDBTable:
    Type: AWS::DynamoDB::Table
    Properties: 
     AttributeDefinitions: 
     - 
     AttributeName: "Id"
     AttributeType: "S"
     KeySchema:
     - 
     AttributeName: "Id"
     KeyType: "HASH"
     ProvisionedThroughput:
     ReadCapacityUnits: "5"
     WriteCapacityUnits: "5"
     TableName: "users"

The above template is used to create a DynamoDB table. It contains a key AWSTemplateFormatVersion which is used to describe the version of the CloudFormation template. This is just to identify the version of the CloudFormation template. Do not write a random date there. It is a predefined version of CloudFormation. AWS CloudFormation is written in a YAML format. It has three important top-level keys beside the AWSTemplateFormatVersion:

1. Resource
2. Parameter
3. Output

Resource

In the above example, the DynamoDB resource is defined inside the Resource key. A resource always has two keys:

Type: The type of resource. In our case it is the Dynamodb so we keep the value as AWS::DynamoDB::Table.

Properties: Properties define the configuration and options for the resource. Such as the read capacity, primary key, table name, etc.

Now, let us create an AWS CloudFormation stack with the above template using the following command:

 > aws cloudformation create-stack \
   --template-body file://dynamodb-cloudformation.yaml \
   --stack-name dynamodb-table

This will start the process of stack creation. This will take some time before completion. You can check the progress of the stack creation using the following command:

> aws cloudformation describe-stack-events \
  --stack-name dynamodb-table

This will return an array of events, during the stack creation. The stack can have various states. Bellow are the possible state while creation

1. CREATE_IN_PROGRESS
2. CREATE_COMPLETE
3. CREATE_FAILED

You can keep polling using describe-stack-events to see if the stack creation is complete or you can use the wait command, which will wait until the stack create complete or fails:

> aws cloudformation wait stack-create-complete \
  --stack-name dynamodb-table

If you do not like using CLI commands you can simply go to AWS console on your browser, go to CloudFormation service and check the status of the CloudFormation stack from there.

Parameters

Parameters in a template make the stack configurable. With parameters, you can provide input and use the same template to provide different configurations for the stack creation. Let us use the above template and make it configurable and create two different DynamoDB tables using the same template:

// dynamodb-cloudformation.yaml
AWSTemplateFormatVersion: "2010-09-09"
Parameters:
     TableName:
        Description: Name of the table
        Type: String
Resources:
     userDynamoDBTable:
         Type: AWS::DynamoDB::Table
         Properties: 
         AttributeDefinitions: 
         - 
         AttributeName: "Id"
         AttributeType: "S"
         KeySchema:
         - 
         AttributeName: "Id"
         KeyType: "HASH"
         ProvisionedThroughput:
         ReadCapacityUnits: "5"
         WriteCapacityUnits: "5"
         TableName: !Ref TableName

In the above example, the TableName can be provided by passing it as a parameter with the following command:

> aws cloudformation create-stack \
  --template-body file://dynamodb-cloudformation.yaml \
  --stack-name dynamodb-table \
  --parameters ParameterKey=TableName,ParameterValue=users

You can use the !Ref  keyword to reference the parameter passed for the table name or any other parameters.

Let us add some more parameters here:

// dynamodb-cloudformation.yaml
AWSTemplateFormatVersion: "2010-09-09"
Parameters:
  ReadCapacityUnits:
    Description: Name of the table
    Type: String
    Default: '5'
    MinValue: '1'
    MaxValue: '20'
    ConstraintDescription: must be between MinValue and MaxValue
  WriteCapacityUnits:
    Description: Name of the table
    Type: String
    Default: '5'
    MinValue: '1'
    MaxValue: '20'
    ConstraintDescription: must be between MinValue and MaxValue
  TableName:
     Description: Name of the table
     Type: String
Resources:
.....

The parameters in the template support constraints and default values. The constraint in the limits the min value and max value for the capacity to be 1 - 20. If no parameter provided it uses 5 by default.

We can create the stack with few more parameters:

> aws cloudformation create-stack \
  --template-body file://dynamodb-cloudformation.yaml \
  --stack-name dynamodb-table \
  --parameters ParameterKey=TableName,ParameterValue=users \
               ParameterKey=WriteCapacityUnits,ParameterValue=20 \
               ParameterKey=ReadCapacityUnits,ParameterValue=10 \

With more and more parameter the command to create the stack gets bigger. We can move the parameters to a file as well.

Create a parameter file:

// paramters.json
[
  {
    "ParameterKey": "TableName",
    "ParameterValue": "users"
  }, 
  {
    "ParameterKey": "WriteCapacityUnits",
    "ParameterValue": "20"
  },
  {
    "ParameterKey": "ReadCapacityUnits",
    "ParameterValue": "10"
  }
]

Now use the file to create the stack:

> aws cloudformation create-stack \
  --template-body file://dynamodb-cloudformation.yaml \
  --stack-name dynamodb-table \
  --parameters file:///parameters.json

Outputs

The third part of the AWS Cloudformation is the output section. This section contains the value that you can export and can be used by another Cloudformation template by importing it.

You can export the Arn of the DynamoDB form this template and use it in another template. Let us export the DynamoDB Arn and import it in another template:

// dynamodb-cloudformation.yaml

AWSTemplateFormatVersion: "2010-09-09"
Resources:
  userDynamoDBTable:
    Type: AWS::DynamoDB::Table
    Properties: 
      AttributeDefinitions: 
      - 
      AttributeName: "Id"
      AttributeType: "S"
      KeySchema:
      - 
      AttributeName: "Id"
      KeyType: "HASH"
      ProvisionedThroughput:
      ReadCapacityUnits: "5"
      WriteCapacityUnits: "5"
      TableName: "users"
  Outputs:
    DynamoDbArn:
      Value: !GetAtt userDynamoDBTable.Arn
      Description: Arn of the user dynamodb
      Export:
        Name: DynamoDBArn

We can now import the Output in another CloudFormation template using the export value with the intrinsic function Fn::ImportValue

Fn::ImportValue: DynamoDBArn

Intrinsic Functions

The CloudFormation template supports different types of intrinsic functions. In the above examples, we used !Ref to refer to the parameter. !Ref is one of the intrinsic functions. There are several kinds of intrinsic functions. Let us look into a couple of them:

Fn::Join

It joins a set of values separated by the specified delimiter:

TableName:  Fn::Join: [ :, [ !Ref TableName, "table"] ]

The table name will now be user:table.

Fn::GetAtt

This function can be used to fetch certain attributes of the resource. If you want the Arn of the newly created dynamoDB you can use this function:

Fn::GetAtt: [ userDynamoDBTable, Arn ]

Updating/Deleting a stack

You can Delete the resources created by a Cloudformation stack using the following command:

> aws cloudformation delete-stack \
  --stack-name dynamodb-table

This will remove the resources present in the AWS CloudFormation stack

If you want to update the resources simply change the template and call the update stack command:

> aws cloudformation update-stack \
  --template-body file://dynamodb-cloudformation.yaml \
  --stack-name dynamodb-table --parameters file:///parameters.json

Custom CloudFormation

AWS CloudFormation supports most of the AWS resource creation. In some cases when a new service is introduced or when we have some custom CloudFormation might not have the support for it. For these kinds of cases, we can create custom AWS CloudFormation.

A custom Cloudformation can be created using AWS Lambda. The lambda needs to handle resource creation, deletion, and update. We will see more about the Custom Cloudformation in my next blog:

https://milapneupane.com.np/2019/10/08/a-guide-to-aws-custom-cloudformation-resource/

AWS Cloudformation is a power tool to provision AWS resources with infrastructure as a Code. Using AWS CloudFormation helps to build large scale applications with good disaster recovery in place.

---

Interested in learning about AWS ECS deployment using Cloudformation? Learn more about it in detail here:

https://milapneupane.com.np/2019/07/28/how-to-deploy-a-docker-container-with-aws-ecs-using-cloudformation/

AWS ECS Cloudformation