AWS EC2 Image Builder for SOE AMI creation & distribution

AWS EC2 image builder is a great service from AWS which will allow you to create a custom SOE image for your AWS environment by including the desired state configuration defined for the SOE image in your organization.

EC2 Image builder Pipeline can be configured to either automatically execute on a particular schedule defined by the user/admin or execute manually from the AWS console/CLI.

The EC2 image builder service mainly performs two activities during the build process. It creates the Image as per the SOE requirements and distributes the image across aws accounts.

The below diagram shows the multiple activities an EC2 image builder will perform during the execution phase.

Image Builder Pipeline Creation Process

The first step in the EC2 image builder is to create an image pipeline. In the AWS portal, search for "EC2 image builder" and click on the service when appear.

On the EC2 image builder landing zone page, click on "Create pipeline" to start the pipeline creation process. The image pipeline wizard will appear in the page.

In the "Specify the image pipeline details" tab provide the name and description of the pipeline. You need to also define whether the pipeline need to be executed manually or scheduled for an automated execution.

The next step is to define a recipe for the pipeline. As the name suggests Recipe should contain which OS need to be used, what all softwares to be installed, configurations to be added, tests to perform etc.
Let us create a new Recipe, the first thing is to select the final Image type. You can select to create either an AMI or a Docker image to be created. The next step is to select the Operating system and version. I have selected Windows Server 2019 the latest version available in AWS Marketplace. We can use private AMIs or can upload custom images also to start with.

Select whether you need to remove the SSM agent as part of the build completion. If you unselect the option, then AWS systems manager will be part of your core AMI. You can also provide the userdata to be executed during instance launch time.

Select the working directory to use during the build and test workflows stage

Under components, you can select either a build component or test component to be used during the image creation. Amazon has default build components which can be used to install amazon specific configurations.

Create Build component

Let us see how can we create custom build configuration to install software from various sources. I had taken one source as my S3 bucket and the other from internet. Click on "Create build component" to initiate the build component wizard. In the Create component wizard, select the component type to create, select from either build or test.

Note:_ For S3 based deployment, make sure your AWS IAM role has permissions to fetch the S3 bucket and the components.

In the component details, select the OS type, KMS key to encrypt, compatible OS versions, name and version of the component.

The component creation is performed using yaml. I have given the source code for both s3 based deployment and web based in my github and the locations are provided below.

S3 based deployment:-

https://github.com/amalkabraham001/MyWordpressRepo/blob/cf680ef97177f5643664af5a0d2d0860e27639db/ChromeInstallCLIMSI

Web based deployment:- https://github.com/amalkabraham001/MyWordpressRepo/blob/cf680ef97177f5643664af5a0d2d0860e27639db/AWS/EdgeInstallfromWeb

Detailed guide on the yaml will be shared via a separate blog. Refer docs.aws.amazon.com/imagebuilder/latest/userguide/toe-action-modules.html for more information.

In the definition document, enter the yaml script and click on Create to create the component.

name: EdgeMSI
description: Install Edge from Internet using the MSI installer
schemaVersion: 1.0
phases:
  - name: build
    steps:
      - name: Download
        action: WebDownload
        inputs:
          - source: https://msedge.sf.dl.delivery.mp.microsoft.com/filestreamingservice/files/dbdd4903-2893-48c6-bd53-a3f03a424e01/MicrosoftEdgeEnterpriseX64.msi
            destination: C:\Windows\temp\MicrosoftEdgeEnterpriseX64.msi
      - name: Install
        action: ExecuteBinary
        onFailure: Continue
        inputs:
          path: 'C:\Windows\System32\msiexec.exe'
          arguments:
            - '/i'
            - '{{ build.Download.inputs[0].destination }}'
            - '/quiet'
            - '/norestart'
      - name: Delete
        action: DeleteFile
        inputs:
          - path: '{{ build.Download.inputs[0].destination }}'

Once the components are created select the required build and test components from the available components list

Select the Volume details for your pipeline VM

Create a new Infrastructure configuration to be used in the Pipeline. The important step is to select the right IAM role. I would suggest creating a custom IAM role with the required policies. I have uploaded the policy details in the GitHub page. You can access it from
https://github.com/amalkabraham001/MyWordpressRepo/blob/9661eddcc1d9a4b780eebfa759bc2274b1d96bd3/AWS/awsimagebuilderrole

Provide the instance type, VPC, subnet and security groups if you need to specify them explicitly.

Create Distribution Settings

The final step is to select the distribution settings, it allows you to replicate your final AMI to different AWS accounts inside and outside of your AWS organization. It will also allow you to configure licenses and launch templates as well.

Click on create once you configure your distribution settings and click on create to create your pipeline. You can test the pipeline by selecting your pipeline and click on "Run Pipeline" from actions.

Click on view details and under Output images, you will be able to see the AMIs created and the status of your pipeline execution.