sanchezg7

Posted on Feb 12, 2023

Setup a node app with Elastic Beanstalk and CI/CD

#node #cicd #javascript #aws

Introduction

Suppose we have a node express application that we need to publish to the cloud. After searching, there aren't many relevant articles and resources covering this topic, so I hope this is helpful to fit the exact needs. AWS CloudFormation is a good solution but it has a big barrier to entry. Instead, we will use AWS Elastic Beanstalk to do a lot of the heavy lifting. It will enable all the required AWS resources for my application; in addition, we will automate this process so that when the code is updated with new changes. The code will automatically deploy to Elastic Beanstalk with the latest changes.

What this article is not

There are many ways to write a node express application. This blog will not attempt to convince how to design or where to host the repository.

Requirements

Node application

We need a node express application. It is an API that is used only for the purposes of stubbing a real node application.

AWS CLI setup and access

AWS provides a free tier for to work with. Check out tutorials online for how to create a free tier account and set up the terminal with the AWS CLI.

Concepts

Elastic Beanstalk

This service, provided by AWS, does a lot of the heavy lifting for procuring all the infrastructure needed for the application to work. It will connect up all the required dependencies.

Elastic Beanstalk Environment

An environment is the space to deploy an application to. All the necessary AWS resources are associated to the environment in order to facilitate the needs of the platform. This tutorial will only use one environment; however, we can have as many environments needed to meet the software lifecycle means.

Application Version

The version is associated to a source bundle, identified by a label. It represents an application artifact that will get deployed to an Elastic Beanstalk environment.

Source Bundle

The artifact of the code and the reference for where it is stored. In this case, we will use S3 as the repository for the source bundle.

Simple Storage Service (S3)

This service is an object store that allows saving any type of data for accessing later. For example, it can store zip files. The source bundle will be stored here.

How To - Manual Steps

First, we need to carry out the steps manually. This will ensure all dependencies, permissions, and sequence of events are correct. This will ease any hiccups when automating this pipeline in Gitlab.

Procure an application to upload to Elastic Beanstalk

You can use this example in https://gitlab.com/ayogsynergy/eb_node_sandbox/-/tree/step1_helloworld

Upload Source Bundle to S3

Bundle the source code from VCS into a zip file.
- git archive -v -o eb-node-sandbox.zip --format=zip HEAD
Create a S3 bucket.
- This example will use manual-eb
Upload the bundle to the S3 bucket.
- aws s3 cp eb-node-sandbox.zip s3://manual-eb

Create application version

Elastic Beanstalk expects an application version to be specified in order to instantiate an environment.

Usage

aws elasticbeanstalk create-application-version \
--application-name ${APP_NAME} \
--version-label ${VERSION_LABEL} \
--description="new version ${VERSION_LABEL}" \
--source-bundle S3Bucket=${S3Bucket},S3Key=${S3Key} \
--auto-create-application \
--region ${AWS_REGION}

Example

aws elasticbeanstalk create-application-version \
--application-name eb-node-sandbox \
--version-label v0.0.01 \
--description="new version v0.0.01" \
--source-bundle S3Bucket=manual-eb,S3Key=eb-node-sandbox.zip \
--auto-create-application \
--region us-east-1

Describe application version

Confirm the application version is created.
- aws elasticbeanstalk describe-application-versions --application-name eb-node-sandbox

Create the Elastic Beanstalk environment

Next, instantiate an environment that will accept the application version to deploy.

Create a configuration template

aws elasticbeanstalk create-configuration-template \
--application-name eb-node-sandbox \
--template-name v1-template \
--solution-stack-name "64bit Amazon Linux 2 v5.6.4 running Node.js 16"

Describe the configuration template

aws elasticbeanstalk describe-configuration-settings --application-name eb-node-sandbox --template-name v1-template

Create Elastic Beanstalk environment

Create an options file for use of making environment

[
    {
        "Namespace": "aws:autoscaling:launchconfiguration",
        "OptionName": "IamInstanceProfile",
        "Value": "aws-elasticbeanstalk-ec2-role"
    }
]

Create an Elastic Beanstalk Environment for the application

aws elasticbeanstalk create-environment \
--application-name eb-node-sandbox \
--template-name v1-template \
--version-label v0.0.01 \
--environment-name eb-node-sandbox-env \
--option-settings file://options.txt

Describe the environment

Confirm the environment is created
- aws elasticbeanstalk describe-environments --environment-names eb-node-sandbox-env

At this point, there is a running Elastic Beanstalk environment accessible via the internet.

Launch the instance and send a request to GET / to verify the appropriate Hello World response is returned.

Update the environment

Now, let’s simulate a new change in the source code. We will need to publish the new changes. We will update the Elastic Beanstalk environment with the new version.

Upload new source bundle with a new key name to the S3 bucket
Create a new application version
- Associate it to the new respective bundle
- eb-node-sandbox-1.0.0-02

Update the existing Elastic Beanstalk environment with the new application version

aws elasticbeanstalk update-environment \
--application-name eb-node-sandbox \
--environment-name eb-node-sandbox-env \
--version-label eb-node-sandbox-1.0.0-02 \
--region us-east-1

Run this command to update the environment. It will take some time. Wait until the environment deployment is finished.

Try to do this in one sitting, if possible. Otherwise, terminate the environment as AWS will keep the meter running for the infrastructure that you are leveraging in your environment (e.g. EC2). This will help avoid unexpected costs.

Troubleshooting (log analysis)

If the application does not respond successfully, take a look at the logs. Here is some sample log output below.

----------------------------------------
/var/log/web.stdout.log
----------------------------------------
Jan 15 14:50:25 ip-172-31-7-140 web: > Elastic-Beanstalk-Sample-App@0.0.1 start
Jan 15 14:50:25 ip-172-31-7-140 web: > node app.js
Jan 15 14:50:25 ip-172-31-7-140 web: Server running at http://127.0.0.1:8080/
Jan 15 15:06:21 ip-172-31-7-140 web: > eb_node_sandbox@1.0.0 start
Jan 15 15:06:21 ip-172-31-7-140 web: > node server.js
Jan 15 15:06:21 ip-172-31-7-140 web: Running on Port: 8080
Jan 15 15:18:20 ip-172-31-7-140 web: > eb_node_sandbox@1.0.0 start
Jan 15 15:18:20 ip-172-31-7-140 web: > node server.js
Jan 15 15:18:21 ip-172-31-7-140 web: Running on Port: 8080

By convention, it will default to npm run start to boot your application. Make sure your package.json has this script defined. It will also run npm install prior to running your application, if it detects a package.json file.

CI/CD

Now, automate this publish process to trigger every time the repository code is updated. This example will leverage Gitlab.

Create a .gitlb-ci.yml file to indicate how/what to include in the CI/CD pipeline

stages:
  - build
  - run

variables:
  APP_NAME: ${CI_PROJECT_NAME}
  ENV_NAME: ${ENV_NAME}
  APP_VERSION: "1.0.0"
  S3BUCKET: "manual-eb"
  AWS_ID: ${AWS_ID}
  AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
  AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
  AWS_REGION: us-east-1

create_app_version:
  stage: build
  image: python:latest
  allow_failure: false
  script: |
    pip install awscli
    echo "Creating zip file"
    VERSION_LABEL=${APP_NAME}-${APP_VERSION}-${CI_PIPELINE_ID}
    S3Key="${VERSION_LABEL}.zip"
    echo "Zipping: ${S3Key}"
    git archive -v -o ${S3Key} --format=zip HEAD

    echo "Creating bundle..."
    echo "Uploading bundle: ${S3Key} ..."
    aws s3 cp ${S3Key} s3://${S3BUCKET}/${S3Key} --region ${AWS_REGION}

    echo "Creating app version"
    aws elasticbeanstalk create-application-version \
      --application-name ${APP_NAME} \
      --version-label ${VERSION_LABEL} \
      --description="new version ${VERSION_LABEL}" \
      --source-bundle S3Bucket=${S3BUCKET},S3Key=${S3Key} \
      --auto-create-application \
      --region ${AWS_REGION}
  only:
    refs:
      - main
deploy_version_to_env:
  stage: run
  image: coxauto/aws-ebcli
  when: manual
  script: |
    VERSION_LABEL=${APP_NAME}-${APP_VERSION}-${CI_PIPELINE_ID}

    echo "Deploying version: ${VERSION_LABEL} to env: ${ENV_NAME}"
    aws elasticbeanstalk update-environment \
      --application-name ${APP_NAME} \
      --environment-name ${ENV_NAME} \
      --version-label ${VERSION_LABEL} \
      --region=${AWS_REGION}
  only:
    refs:
      - main

The pipeline is split in two stages:

Stage
- create_app_version
  - Bundle the source code at HEAD and publish to the S3 bucket
  - Create a new application version and associate it to the bundle that was uploaded
Deploy
- deploy_version_to_env (manual)
  - Upload the new application version to the (already established and running) Elastic Beanstalk environment

This pipeline will trigger when the main branch is updated.

Setup CI/CD pipeline env variables

Settings → CI/CD
Variables → Expand
Add the following keys
- APP_NAME: eb-node-sandbox
- AWS_ID: xxxxx
- AWS_ACCESS_KEY_ID: xxxx
- AWS_SECRET_ACCESS_KEY: xxxxxxx

Conclusion

If you are following this tutorial, make sure to terminate your environment. This will help you avoid unexpected running costs that AWS will bill you for. Refer to the action drop-down menu to Terminate environment

Avoiding manual processes by automating can save a lot of time and unintended mistakes. It is possible to also add a test stage that is not allowed to fail in order to prevent publishing bad software to your environment.

DEV Community