Deploy AWS Lambdas on different stages with Environment Variables using CircleCI

It might seem straight forward to deploy Lambda functions using CircleCI, and in fact it is. but in my case, I wanted to deploy Lambdas on different stages using the same CircleCI workflow, and what made it a bit challenging were Environment Variables because each stage has its own.

In this post, I am going to discuss two main points 1) securing env variables in Lambdas using CircleCI. 2) Having different variables for each stage.

Working with Env Variables

The golden role in development says that env variables (.env or .secret files) should always be stored locally and not pushed to your Repo no matter. That means, if you are using .env file in your Project and wanted to deploy it to AWS Lambda using CircleCI, your .env file will be missing.

You can add those env variables to your serverless.yml file, but that one is also going to be in your Git repo, which returns us to point 1 again. Some might argue about using AWS Secret Manager, which is really a good solution but is a bit expensive, 0.40$ for each key stored.

In order to solve that issue I went with creating my .env file on the go, while building the App before deploying it. The idea is really straight forward, you run a shell command to create .env

printf "NODE_ENV='$NODE_ENV'\nGOOGLE_MAPS_KEY='$GOOGLE_MAPS_KEY'
NODE_VERSION='$NODE_VERSION'\nYOU_NAME_IT='$YOU_NAME_IT'" > .env

Now, when you package your app to deploy it on Lambda, .env file will be there for you.

Well, this method has only one down side, your .env file might be readable inside your Lambda function. If you go to AWS Lambda functions' console, you can see Lambda files there, and of course, .env file will be a plain text. But simply restricting access to your Lambda functions will solve this issue, and if the Application is a bit big, Lambda will not load a preview for you, so you are safe.

Different Variables for Different Stages

Well, the aforementioned method still has something missing, the real values, from where does shell grabs $NODE_ENV's value, no? and here where CircleCI Contexts becomes handy.

In CircleCI, and in most CI/CD tools, you have the ability to store Env variables to use them while building and deploying. However, Contexts in CircleCI allows us to store Env variables securely; once you store the variable you cannot see its value in plain text.

You can simply add the value of $NODE_ENV, and other variables, there and it will be available for you while building and deploying, which now brings us to the second issue: How to have different Env variables for different stages in the same file?

Contexts again is the answer here, because in Contexts you can create different projects and store different values inside each one. Then, we can call them inside CircleCI's config file before running any job like this

workflows:
  version: 2
  build-then-deploy:
    jobs:
      - build
      - build-test-app:
          context: app-dev #Context Name
          requires:
            - build
      - deploy-staging:
          context: app-staging #Context Name
          requires:
            - build-test-app
      - deploy-production:
          context: app-production #Context Name
          requires:
            - build-test-app

To be honest, I was stuck a bit in creating a reusable code to create env files, but using commands I was able to solve this, and the code snippet looks something like this

commands:
  create-env-file:
    steps:
      - run:
          name: Create ENV File
          working_directory: myApp
          command: |
            printf "NODE_ENV='$NODE_ENV'\nGOOGLE_MAPS_KEY='$GOOGLE_MAPS_KEY'
            NODE_VERSION='$NODE_VERSION'\nYOU_NAME_IT='$YOU_NAME_IT'" > .env

version: 2.1
jobs:
  build:
    <<: *defaults
    steps:
      - checkout

  build-test-app:
    <<: *node-defaults
    steps:
      - create-env-file
      - run:
          name: Deploy App to DEV if Branch is Master
          working_directory: myApp
          command: |
            if [ "$CIRCLE_BRANCH" = "master" ];then
              sls deploy --stage development
            fi

  deploy-staging:
    <<: *node-defaults
    steps:
      - create-env-file
      - run:
          name: Deploy App to Staging if Branch is Master
          working_directory: myApp
          command: |
            if [ "$CIRCLE_BRANCH" = "master" ];then
              sls deploy --stage staging
            fi

  deploy-production:
    <<: *node-defaults
    steps:
      - create-env-file
      - run:
          name: Deploy App to Production if Branch is Master
          working_directory: myApp
          command: |
            if [ "$CIRCLE_BRANCH" = "master" ];then
              sls deploy --stage production
            fi

workflows:
  version: 2
  build-then-deploy:
    jobs:
      - build
      - build-test-app:
          context: app-dev #Context Name
          requires:
            - build
      - deploy-staging:
          context: app-staging #Context Name
          requires:
            - build-test-app
      - deploy-production:
          context: app-production #Context Name
          requires:
            - build-test-app

& Voila!

Thanks for reading & I would like to hear and discuss different approaches as well.