Advanced GitHub Actions - Conditional Workflow

This is more about describing my personal development and debugging experience, less of a step-by-step tutorial on the topic. The article presents some key points and questions I faced when going through the project. Correction to any of the errors is welcome! ☺️

🍀 Problem statement

GitHub Actions provides the ability to create automated workloads, CI/CD pipelines. I have 3 different branches: dev, beta, and main. They have the same workflow, which is to build, test, and deploy my application to GCP App Engine. More details were discussed in my previous article.

In my case, only configurations (secrets and environment variables) are different. Maintaining 3 different workflows, one for each, is not recommended due to duplicated code. With that said, how can I create one workflow which uses different secrets based on a triggered branch?

🍀 Key points

💡 Can I make all 3 branches trigger the same workflow?

Yes, there is a key on to serve this purpose.

name: My workflow

# Define which action triggers the workflow
on: 
  # Name of the action
  push: 
    # Filter, define which branch triggers the action
    # I want all branches to trigger this workflow
    branches:       
      - dev # Direct push is allowed, this deploys to a staging server
      - beta # Protected branch, no direct push, only accepted merge is allowed
      - main # Protected branch, no direct push, only accepted merge is allowed

💡 How about if / else if / else statement to check for branches?

GitHub Actions does support the if key, but there is no else if / else. Besides, I can use contexts to read the current branch name.

The syntax of contexts is ${{ <context> }}, so I have something as below

steps:
  - name: My step
  # github.ref context is used to determine the branch name
  # It is evaluated against a git's ref name

  # Wrong syntax
  # if: ${{ github.ref }} == 'something'

  # Correct syntax
  # I must use a single quote here
  # It took me a while to determine that a double quote broke the workflow
  if: ${{ github.ref == 'ref/heads/main' }}

  ...

💡 There is no else if / else key.

Well, I can use 3 different if in 3 different steps. Each one represents a specific branch.

I use google-github-actions/auth in the first step in my job to authenticate to GCP. At this point, I have 6 different GitHub secrets to test out the concept. Each branch has two secrets with the format BRANCH_WIP and BRANCH_SA.

# Step for the main branch
- name: MAIN authentication
  if: ${{ github.ref == 'ref/heads/main' }} && ${{ secrets.MAIN_WIP != null }} && ${{ secrets.MAIN_SA != null }}
  uses: google-github-actions/auth@v0
  with:
    workload_identity_provider: ${{ secrets.MAIN_WIP }}
    service_account: $${{ secrets.MAIN_SA }}

  ...

# Steps for beta and dev branches follow the same structure
# But, the code above will NOT work, I will explain below

I use nektos/act to test the workflow locally, but it fails with the message Error: exit with 'FAILURE': 1. Guess what, secrets is not usable in if key, hence the error.

💡 GitHub Secrets cannot be read in a conditional statement.

GitHub Actions has a key env to define environment variables at different scopes in the workflow. I use it at step level to import the secrets because env can be read in an if key.

- name: MAIN authentication
  env:
    # Define a key pair in using an environment variable.
    WIP: ${{ secrets.MAIN_WIP }}
    SA: ${{ secrets.MAIN_SA }}
  if: ${{ github.ref == 'ref/heads/main' }} && env.WIP != null && env.SA != null
  uses: google-github-actions/auth@v0
  with:
    workload_identity_provider: ${{ env.WIP }}
    service_account: ${{ env.SA }}
    # Note, I need to use '${{ }}' in reusable workflow 'auth@v0' to read 'env'
    # Meanwhile, it is not necessary in the 'if' key.
    # This cost me a fair amount of time to debug

💡 Pre-compiled codebase can read environment variables.

However, remember only what is performed inside GitHub VM can read the variables. The backend, which is uploaded to GAE, is built and tested in a GCP environment. It does not have access to GitHub resources.

- name: Configuration for React environment variables
  if: ${{ github.ref == 'refs/heads/dev' }}
  # Build step can read REACT_APP_DEV
  run: |
    echo "REACT_APP_DEV=${{ secrets.DEV }}" >> $GITHUB_ENV

💡 Creating an action 3 times for 3 branches results in duplicated code, should there be a way to refactor?

As of now, the environment variables have their scope limited to a step it stays in. Inside a job, I have steps key, which contains multiple steps underneath. Therefore, at the minimum, I need environment variables to have steps scope.

Indeed, steps scope works. In a job, I can write the variables to $GITHUB_ENV at the first step in a steps key, so all subsequent steps can access the environment variables. Note, I cannot create a conditional statement at job scope, so maybe only steps scope works after all.

I came across an approach using set-env command. However, set-env was deprecated due to CVE-2020-15228 and is not usable anymore.

Now, I use echo "MY_ENV=${{ secrets.MY_SECRET }}" >> $GITHUB_ENV instead.

name: CI/CD pipeline

# Define which action triggers the workflow
on: 
  push: 
    branches:       
      - dev # Direct push is allowed, this deploys to a staging server
      - beta # Protected branch, no direct push, only accepted merge is allowed
      - main # Protected branch, no direct push, only accepted merge is allowed

jobs:
  auth:
    name: Authenticate to GCP
    runs-on: ubuntu-latest
    strategy:
      matrix:
        node-version: [16.x]
    permissions:
      contents: read
      id-token: write 

   steps:
     # I only need to declare environment variables once at the beginning
     # The subsequent steps can access the variables by default
     - name: Configuration for main branch
       if: ${{ github.ref == 'refs/heads/main' }}
       # Side note, I can run any Linux command here, not just 'echo'
       run: |
         echo "GCP_WIP=${{ secrets.MAIN_WIP }}" >> $GITHUB_ENV
         echo "GCP_SA=${{ secrets.MAIN_SA }}" >> $GITHUB_ENV

    - name: Configuration for beta branch
      if: ${{ github.ref == 'refs/heads/beta' }}
      run: |
        echo "GCP_WIP=${{ secrets.BETA_WIP }}" >> $GITHUB_ENV
        echo "GCP_SA=${{ secrets.BETA_SA }}" >> $GITHUB_ENV

    - name: Configuration for dev branch
      if: ${{ github.ref == 'refs/heads/dev' }}
      run: |
        echo "GCP_WIP=${{ secrets.DEV_WIP }}" >> $GITHUB_ENV
        echo "GCP_SA=${{ secrets.DEV_SA }}" >> $GITHUB_ENV    

    - name: Authenticate to GCP
      if:  env.GCP_WIP != null && env.GCP_SA != null
      uses: google-github-actions/auth@v0
      with:
        workload_identity_provider: ${{ env.GCP_WIP }}
        service_account: ${{ env.GCP_SA }}

    ...

🍀 Wrap Up

Initially, my usage of GitHub Actions only involves simple actions such as build, test. With conditional workflow, there is a whole new set of possibilities. The article mostly focuses on conditional workflow. However, I should also be aware of some concepts that are occasionally mentioned, either directly or indirectly, throughout the article.

1. Contexts
2. Scope and hierarchical access
3. Reusable workflow
4. Custom Linux commands (a workflow is run in a VM)

I will spend some time on the GitHub Actions documents then.