Run cypress tests in parallel on CircleCI

Background

I use and enjoy Cypress for testing, and use CircleCI for CI/CD purposes.

In the spirit of learning by tinkering, I wanted to tweak our CircleCI workflows to see if we could cut down on our CI run times and potentially save on some credits.

I wanted to dynamically split up tests and use parallel machines each running only some of the tests to speed up the testing step.

Our current workflow, and then the updated one

The jobs are as follows in simple terms -

1. npm ci, eslint and prettier
2. Run our tests - we use both react testing library + jest and cypress
3. In parallel with Step 2, make sure app can build

I restructured them a bit, and separated react testing library tests from cypres tests. Now it looked like this -

1. First step
   • Start with npm ci
   • Run eslint and prettier for obvious reasons
   • Save to workspace
2. Run react testing library tests and build app in parallel
3. If everything goes well, run the comprehensive cypres testing suite.

CircleCI Parallelism

CircleCI allows you to run containers in parallel for any of the jobs with extreme ease -

# Resource Class allows you to choose a machine level to adjust cpu/memory requirements
resource_class: medium+
# Parallelism allows you to launch multiple machines which run same code
parallelism: 4

However, the issue was that ALL the tests were running in all the containers which defeated the purpose of parallel testing.

Next was to figure out how to split up our cypress tests.

Setup for cypress

We use two types of tests for cypress -

• Normal js tests
• Using Cucumber

It deserves a separate post about how I set up Cucumber/Cypress and why it’s awesome!

Anyways, the first question in front me was - How can I split up tests across multiple machines? Answer - CircleCI to rescue!

They provide an extremely easy way of splitting tests up.

So I used this -

- run:
    command: |
      mkdir cypress/tmp
      mv $(CircleCI tests glob cypress/integration/*.spec.js | CircleCI tests split) cypress/tmp || true
      mv $(CircleCI tests glob cypress/integration/*.feature | CircleCI tests split) cypress/tmp || true
      ["$(ls -A cypress/tmp)"] && npm run test

• I create a temp directory
• I run CircleCI test splitting on both types of files - *.spec.js and *.feature
• Lastly, I just point cypress to the newly formed temp folder and let it loose!
• npm run test just runs cypress run with custom env defined in package.json

This worked reasonably well already. Then I realized that the default browser used was electron and I wanted to use Chrome to run tests.

Adding chrome to the mix

I first told cypress to use chrome as a browser when it ran the tests

cypress run --browser chrome

In the first run, it complained that chrome was not installed on the system.

Fixing issues

After some struggle, I found a ( hacky ?) solution to get it to work -

1. Add docker image

docker:
  - image: cimg/node:17.1.0-browsers

1. Install Chrome and chromedriver

steps:
  - browser-tools/install-chrome
  - browser-tools/install-chromedriver

I was all excited to see the magic happen when I was hit by an error about xvfb in my face! Time to google again !!

Finally I added xvfb-run -a npm run test to the yml file and all worked well.

What’s still missing

As I said, this was tinkering at best and more can be added.

1. CircleCI allows test splitting by timing data, which will certainly optimize a run.
2. Adding chrome like this seems hacky to me, surely there are simpler ways to do it.
3. Just by writing better tests, I should cover more user flows quicker.

In the end, this was a fun side quest for me, and learnt a few things about setting up CircleCI flows.