TestCafe: Organizing Tests with Metadata

This is a cross-post from my blog on Medium

The TestCafe framework has a nice little feature for fixture and test organization. All fixtures and tests have an optional meta property that you can use to attach any additional information. Let's look at how we can use this meta property for more granular test runs. We'll first look at a couple of different ways to filter your tests.

Filter by filename

This I think is the most common way of splitting up your tests. You would put your authentication tests in a file like authentication.spec.ts, or profile page tests in profile.spec.ts, etc. What if you want to run a subset of those tests as smoke tests? Or you have a set of tests that you only want to run in Internet Explorer or Safari? If you are relying on file names, you could do something like create a file named smoke-tests.spec.ts, and copy/paste into it any tests you would want for a smoke test run. This requires duplication of testing code though, and can be difficult to maintain.

Filter by fixture or test name

You can filter what tests are run just by the fixture name:

$ testcafe chrome login.spec.ts --fixture "Bad Passwords"
# or by regular expression 
$ testcafe chrome login.spec.ts --fixture-grep "Bad.*"

To tell TestCafe what fixtures/tests will be smoke tests, you could name your fixtures something like:

fixture('Bad Passwords: smoke'); 
test('Incorrect passwords should show an error: smoke');

Then you could filter with:

--fixture-grep "smoke"
# or
--test-grep "smoke"

This could make the fixture names brittle, however. What if a fixture name has the word "smoke" in it, even though you don't want it to be a smoke test?

Filter using metadata

We can use the meta property to create a subset of tests like this:

// login.spec.js 
fixture('Login Page') 
 .page('https://some-app.com') 
 .meta('smokeTest', true) 
 .meta('ieOnly', true); 
// Or you can pass an object instead 
...
 .meta({ smokeTest: true, ieOnly: true})

With the meta property in place, you can now run tests like this:

$ testcafe chrome login.spec.js --fixture-meta smokeTest=true

You can even use the meta property on a test:

// login.spec.js 
fixture('Login Page') 
 .page('https://some-app.com'); 
test('should show an error with incorrect username', …) 
 .meta({ smokeTest: true });

and you can use the --test-meta option just like the --fixture-meta option:

$ testcafe chrome login.spec.js --test-meta smokeTest=true

Now we have a clean way of creating test subsets that don't rely on fixture/test name, or file name. Not only can metadata be used for filtering tests, but that information can be used in your own custom test reports.

Conclusion

TestCafe's metadata feature makes it simple to separate and organize test runs within your end to end suite, not only at a fixture level, but at the test level. I think you should definitely work it into your tests to get the most out of TestCafe.

Comments

[Ben Lind]

Great tips!

I couldn't get .meta({ foo: true }) to work with --test-meta foo=true because of the boolean. So I had to use a string, like .meta({ foo: 'yes' }) with --test-meta foo=yes.