Rust BDD tests with Cucumber

This post is out of date. The cucumber crate version used here is obsolete. Fortunately, the new version is not only better, but also easier to use and has excellent documentation, which you can find here.

Cucumber is a tool for behavior-driven development (BDD) that uses a language called Gherkin to specify test scenarios with a syntax that is very close to natural language, using key-words such as When and Then.

Here I will not explain the particularities of neither Cucumber nor Gherkin. My goal is to just show you how to use them with Rust. If you know nothing about them, I recommend you to:

• Watch this soft introduction.
• Refer to the documentation, specially the Gherkin reference guide.

That being said, the examples I am using are elementary, so you should have no problem following along even if you have never seen Cucumber and Gherkin before.

---

Sample project

The code used in this tutorial can be found here.

To focus on how to use Cucumber with Rust, I decided to code a dummy multiplication function, so you don't get distracted by the idiosyncrasies of a particular project.

First, create a library crate:

$ cargo new --lib bdd

Then, remove the mod tests from lib.rs and code this:

pub fn mult(a: i32, b: i32) -> i32 {
    a * b
}

And that's all for lib.rs.

---

Setting up the manifest

The manifest (Cargo.toml) require these entries:

[[test]]
name = "cucumber"
harness = false

[dependencies]
tokio = { version = "1.9.0", features = ["rt-multi-thread", "macros", "time"] }


[dev-dependencies]
cucumber_rust = "0.9"

In [[test]] we have:

• name, which is the name of the .rs file where we will code the tests, cucumber.rs in this case.
• harness is set to false, allowing us to provide our own main function to handle the test run. This main function will be placed inside the .rs file specified above.

In [dependencies] we have tokio, which is an async runtime required to work with cucumber-rust (even if you are not testing anything async). You may use another runtime.

And in [dev-dependencies] we have cucumber_rust, the star of this show.

---

Project structure

We have to create a couple of files:

• tests/cucumber.rs, where we will have the main function that will run the tests.
• features/operations.feature, where we will code our test scenario using Gherkin.

At the end, we have this:

bdd
├── features
│   └── operations.feature
├── src
│   └── lib.rs
├── tests
│   └── cucumber.rs
└── Cargo.toml

---

Coding the test with Gherkin

This is how the operation.feature file looks like:

Feature: Arithmetic operations

  # Let's start with addition. BTW, this is a comment.
  Scenario: User wants to multiply two numbers
    Given the numbers "2" and "3"
    When the User adds them
    Then the User gets 6 as result

Here we have a context set by Given, an event described by When and an expected result expressed by Then.

Although my purpose is not to teach Cucumber, I have to say that this is not a good BDD scenario. Not so much because it is dead simple, but because it is a unit test in disguise. I have, nevertheless, kept it here because it has the virtue of making things crystal clear regarding how we are going to interpret this scenario with Rust. For best practices, check this.

---

Handle the scenario with Rust

From now on, we stick with the mighty Crabulon Rust.

World object

The first thing we need is a World, an object that will hold the state of our test during execution.

Let's start coding cucumber.rs:

use cucumber_rust::{async_trait, Cucumber, World};
use std::convert::Infallible;

pub enum MyWorld {
    Init,
    Input(i32, i32),
    Result(i32),
    Error,
}

#[async_trait(?Send)]
impl World for MyWorld {
    type Error = Infallible;

    async fn new() -> Result<Self, Infallible> {
        Ok(Self::Init)
    }
}

For now, we created an enum named MyWorld that will be our "World object", holding the data between steps. Init is its initial value.

The MyWorld object implements the trait World provided by the cucumber_rust, which in turn gives us the methods to map the steps (Given, When, Then, etc.).

The Error type is a requirement from this trait, as is the attribute #[async_trait(?Send)].

Step builder

Now it is time to actually code the interpreter. I will explain the code block by block, so it might be useful to also have the complete code open, so you don't lose sight of the whole picture.

Given

mod test_steps {
    use crate::MyWorld;
    use cucumber_rust::Steps;
    use bdd::mult;

    pub fn steps() -> Steps<MyWorld> {
        let mut builder: Steps<MyWorld> = Steps::new();

        builder.given_regex(
            // This will match the "given" of multiplication
            r#"^the numbers "(\d)" and "(\d)"$"#,
            // and store the values inside context, 
            // which is a Vec<String>
            |_world, context| {
                // With regex we start from [1]
                let world = MyWorld::Input(
                    context.matches[1].parse::<i32>().unwrap(),
                    context.matches[2].parse::<i32>().unwrap(),
                );
                world
            }
        );

        // The rest of the code will go here.
    }
}

After declaring the mod, we created our builder, a Steps struct that will store our steps.

The crate cucumber_rust provides three variations for the main Gherkin prefixes (Given, When, Then):

• The "normal" one that matches fixed values (e.g. when());
• The regex version that parses the regex input (e.g. when_regex()).
• The async version to handle async tests, something I am not covering here (e.g. when_async()).
• The async+regex, which is a combination of the last two (e.g. when_regex_async()), also not covered here.

I am using given_regex() to parse the two numbers. Remember that in operations.feature I specified this:

Given the numbers "2" and "3"

When you call a step function such as given_regex() you get a closure containing the World object and a Context. The latter have a field called matches that is a Vec<String> containing the regex matches (if you're not using a _regex step, the Vector will be empty). In this case, as I am using regex, it has three values:

• [0] has the entire match, the numbers "2" and "3" in this case.
• [1] has the first group, 2 in this case.
• [2] has the first group, 3 in this case.

This is the regex "normal" behavior. If you are not familiar with regex, this is a good intro (thank you YouTube for holding my watch history for so long).

With these values, I return my World object now set as Input.

Before we move to when, I have two quick remarks to make:

• I am not checking the unrwap() because the regex is only catching numbers with (\d). Sometimes you might want to capture everything with something like (.*) and validate the content inside your code.
• If you want to change your World object (for example, if it is a struct holding multiple values and/or states), just place mut before world in the closure, and you will get a mutable object.

When

builder.when(
    "the User multiply them", 
    |world, _context|{
        match world {
            MyWorld::Input(l, r) => MyWorld::Result(mult(l,r)),
            _ => MyWorld::Error,
        }
    }
);

This one is very straightforward. I use match to get the enum inner value, multiply both inputs and return the World object with a new value.

The function mult is ratter useless, but it has a role to play here: to show you how to import what we declared within the library crate.

Then

builder.then_regex(
    r#"^the User gets "(\d)" as result$"#, 
    |world, context|{
        match world {
            MyWorld::Result(x) => assert_eq!(x.to_string(), context.matches[1]),
            _ => panic!("Invalid world state"),
        };
        MyWorld::Init
    }
);

builder

Here I use regex again to compare the value that was calculated in the Then step with the value provided by Gherkin (which is, as I said, a very suspicious BDD scenario).

At the very end, I return builder.

---

The substitute main function

After the mod, we declare our main function:

#[tokio::main]
async fn main() {
    Cucumber::<MyWorld>::new()
        .features(&["./features"])
        .steps(test_steps::steps())
        .run_and_exit()
        .await
}

Here we are:

1. Creating a World object
2. Pointing to the .feature file containing the Gherkin test.
3. Adding the steps defined with cucumber_rust.
4. Running and exiting once it is done.
5. The await because cucumber_rust requires the whole thing to be async.

That's it! To test it, all you have to do is to run

$ cargo test

Rust will run the main function found in the file specified in the manifest: cucumber.rs. This is the result.

I recommend you to mess with the values and run the tests, so you can see how the errors are captured.

---

Much more can be done with cucumber_rust. I hope this tutorial helps you get started with it.

Cover image by Roman Fox.

Comments

[Kai Ren]

It's better and more ergonomic to use macros feature and proc-macro support for regular Rust functions to define steps. For some reason it has been removed from README and isn't shown in examples, but if we do look at previous versions, we can see it.

[Hakanai]

That approach brings with it a number of problems:

1. I would have to think of a name to give the function. Naming things is hard. Anonymous functions by their nature don't have this problem.
2. Whatever function name I come up with has to more or less match the step definition name, otherwise a reader gets confused when looking for it in a structure navigator. This initially sounds like it makes problem #1 easier, but if you have any placeholders inside the step definition, you probably can't have those in the function name, so you end up needing some kind of convention for naming them.
3. If the step definition ever changes, there is a risk of forgetting to rename the function to match, making problem #2 harder.
4. Sometimes the smart naming scheme you come up with to solve problem #2 results in two functions having clashing names, and then you end up disambiguating them with things like "_1", "_2".
5. Stylistically, having to name everything twice is not DRY. (It's literally WET, because you are Writing Everything Twice.)

Literally all these issues we experienced with Cucumber Java, so I'd recommend the Rust community not to copy Java here, and to make the most of anonymous functions. You have the privileged position of having the language feature from the outset. :)

[Kai Ren]

We've been using this approach successfully for several years, as of now. And none of the issues we've experienced were the ones you've described. I have no experience with Cucumber Java, maybe it just things work other way there, making the problems, you've described, notable.

We don't care about step functions naming in our codebase at all. They are none vital for the framework. If the failure occurs, it points directly to the file/line/column where the step function is defined, and IDE integration allows us to reach that place in one click. We don't need function names to disambiguate step function. More, the fact that they are regular functions, allow us to distribute them in the modules tree quite well, so our tests files rarely have more than 2-3 step functions inside, and so, naming collision never was a problem for us.

Using anonymous functions, on the other hand, doesn't allow us to keep steps in separate files, or makes this quite monstrous. Moreover, macro magic atop of regular functions makes things cleaner and much more ergonomic, empowering with additional batteries like compile-time regular expressions validation and similar.

You're welcome to get familiar with the cucumber-rs via the Cucumber Rust Book.

[Roger Torres (he/him/ele)]

Hi @tyranron , thank you for pointing that out!

[Sam J.]

Nice article! I must say it's a lot of code for 1 simple test case...
You didn't explain what you can do with the _world variable in the "given". It's a bit weird to have a second world being created in the same scope.

[Pritesh Usadadiya]

[[...Pingback...]]
Curated as a part of #21st issue of Software Testing notes