The software testing tool of my dreams

The dream

Let's not kid ourselves, nobody is interested in writing tests.
It's every software engineer's dream to see tests generated automatically.
But generated from what?

In most project, there is no formal method used for behavior specification, but at least there is some description in plain english.

For example, to describe a gcc simple use case, it could be :

## Scenario: compiling and executing an hello word

- Given the new file `main.c`
```c
#include <stdio.h>
int main() {
printf("Hello, World!");
return 0;
}
```
- And given there is no `main` file

- When I successfully run `gcc main.c -o main`
- And  I run `main`

- Then the output is `Hello, World!`

It's a markdown file, using a classic Gherkin framework, perfectly suitable for documentation.
And it contains all the necessary information to actually run the test.

So why should I have to rewrite the same story in some test framework using some specific programming language? Not only is it a useless task, but it also creates a constantly problematic second source of truth.
No, I should be able to run it just as is.

And that's my dream : be able to just run the doc

Let's try something...

In spring 2024, I started playing with this idea.
Most of my end-to-end tests use the same framework :

• setup some input
• run the app with some options
• check the output

The needed vocabulary being limited in that context, I considered that it was not unrealistic to use an input file in plain english like the example above instead of some script, and decided to build a prototype. It turned out that implementation was... manageably complex, and it wasn't long before the prototype was usable.

I use (and love) the test first approach. Not because of the usual bullshit "coverage" argument, but because writing a use case is a great help to design, and an effective way to communicate about the behavior. As a consequence, description like the above being already part of my design process, the test comes for me at zero cost.

Ultimately, the prototype demonstrated not only an immediate boost in productivity, but also a rapid learning curve.

I want it too!

Good news, this tool, called bbt, is now available here.
It's easy to install on Linux and Windows.
(It compiles on MacOS, but isn't tested on my side for now).

Current version is 0.0.6, meaning that bbt is still in a early development stage, and open to improvement : comments, use cases, good practices, ideas, pain points are all more than welcome here.

Acknowledgments

• Many thanks to bbt's early adopters and contributors Paul and Manuel
• Many thanks to AdaCore, who awarded bbt the Ada Crate of the Year Prize!