Why TDD might not be that underrated

What triggers this post?

A while ago, the native web GmbH published this video on youtube (german language):

While I genuinely like the video and do not by any means oppose its content, I want to share a different point of view

Quick recap: What is TDD

TDD is the practice consisting in writing the test first and engaging in what is called the TDD-Cycle (also commonly known as Red-Green-Refactor-(Commit): Write a failing microtest, write the according program code, see the test going green, refactor your program code while still keeping the test green. While it can be severely head-damaging at first, with some practice you will find yourself not only producing better - that is, more testable, better designed and better isolated - code, you will also find yourself with a coherent guide to your development as well as an understandable documentation of your intent while developing afterwards.

The essentials of the video

In the video, Golo Roden says, in essence, that TDD is a great programming technique by any means, but falls short when exploring and learning. His core statement is that, in order to do TDD in a not-so-painful way, you need to know pretty well where you are going, or else you might find yourself needing to invest an excrutiating amount of energy refactoring your tests as you go.

A different point of view

First of all: this behaviour is by all means perfectly understandable, and is an often implemented one when coming to TDD. There's absolutely nothing wrong with it in my mind (I myself thought that way for a very long time), and I fully get that dogmatism won't lead you anywhere.

I happen to use TDD for almost everything, excluding maybe small, easy one-shot-scripts. I happen to use it a lot for exploration, and want, in the course of this blog post, to show how I deal with the (very much real) problem of having to adapt existing tests. My goal is to encourage you to try it out if you think that that can't possibly be a happy way of working. So let's dig in.

The most underrated feature of TDD

While it is pretty common knowledge that TDD will give you better structure and test coverage (I will come back to this point later though), there is one feature which I find missed in nearly every discussion I witness about TDD: the guidance which it brings to you while developing. To me, it is one of, if not the most singular, important benefits of the practice.

When I code, I need to get into a flow to be proficient. I manage this flow by managing my cadence: the more a problem I face is complex to solve, the more I slow down my cadence and decompose my work into smaller steps. TDD is the most efficient practice I've come across so far to enable me to do that, because I only ever write the test for the next step, and only ever work on this next step. When not using TDD, I find myself jumping back and forth, having trouble deciding what to focus on and ending up in yack-shaving more often than not. The cycle helps me to concentrate and to ask myself, on every step along the way, if this is really the thing I should solve now. This is the main reason why I don't want to deprive myself of it when exploring something: it channels and streamlines my work.

Changing tests and requirements along the way

I'll not lie to you: you will need to make some changes in your tests when you don't know what you get into. There are, however, several options you could explore in order to mitigate this problem.

Test-Tables

The "normal" way of implementing TDD is by writing microtests. Assuming, to take the example of the video, you want to implement a function isValid(IBAN string) , you will have several Tests which could look like this:

func TestIsValidIBANValid(t *testing.T) {
    given := "DE123456789123456789" //valid IBAN
    when := isValid(given)
    assert.True(t, when)
}

func TestIsTooShortIBANInvalid(t *testing.T) {
    given := "DE123" //invalid IBAN: too short
    when := isValid(given)
    assert.False(t, when)
}

func TestIsIBANWithoutLeadingCharsInvalid(t *testing.T) {
    given := "12123456789123456789" //invalid IBAN: missing chars
    when := isValid(given)
    assert.False(t, when)
}

There are however other possibilities to still leverage one test per test-case while grouping tests. In go, we use test-tables and table-tests for this, in PHP there is the data-provider which gives you the same possibilities. Other languages will have similar tools baked into the test-framework.

The concept is pretty simple: you define a test which is then executed in isolation for several, extensible outputs. The prior example will then look like this:

func TestIsValid(t *testing.T) {
    var table := []struct{
       testCase string
       given string
       then bool
    } {
      {testCase: "valid IBAN" given: "DE123456789123456789", then: true},
      {testCase: "invalid IBAN: too short" given: "DE123", then: false},
      {testCase: "invalid IBAN: missing chars" given: "12123456789123456789", then: false},
    }

   for _, tc := range table {
      t.Run(tc.testCase, func(t *testing.T) {
         when := isValid(tc.given)
         assert.Equal(t, tc.then, given)
      }
   }
}

This might not look like a microtest anymore, but: every test runs in isolation and checks one single condition. The only change is that you now only have one place where you need to make some changes when, for instance, you'll need to change requirements to include a validation on country level: you'll then need to pass a string countryCode to your function, which would result in this:

func TestIsValid(t *testing.T) {
    var table := []struct{
       testCase string
       given string
       withCode string
       then bool
    } {
      {testCase: "valid IBAN" given: "DE123456789123456789", withCode: "DE", then: true},
      {testCase: "invalid IBAN: too short" given: "DE123", withCode: "DE", then: false},
      {testCase: "invalid IBAN: missing chars" given: "12123456789123456789", withCode: "DE", then: false},
    }

   for _, tc := range table {
      t.Run(tc.testCase, func(t *testing.T) {
         when := isValid(tc.given, tc.withCode)
         assert.Equal(t, tc.then, given)
      }
   }
}

You'll have to rewrite your test-cases a bit, but not that extensively.

Test helpers

I don't seek to apply DRY principles to my tests too much. I find it way more useful in the long run to have my tests easy to be read in a linear way, and I like the profound documentation this adds to my code. Nevertheless some test helpers are very useful to me: when I instantiate a class for instance, I try to wrap this in a helper function in order to reduce the amount of changes I have to make if my instantiation has to change - for instance if I find out that I need another parameter passed to my constructor.

IDE tools

Changing a function signature is something which is well covered in IDE-tooling. If you use it, you can mostly identify at once where you need some adaptation.

Also, most IDEs (in most languages) will tell you about failures well before you compile if you bother looking. No need to even bother compiling before these errors are fixed.

Smaller steps

You need to adapt tests along the way, understood. The task gets easier the smaller the isolation layer you work upon is, as the scope of the change get's smaller. If your function is short enough, and does less enough things, You will have a smaller change delta. I typically start with the smallest possible thing and focus on broader API afterwards, when the smallest possible things are done.

Be pragmatic

Even when doing TDD you are allowed to deviate. If you sense that something will be a hassle as you might need to add several things, then don't test it right now. This is typically what I do for constructors as long as I don't know which objects I will have to pass to my class in the end.

So there's nothing overrated in TDD?

I would argue there is, but at a different level: I'm talking about the T of TDD.

A common misconception, in my opinion at least, is to see TDD as a testing technique. In my mind, it really is solely a development technique. The tests you produce during TDD are there to guide your development. They will result in a near-100% test-coverage, which means: every line of code (or most of them at least) will be covered by a test. This doesn't mean that your code implements the correct behaviour.

In order to make sure that this happens, you have to change perspective, and go from the development realm to the test realm. How might your code break? What edge-cases (both technical - off by one errors for instance - and logical - can my customer, whose IBAN I am trying to validate be living in a different country than the one in which their bank account lies? are austrian IBANs (they're shorter) validated correctly?) are there hidden in the problem you are trying to solve? What happens if an upfront service sends bizarre data? What happens if an underlying service fails?

TDD will help you to build the thing right. It will enable you to produce easily testable, well structured and maintainable code. It won't help you building the right thing. And it won't necessarily help you identifying regressions once your business requirements change. That is still on you to test after having completed your development - of course in an automated way.