Cover image by Isaac Smith on Unsplash
In my last post I talked about code coverage and its usefulness as an educational milestone for any aspiring developer to conquer.
I finished the post by saying that experienced TDD practitioners have little need for code coverage because they will be hitting 100% coverage by the very nature of their development process.
But unfortunately for us, we have no good metric for measuring how effective we are with TDD.
Software design itself is fairly subjective, so thatโs one reason why we donโt have a good measurement. None of us can agree.
If youโre a TDD practitioner, how do you know youโre any good?
The old โI donโt need code coverage because I do TDDโ excuse
Itโs been a long time since I bothered to use code coverage metrics as part of my regular development process. (To be clear, there are times when I do use code coverage--to verify coverage of a unit that I didnโt write, and to check how Iโm doing when Iโm using test-after with legacy code.)
When I have clients ask me why I donโt bother with code coverage, I tell them I follow โstrictโ TDD so I donโt need it.
But itโs an excuse. I say this because people trust me that when I say it. They think itโs true, a fact. But of course itโs not. I say it because Iโm lazy and code coverage it just one of those things that very rarely helps me in my day-to-day. And itโs mostly true.
But what happens if Iโm having an off-day, or Iโm in a rush?
What happens if I make accidental mistakes in my tests?
What keeps me honest?
More to the point, how do I know that Iโm writing good tests, and how do I know Iโm any good at software design in general?
I donโt really have the answer to that. Thereโs only one way I know that really helps: working with others.
Keeping yourself honest by working with others
Pairing, and mobbing, helps you follow the TDD cycle religiously. It goes some way to stop you slipping up, cutting corners, making mistakes. Plus, itโll improve your software design.
It can help with building the right thing, too. When you mob you might have the opportunity to include a QA person within the mob. With any luck, theyโll have some input on what requirements you may be missing.
Then, as the mob signs-off work, everyone involved should be able to say they they believe they tested everything to the best of their ability.
If you have any of your own ideas about how to measure your TDD skills, Iโd love to hear them.
Top comments (10)
I tend to measure my skills as a programmer based on how fast and how well engineered I am able to build features and fix bugs. Generally using my passed self as a reference but sometimes comparing to my coworkers.
Speed is an obvious metric. Quantity I judge by the user experience and the quantity of bugs found over time.
Using TDD helps me build bug free and faster then without. So I don't usually judge my TDD skills as much as the results of TDD.
I do have code coverage automated with every test run so I can see it basically stays at 100%, especially useful with test watchers.
I do consider myself well versed in TDD because I teach and run workshops on it though. But a separate metric.
Edit: forgot to mention code quality such as readability, DRY, SOLID, YAGNI, etc. Very important metric. Tested code is great but readable is better, if I could only choose one.
"DRY, SOLID, YAGNI, etc. Very important metric" -> are these metrics? How do you measure them?
These are sort of byproducts of existing metrics. I think of them as the result of having a solid codebase that is sort of abstract but very tangible when you get in and do some work.
YAGNI or "You aren't going to need it": When looking at the codebase, take the number of features/conditions coded and divide it by the number of features/conditions that are used by your users at this moment. You want a value as close to 1 as possible. Note that you should take into account features designed, developed and deployed but is not popular as unused. Such that if you developed something and only 0.1% of users know and use it, it is basically not used. This can maybe be hard to judge but it is possible.
SOLID: Hard to judge but if you as a developer, or as a manager request of your developers, to do a new feature or system and they respond with something like. "That sounds good, but we will need to do a bit of refactoring" or "that is not possible and will take a lot of work to get that". This is a sign the code base is probably not built on a solid ( see what I did their? ) foundation. Note that SOLID is not the only method of doing so, and may not fit for your language, framework or patterns.
DRY or "Don't repeat yourself": This one is similar to YAGNI, as part of most ( if not all ) code coverage analysis tools you get two important values. The number of statements and the number of branches. I am always trying to maximize the ratio of statements and branches over features delivered. That is, how much value to the user can I get with the least amount of code written. Preferring to use statements over lines as lines can be deceiving value in a lot of languages. You tend to want this value to be lower as it means you add as much value per statement and branch as possible.
These metrics can be a bit abstract when it comes to features or value to the user. But you can usually ground it in some tangible value as an approximation such as the number of rest API endpoints ( assuming they are all used ), average time delta from bug report to deployed fix, developer onboarding and ramp up time. Combined with some quality metrics like uptime, e2e response time, memory usage or similar you can get a lot of measurements.
There are three points here:
1 Programming is just a small part of software development and so techniques like pair or Mob programming make perfect sense.
2 TDD is another way of acknowledging the importance of all the stuff that isnโt programming (which is also likely to change in the future).
So โmeasuringโ 2. is simply measuring the consequence rather than 1. which is actually the real issue (that lots of โprogrammersโ fail to grasp).
3 Code coverage is a measure of confidence that can be derived from the ongoing practice of TDD.
I tend to be pretty good at TDD when the task at hand is "easy", when I know what I'm doing and don't need to experiment much, basically doing things I've done before.
When I'm working on stuff that may or may not work (trial and error) I get very frustrated with TDD and feel it's just slowing me down in my ability to prove a concept. Of course I will always go back and write some tests, but that's far from being TDD.
I wonder how people handle these situations? Really curious :)
I spike a lot (i.e., write code without tests) until I have an elegant design figured out. Strict TDD would then have you scrap that solution and re-write it with tests, but you may just want to test after instead. If youโre doing it that way you have to be very critical as you comb through your existing solution for untested behavior. (Thatโs one occasion when a locally-run code coverage tool is useful).
How do you measure your TDD skills? You don't. Fundamentally, TDD skills don't mater, as they aren't the end goal. You should measure your skills on how much working software you deliver to your product owner and how many defects are raised against it as well as the severity of them. The aim isn't TDD, or coverage; that's purely a way of working for an engineer to deliver work with a high bar for quality.
About mobbing; I'll be open and honest - I dislike mob programming.
The general idea being, "I spend more time programming, because I spend less time stuck.".
If you are constantly getting stuck then read the docs, speak to people about specific problems, or go on training. I really don't agree that holding up another 2+ engineers in the team to work at the pace of the slowest engineer is the right approach. I get fed up at the idealism that seems to imply that it's OK for 3+ people to not be able to do their jobs and instead all sit around one computer in the name of collaberation. No. Collaberation is working together to achieve a common goal without artifical limitations such as using 1 machine. How about we do a few playbacks or show-and-tell style sessions to run through how the tools we use are actually working? How about we sit at a desk next to each other and make sure that it's entirely safe and OK to ask questions whenever you need? That's real collaberation. Individuals and interactions... that's where it's at; and it doesn't require 3 people to watch one screen.
Am I saying I don't make mistakes? No - I do quite often. I write tests as I go that find a lot of them, and I do this because I care about my work, not because I've got some person sat with me muttering "should we test that?". We also have a QE function that I'll work closely with whilst developing a feature or fixing a bug. We define scenarios together, and either of us can write the steps for any test. 3 people is half the capacity of most teams. Does it make sense to have half the team watch whilst one modifies some json file, messes with some properties and waits to watch something deploy? The only thing worse is watching a full team sat around whislt one person creates stories in jira.
I agree with a lot of what youโre saying. Certainly about measuring yourself by your delivery rate and your bug rate.
BUT I think youโre taking quite a binary view to say that TDD skills donโt matter. Because one way to improve your delivery rate would surely be to โimproveโ your TDD, for some definition of TDD.
Just based on my own experience, it took me many years (close to a decade?) of unit testing to get to the point where I could confidently say that I was testing well. Even though Iโd read a bunch of books about testing and software design, it wasnโt until I joined a TDD consultancy that I worked with people who actively showed me how to improve my code. An example would have been number of lines in a test, and keeping tests DRY: before that my average test might have been 7 lines; for them it was 4 lines or less. As soon as someone pointed it out to me as I was instantly able to improve. That kind of little thing.
Now that I work with juniors a lot, I know they get TDD but thereโs a whole bunch of software design they still donโt get, like how to spot opportunities to create abstractions. Pairing with them helps me point out patterns in their code that arenโt directly visible, and then convince them that pulling out the abstraction is worthwhile.
The other reason I like pairing and mobbing is because, as someone who is very easily distracted, it actually makes me more productive, since I'm not just checking the news every 5 minutes. I appreciate not every developer suffers from this problem, and I do think that proponents of mobbing underplay this psychological benefit, but itโs certainly a major benefit for me.
Wrt JIRA, Iโm completely with you on that one.
This is so true. Iโve had this convo numerous times.
Me: identifies a boring test i donโt yet know how to write. Being lazy, say nothing.
Co-worker: Should we have a test for that part?
Me: Yeeaaah... we have to write that. Thanks for keeping me honest.
Btw, thanks for this brilliant series of posts ๐ช
Thanks for reading :)