What is the ROI of Test-Driven Development?
June 26, 2021Common break-even calculations for TDD are wrong when experienced TDD devs are involved becuase TDD makes development faster.
Test-Driven Development, or TDD, is the practice of writing tests, then code, in short iterations of a few seconds to a few minutes.
Should everyone do TDD, as some claim? That probably depends on the situational ROI, right?
Perhaps some code has a long-term ROI on TDD, and other code doesn’t.
Perhaps some situations are appropriate for the up-front cost of TDD, because it’s expected to pay for itself in maintainability over the long haul, while in other situations the code simply won’t live long enough to benefit.
The per-project ROI of TDD
In other words, we need to calculate the ROI of TDD in a given situation; a product, project, or feature. And we probably assume the calculation looks something like this:
In this chart the X axis represents sprints, but could be days, months, years, or any other unit of time. The Y axis represents effort, but could be story points, hours, days, or any other meaningful measurement of effort.
We expect a jump at the beginning as a new feature, module, or product is developed for the first time, and we expect that jump to be greater if we apply TDD practices, because TDD involves writing more code.
But we acknowledge that the extra effort pays off in easier mainteneance, so if the project lives long enough, we expect the TDD and non-TDD lines to converge at our break-even point. From this point forward, TDD has paid for itself.
Anyone who has ever done a break-even calculation is familiar with the general shape of this chart.
It’s completely wrong for the types of projects where TDD applies.
A more accurate view
Let’s recall that the reason we use TDD isn’t primarily to have automated tests at the end. We can get that with test-last development, right?
The reason we use TDD is so that we have faster feedback loops during development. The reason we want faster feedback loops during development is because it makes development faster.
With that in mind, a more realistic chart would look like this:
Here we see that the break-even point for TDD is immediate. Despite the poor name, TDD is about development, not about testing, so it makes sense that the biggest payoff is during the development stage.
This means that even if we throw away our tests after each TDD session, TDD still pays for itself (once you’re proficient at TDD of course—more on that below)!
Of course, you’re probably thinking “I agree tests are good, but I prefer to write them after I write my code, so I don’t need TDD.”
Well, naturally the ROI chart comparing TDD vs Test After looks a lot different than TDD vs No Tests. But TDD still comes out ahead, since the biggest benefit of TDD is during development.
If you know you’re going to write a test anyway, why not write it before your code, so that you have the benefit of that test during coding?
There’s also the issue that a project with no tests at all (test-after or test-first) tends to become harder and harder to modify over time.
A more complete view
With these two modifications in mind, here’s the updated chart for your consideration:
TDD still wins, but test-after is still a huge improvement over no tests at all.
These charts, I hope, make logical sense. But for many (myself included) there can be a disconnect between what they’ve experienced attempting to do TDD, and what these charts indicate.
Doing TDD often feels slower than not doing TDD. Why?
Because there actually is a penalty to be paid for TDD. The problem is that this penalty is not on a per-project basis. The penalty happens during the learning phase of an individual or a team adopting TDD.
This is primarily due to learning a new mindset, not learning a technical skill. Red/green refactoring is technically simple, it can be explained in just a few paragraphs.
If you’re willing to apply yourself, I estimate that most developers can become sufficiently effective with TDD that it will save them time with 3-4 months of steady practice. The exact time frame depends a lot on your experience, the type of project you’re working on, the testing frameworks and tools available in your language of choice, and many other variables. So naturally, your mileage may vary. But during the early period, TDD will often be extremely frustrating, and it will feel counter-productive.
Don’t give up!
A merge a day keeps the conflicts away
Merge a minimum of one PR each day. Make small PRs. Don't worry if the feature is incomplete, only that each PR works.
Which skill is more important: Testing, or debugging?
One of these skills, if you're good at it, diminishes the need for the other other.
Work from the bottom up
When copying a test, make your changes from the bottom up, renaming the copied function as the last step.