A number of interesting findings in this article

1) Absolute Test Code Coverage isn’t a sufficient predictor of quality 
2) Dev Org structure matters more than physical location/distribution 
3) Density of Assertions is positively correlated with quality 

But most interestingly:

4) TDD generates 60-90% fewer defects, but a 15-35% longer dev cycle

“We picked three development projects under the same senior manager and looked at teams that used TDD and those that didn’t. We collected data from teams working on Visual Studio, Windows, and MSN and also got data from a team at IBM, since the project was a joint study.”

The study and its results were published in a paper entitled Realizing quality improvement through test driven development: results and experiences of four industrial teams, by Nagappan and research colleagues E. Michael Maximilien of the IBM Almaden Research Center; Thirumalesh Bhat, principal software-development lead at Microsoft; and Laurie Williams of North Carolina State University. What the research team found was that the TDD teams produced code that was 60 to 90 percent better in terms of defect density than non-TDD teams. They also discovered that TDD teams took longer to complete their projects — 15 to 35 percent longer.

I’d be interested in digging into that last one a bit: fewer defects are clearly A Good Thing, and will help towards lower costs/higher throughput, as you’re avoiding the Waste of fixing them.

If that comes along with a tendency to increase dev cycle times, then the challenge (a la Mike Rother) is not to view this as a factor for not using TDD, but as a spur for problemsolving on the lead time while sustaining the quality/cost advantage.

My instinctive thought would be to avoid doing it as one big batch and approaching it as a WiP-limited flow. Which I would have thought would be a lesson already learned at Microsoft, given the origins of Kanban in a Software context.

Follow me
Latest posts by Martin Burns (see all)

CC BY-NC-SA 4.0 Test Driven Development: Better but Slower? by Martin Burns is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

%d bloggers like this: