I think it’s obvious to everyone who’s slogged this far that I have read more of the book than you, that I have thought more about the concepts espoused the book than you—and even the authors!

I never denied The Pragmatic Programmer is an influential book (so is Code Complete which is also deeply flawed). It is because of that acceptence that five years after it’s publication, I didn’t find a new idea I hadn’t already run across in the first 75%. At the time, I attributed this to the influence of the book—but I’ll admit that I could be wrong and it may be that the writers, like the developers of Ruby on Rails, had nothing to add but clever repackaging.

If you had read the book, maybe your arguments against the book would hold my interest. I can argue with you about the book all day long, but it’s not going to do any good, because your arguments will not be based on the text, they will be based on what you think the text says, sometimes based on nothing more than the title. What’s the point?

Furthermore, I am not defending the book itself. I am not a devotee nor blind follower of the book or its authors. As I said originally, not everything in that book is going to work, and one needs to use critical thinking skills to determine if their project would be helped or hindered by heeding the authors’ advice.

Agree or disagree, it IS one of the premiere books in our industry, and it’d be a shame not to read it.

Notice how he, quite independently of me, mentioned the pareto principle (he calls it the 80/20 rule, I called it the 90/10, but it’s all the same).

An interesting post worth reading.

(Also, I might note that I originally went straight from Paul Jones’s summary to the actual IEEE paper so I hadn’t had any cause to defend Jakob, but reading his points, I can see that, while he may not have done Test Driven Development, there is no doubt that he’s written more unit tests than many of his detractors and that he has a thorough understanding of many agile processes. I’d say that discounting him on those two counts is more speaks more of the sloppiness of his detractors than on an indictment of him.)

Yours is the very definition of ad hominem: to discount the message because of the messenger. His argument correctly points how the interpretation and the results of the IEEE study are misinterpreted in order to further a cause (test first development), he is quite magnanimous in attributing it to confirmation bias, whereas the evidence strongly implies that the misrepresentation of the study by these Test Nazis is deliberate (i.e. the post did not just stop with the abstract).

They have seen the black and called it white.

And just because Jacob points out that the emperor has no clothes, you say we must discount him because he hasn’t been audited, doesn’t have an e-meter and hasn’t paid for his Operating Theta III level ups!

I have read a number of accounts write about the virtues of test driven development starting with Kent Beck and William Fowler. And yet, outside a couple of isolated use-cases (XP used on simple projects like a billing system for Chrysler corporation), I have not seen it applied to huge projects successfully. It’s largest successes have been in refactoring very delicate parts of testless enterprise spaghetti code rewrites but the size of the project was strictly determined from the outset, and the pareto principle was already applied (it wasn’t the entire codebase that was refactored using TDD, only the most delicate parts).

You still have the burden of proof because the vast majority of projects out there, even in enterprise web space where the TDD zealotry is at its zenith, are not built using test-driven development. And among the most successful large-scale web projects out there, I cannot think of a single one that uses Test-Driven Development throughout the entire codebase. The conventional wisdom—and the bulk of the studies in this area confirm this—is that more tests equals better quality and less productivity and that Test Driven Development encourages more tests.

Indeed, as you point out, TDD isn’t a new theory. And yet, the overwhelming weight of evidence is that it is not close to a magic bullet of proven benefit in certain instances, many of which have been mentioned already. As for SitePoint, why bother? Paul Jones and Marcus Baker are active participants there already and I have a number of people call me out privately and publicly on my blog entries.

As for the question about “Where can people in the world of PHP go to learn about this stuff?” My solution is to say, “If you are going to exhibit blind devotion instead of critical thinking, I’d rather you not stay in the PHP world and go to the Ruby world where your zealotry will be appreciated.” The PHP world has done an excellent job of adopting the most successful programs, practices and products in the web space. I myself have been lecturing for over five years now on unit testing, agile processes, and design patterns in PHP! (I first read most of the The Pragmatic Programmer four years ago.)

…

The 90/10 comment shows the consequences of a choice, a consequence, not mentioned once in that entire chapter of The Pragmatic Programmer. That is not unexpected, today a lot of books on Pattern Literature seen never to mention the consequences of a design pattern choice. This is why there is so much backlash.

If you look at the IEEE study, you’ll find this common sense is exactly confirmed by the study. The overall quality of the control group was slightly better than the TDD group, but not in a statistically significant way. The number of tests written by the TDD group was more than the control group. And therefore (Wow! the pareto principle): The control group was much better at writing more useful tests with no detriment of quality!(*) Common sense (born from experience) strikes again. Who would have thunk? Does that mean Test Driven Development is shit? No, but it certainly shows it isn’t THE shit. (The control group were “Test Last” not “No Test” people.)

As for the Paul Jones comment, I was corrected up above. It’s rather lame of you to further that after what clearly occurred after I moderated his comment. But not to be expected from someone who needs to resort to ad hominem to defend his points.

I have asked each of you, who is being “pragmatic?” Me, who has done TDD and says simply that TDD is probably best applied to situations in software rewrites where quality of the main is valued over productivity of its individuals (medical equipment, financial transactions, but hardly in a release-often, easily-revertible, large-scale, consumer-facing website like Facebook or Tagged :-D) and because it has these consequences (productivity/quality tradeoff, and up-front egalitarian approach to test-writing) is not a best practice to be applied blindly; or the authors of The Pragmatic Programmer who shout at you that “Design to Test” is the very best way of coding everywhere and always—with not a practical example where they’ve actually experienced it, but numerous contrived theoretical suppositions on why you this is the case.

(*) I will point out that there is some weirdness going on in the IEEE paper, when I skimmed it the confusing part is in the “PROD” measurement (which is supposed to be the amount of “effort” taken to complete a user story, but a common sense and more measurable definition of productivity would be the amount of user stories completed in a given unit of time!). The conclusion in the paper is that the TDD-ers had a statistically-significant higher PROD than the control group. It’s hard to understand what is going on in the study because the strong correlation between the PROD rating and the number of tests. Few disagree with the basic statement: more tested code is in general more reliable, and it this is the reason test first design is under consideration at all (it almost always guarantees more tests get written). Go look at the last graph in the paper. It’s very strange (though not surprising).

What I conclude from the study is that TDD exhibits its strongest improvements among the worst programmers (by causing more tests to be written overall), but that it may pull down the quality of the best programmers (certainly enough to make the difference in overall quality between the test first and test last group slightly favor the test lasters); that test-first design seems to decrease the predictive powers of counting tests as a function of code quality; and that another study should be referenced when exploring the general concept of “productivity” (in the allotted time for the user story, I could picture the best of the control group were just idling trying to think up Unit Tests to fill their time).

A common thread in all of these blogs is the attempt to frame the discussion to match the author’s lack of experience. The onus is wrongly placed on other people to make the effort to teach rather than on them to make the effort to learn. The language used is also a bit of a giveaway. Sneering at imagined dogmatism or religious zeal in a subject of which you have little knowledge is a pretty poor thing to do.

TDD isn’t a new theory passing through an initial peer review stage. I don’t have to “prove” it. On the other hand I’d be glad to discuss TDD with anyone reading this blog who is genuinely curious. Drop by sitepoint any time and I’ll promise you a eureka moment which will change your programming life.

This is the real issue: where can people go in the world of php to learn about this stuff? How can we encourage more php programmers to try it out?

To answer your 90% / 10% comment: the main purpose of TDD is to elicit 100% of the design rather than to “test” - although that too.

PS: P M Jones isn’t a SimpleTest developer.

Sorry about the error, Paul. I always confuse you guys.