You can use test refactorings to get from brittle tests to robust tests. I describe how to do this (and how to know that you should do it) on my web site http://xunitpatterns.com and my book. (xUnit Test Patterns – Refactoring Test Code)

Closer to what I really meant is: “Tests are intentionally brittle code. Adding them to a codebase will increase the average brittleness of the codebase as a whole (ie app + test) ”

I’ll update the post sometime.

I was drawing a comparison with this: http://en.wikipedia.org/wiki/If_a_tree_falls_in_a_forest

I meant that the tests don’t make a codebase brittle; they may allow its sensation but its reality is independent of the presence of tests.

I do agree that tests can themselves be brittle and I really like the 4 qualities you explored in your last post.

Sorry, Abhi, you lost me with this bit??

It’s certainly provocative, but also somewhat misguided IMHO but I’ll try to avoid digressing into a philosophical discussion about trees falling in forests.

The experience you describe says more about the quality of the tests themselves rather than the affect their mere presence has on a software system. Perhaps one lesson here is that poorly written automated tests can be more brittle than well written ones.

If unit tests are well written they should be focused and orthogonal only one should ever fail at a time.

If functional tests are well written, in addition to other attributes, they should be agnostic of implementation specifics meaning none will fail due to a mere refactor.

If many well written functional tests fail it might be a sign you’re trying to refactor a bicycle into a Ferrari (analogy plagiarised from http://www.eaves.org/jon/work/agile.shtml)

Now there’s a series of provocative statements