[TDD] encourages a focus on sorting out detail to pass tests rather than looking at the program as a whole.
I have actually found the opposite to be true.
I have to make large refactorings to move things around to arrange the whole system so that each part can be tested without too much effort. To do this I have to view most things in terms of the interfaces they provide. On the test side, I have to write the test code so that the what the test does is strictly separated from the how the test does it, so that changing the system causes only minor changes to ripple to majority of the test code.
Based on this, it seems that programming with TDD is a distinct skill-set that requires significant effort to get reasonably good at, i.e., to be more productive than without TDD. I also have given it a try on medium-size projects and it does pay off in terms of simplicity of the design (I have to manage dependencies and decouple external systems and components quite heavily), low defect rates in production/qa, waaay less time spent in debugger, and high velocity (based on customer and product owner feedback at least).
However, the problem with TDD is that all of the above (tests decoupled from interface, interface decoupled from implementation, system decoupled from external systems, components decoupled from each other, design skills to recognize this, and refactoring skills to do this fast enough to remain productive) need to be done well enough at the same time. Otherwise the approach falls into pieces at some point.
To paraphrase Uncle Bob from some years ago: I have only been doing TDD for five years, so I am fairly new to it.. Half of the programmers have that much experience in programming in general, so the amount of time required to hone TDD and refactoring skills may not be there yet.
So maybe I am saying that you are not doing it right, but I don't really know. Maybe I am wasting my time writing tests, but anecdotally, I seem to enjoy extending and maintaining the TDD-based pieces of code more than the non-TDD pieces in our codebase.
I've been doing TDD for 4 years now, and I would totally agree that it takes a lot of time to hone the skills. My experiences of TDD on projects is quite similar to yours. However, it took me a year to simply really understand how to do TDD in any kind of sane way.
TDD is not something you can easily pick up in 6 months without a lot of mentoring and training from experienced TDD'ers.
Part of the problem is that there are not that many TDD codebases or TDD'ers around. Also, this is probably not something you can pick up while doing toy projects or school assignments. The benefits start to show in the 100 kloc and above magnitudes, and as there are so many ways to paint yourself into a corner with bad overall design, coupling, unmaintainable (or slow!) tests, chances are, you don't figure out all the necessary things yourself. On top of that, there is no time to learn this much in most dev jobs, so you are left to learn with hobby projects (which do not usually grow big enough).
I have actually found the opposite to be true.
I have to make large refactorings to move things around to arrange the whole system so that each part can be tested without too much effort. To do this I have to view most things in terms of the interfaces they provide. On the test side, I have to write the test code so that the what the test does is strictly separated from the how the test does it, so that changing the system causes only minor changes to ripple to majority of the test code.
Based on this, it seems that programming with TDD is a distinct skill-set that requires significant effort to get reasonably good at, i.e., to be more productive than without TDD. I also have given it a try on medium-size projects and it does pay off in terms of simplicity of the design (I have to manage dependencies and decouple external systems and components quite heavily), low defect rates in production/qa, waaay less time spent in debugger, and high velocity (based on customer and product owner feedback at least).
However, the problem with TDD is that all of the above (tests decoupled from interface, interface decoupled from implementation, system decoupled from external systems, components decoupled from each other, design skills to recognize this, and refactoring skills to do this fast enough to remain productive) need to be done well enough at the same time. Otherwise the approach falls into pieces at some point.
To paraphrase Uncle Bob from some years ago: I have only been doing TDD for five years, so I am fairly new to it.. Half of the programmers have that much experience in programming in general, so the amount of time required to hone TDD and refactoring skills may not be there yet.
So maybe I am saying that you are not doing it right, but I don't really know. Maybe I am wasting my time writing tests, but anecdotally, I seem to enjoy extending and maintaining the TDD-based pieces of code more than the non-TDD pieces in our codebase.