Is it just my being aware of it, or is this current of thought popping up in more places lately? "The Black Swan" comes to mind, "The Wisdom of Crowds" is along these lines. It's certainly not new - it seems to be one of Hayek's central ideas.

One of the things that greatly attracts me to startups is the ability to work like this... I loathe rigid schedules and attempts to plan out things in minute detail. What I prefer is an aggressive schedule (you need to set some dates, lest you wander), and a general plan of attack, and quality people that can change tack if needs be.

The 'Black Swan' is indeed a bit of mind-bender. I am reading it for the second time. I just noticed, that I have underlined something, in practically every page.

"Why most things fail" by Paul Ormerod is pretty good too, although the writing style isn't quite so direct, and it leaves you wondering a little bit more than the Black Swan.

I'm really glad to see that people are coming around to this school of thought. It seems very closely related to the decline of waterfall and the rise of iterative development.

It's really important that we (the people who believe in prototyping and iterative development) emphasize that this is not a chaotic way to approach a problem. We absolutely do plan - however, we plan by coding and prototyping, rather than writing detailed specs. Specs are good - but they emerge from the code and the prototype. We absolutely do consider design issues - however, we discover and implement these things through refactoring. And in the end, we may very well throw out our first approach and rewrite, if it appears to be necessary.

But I think this is leaving the realm of opinion and entering the realm of reality. The people who try to do the pure waterfall where an architect designs and writes specs, a coder writes code, a tester does testing - they seem to be losing badly in the race to get excellent software to market. From what I've read (successful startups, Google) the free market appears to be rewarding the better approach.

I've been learning about this transition in my software design class, and the funny thing is the waterfall method should never have been in the first place. The method is actually based on a misinterpretation of Dr. Winston Royce's paper, where he advocates a system like rational unified process, but a bit more nuanced.

I like to take this kind of thinking a step further...

Planning to code: good.

Coding in a garage: better

Developing in a living breathing organization: best

No matter how much time I spend on my startup, I always keep an active consulting gig on the side. Routine situations come up every single day that I would have never conjured up in my "ivory tower". The value of fresh ideas and feedback from live users far outweighs the income.

The problem with planning is that it oversimplifies your lived experience. Very few people live from step to step, they live by their desire, therefore the statement, "the best performers seesaw between "ideas" and "actions" is more of linear effect. You have an idea, you want it to become real, so you try, if some actions to get you to the idea, the key is knowing what actions best produce the idea. An now we are back to planning, which people do more subconsciously than consciously, and let's face it, sometimes you just need to cogitate on an idea for a few months and then it strikes you what to do to make it real. I think the more important question is how do you not loose sight of the ideas you want to make real?

I love this philosophy. I first realized this while writing essays. I used to stare at the "Keyhole Method" diagram and conjure an intro, examples and a conclusion. They always sucked but the diagram made it seem so easy. When I started writing on my own for fun I would just launch right into the examples and find a pattern as I went. It almost always turned out that the last paragraph I wrote, the moment of clarity that tied it all together, would become the introduction.

In the context of software development, the ideas expressed really fall into alignment with incremental, Agile development. While I have been fortunate enough to spend most of my career so far in an Agile environment, I have had some time in a more traditional waterfall environment as well. In comparing the two, it seems like a key benefit of an Agile environment is that momentum much more quickly builds upon itself. It's much easier to work with something tangible, tweak it and improve it, than it is to abstractly imagine what it needs to be like, and get it right the first time.

It seems like the key point of his idea is that induction is the key to success. Edison used a massive amount of induction (experimentation) and prototyping while exploring the big idea of a light bulb - and things worked out well for him :). I think you can also argue that a plan based on prior induction makes for a much cleaner, faster, and better built prototype.

i think knowing a system by playing with it gives a much better perspective than simply theorizing about it. generally, if you "know" the system, you don't have to do much theorization -- the solutions are obvious

unfortunately, not everyone is suited for that sort of learning. and in fact i would say most people aren't able to "know" using any method. for these people a decent plan can be beneficial, but technical subjects require a minimal degree of playfulness. their capricious details aren't very forgiving to fine plans, and you'll probably find most of these people don't become programmers :)

... or they become java programmers?

Wow. I like this article a lot. I think it captures the essence of what we're all talking about.

Having said that, it's not a one-size-fits-all kind of deal. I certainly wouldn't want my brain surgeon saying "Hey. Let's just cut this schmuck's head open and see what we get, okay? Where's that can opener?"

A startup looking for user desires and aligning them with economics is a simple system. Act more and plan less to do it right. Brain surgery, creating the space shuttle code, or integrating all of GE's client software is not a simple system. Here's the kicker: some of these are complex because of the underlying nature of the problem (brain surgery) and some are complex because of the underlying nature of the _people_ (GE). Still, the end is the same. No amount of fast prototyping and "users love me" is going to fly in a complex environment. Now you might invent it and sell it to GE, but that's a different story. Simple environments mean you have to plan to answer simple questions, like "am I making something that users want?". Complex environments mean you have to plan to answer all kinds of other questions, like "How do I communicate what we're doing to 30 important stakeholders, some of whom may not like our project?" BTW, I hate complex environments.

So yes, by all means make something quickly and adapt. But know when that strategy works and when it doesn't.

It is a one size fits all kind of deal. You just have to try putting it on the right appendage. The difference here is that surgery is a service and software is a product. I think that basically they're the same. You "develop" a service the same way you would develop a product. You practice. In effect, practice is the iterative design of a service, just like rapid prototyping is the iterative design of a product.

Now, the surgeon can't just crack open your head and see what happens, but before he ever does that he's going to have practiced cracking open heads and mucking about with them somewhere (probably on a cadaver). He's probably going to have made some mistakes doing it too, and learned from them the right way to go about cutting up people's brains.

Likewise, you don't just ship the first piece of software you scribble out and see what happens either. Now, certainly with a web 2.0 AJAX calendar-organizer-automated-fish-feeder there's a greater margin for error than brain surgery, but it's a difference in degree, not in kind. If you launch a site that just returns "Internal Server Error" when someone surfs to it, you're going to be about as successful as a brain surgeon who never learned how to handle a scalpel.

So the surgeon cutting up a cadaver seems to me to be the same as a programmer blasting out a broken version 1.0 of his software. They're both trying something out in a situation where they can fix mistakes and reincorporate them to product. The surgeon makes a cut, looks to see if it's in the right place and is the right size, if not he keeps trying until he gets it right. When he can make the right cut every time without thinking about it, his service of cutting people is finished being developed. Likewise, the programmer writes some code, and checks if it produces the right output. If not, he tweaks the code until it does. Same thing. You can't compare a finished version of one product (performing brain surgery) to the development process of another product (writing software).

Oh yeah, his won't work for all fields. Iterative development is excellent in software because the cost of construction (ie., running the code) is nearly zero. You can let constant feedback from application guide where you are going.

This is why interviewers like to ask about languages that aren't in the corporate playbook. If someone says "Ruby? Python? Lisp? No, I haven't had the opportunity to use any of those yet," you know you can pass. Not because all good programmers use ruby, python, and lisp, but because all good programmers know that you don't need to "have the opportunity", you just need to install the language and write some code.

It would be unconscionable to learn brain surgery this way, so if someone says "I haven't had the opportunity to operate on a brain yet," that's a reasonable statement.

I actually suspect that writing software is one of the most complicated things a person can do. If we didn't have the advantage of iterative development with cheap compile cycles (or run cycles, if you don't compile), then no system of any real complexity could be built.

Actually, I would bet that brain surgery is difficult, rather than particularly complex. Indeed, your scope for action is probably pretty limited compared with writing a huge program.

And... brain surgeons probably practice (start doing) on animals or something else before trying it on people. At least I hope they do.

I've always heard the brain was the most complex thing that we know about. And I've always heard that brain surgeons spend a considerable amount of time planning some operations due to the complexity involved (although I am sure not all operations).

The brain being incredibly complex is probably right, but my guess is that our ability to slice and dice it in beneficial ways is probably much more limited.

Still though, yes, in some activities you must do your utmost to get it right the first time.

It's also not an either-or choice. Sometimes you can plan to do two weeks of work, then observe, then do it again.

I think it's a mistake to view the spectrum as only having two spots, total planning and completely inductive. Learning how to mix and match is probably a better skill than blindly following either extreme, imo.