What the post really says is, if your big business really depends on the results of days of computation, do your software in the language most efficient for the developers and throw money at hardware, because the value of the results is way more than the savings in hardware costs from using the most efficient language. He is arguing the case of lisp vs FORTRAN in the context of oil exploration in Norway.
He is not arguing purely for throwing money at hardware:
A good engineer would balance the tradeoffs and solve the problem within the existing resource constraints. A theoretical computer scientist would whine until he got a big enough machine to implement the mathematical solution with the least amount of fussing about the constraints of the real world.
The post at the start of the Usenet thread asks if any Lisp implementation can handle inverting a 20,000 by 20,000 matrix. Naggum is arguing that if you really had that problem, you are probably working in an industry where the choice of Lisp implementation is irrelevant, because solving the problem is so lucrative that you could afford to write your own Lisp implementation, if needed. He's saying that you will throw money at both hardware and developers to get the job done.
I do AI. I used to live by McCarthy's quote that "Real AI was probably possible on the computers from 30 years ago, if only we'd known how to program them". So I bought my latest computer thinking: I've got 1000 times more processing power than necessary, so why buy a top-of-the-line computer? Why care about the mere factor of 2 that I could get for an extra 2000 bucks?
What happens is this: today I had to solve software mini-problem #403 to put in experiment #14. I coded the naive O(n^2) algorithm, and damn it, it started paging, and performance crashes. I can't finish the experiment, unless I spend a few days looking up the right way to do it, O(log(n)); implementation, debugging, screams. Then I'll throw away the whole thing when I see that it doesn't work.
Ah, I think, if only I'd have a 64 bit Common-Lisp.
Arh, I was hoping this wasn't true... I'm already using SBCL. The point is: software and hardware constraints limit your flexibility, and Naggum's argument that "a good engineer works around those constraints" is true if you have a fixed problem to solve and a lot of time, but it's not when rapid prototyping, which is what most of us should be doing. He says that no one should need a 64-bit CL (in 2002); I took his line back to make a crummy newspapery punchline, which turns out to be false as well.
Naggum's argument is that if you're working with a large dataset, it must have taken you a lot of time/money to obtain it and so you should be able to afford custom hardware and software to deal with it.
I say that naive, brute force methods on modest datasets will generate huge memory and CPU requirements. The solution is either: waste some programmer time coding a better solution, or get better hardware/operating system. There's a tradeoff. Modern programming is super wasteful by the standards of 30 years ago. We traded computer ressources for programming convenience. There's still a lot of that to be done before we have all the convenience we want.
Computers will be fast enough when I can do arithmetic with Church numerals.
Is he seriously saying that nobody needs to use large matrices in these days of internet data? Google uses online implementations of some algorithms because they can't hold the whole matrices in memory at the same time. Maybe Google is not the average hobbyist, but hell yeah there are some large data sets out there for free, exhibit A the World Wide Web.
I don't necessarily disagree with him about this, but I do take issue with his disdain of academic or hobbyist projects.
The internet is increasingly providing free access to very large and very interesting data sets, and so of course we're seeing more and more people take advantage of this to do interesting things with this data. And as working with these sorts of datasets becomes more and more common, it is silly to think that we should have to reinvent the wheel every time instead of having tools evolve around their uses.
Moreover, we're in an age where a couple people can successfully take an idea from a hobby project and turn it into a working business with little more than time, effort, and knowledge. Given this, I would ask where the line is drawn between "I need to work with this data" and "what sorts of cool things can I do with this data". My personal belief is that the line is very fine and blurry, and that the work of "jerkoff hobbyists" today is likely to lead to the serious uses of the future.
"I am unable to keep my company in operation for health reasons, but it is one of those things I just cannot give up. Instead of being a commercial undertaking, it is now only a hobby. Until my health improves, which it might never do sufficiently, I am generally unable to accept programming projects, teaching opportunities, or requests to advise students, although I would really like to see what I can do if you would like my input."
I've read the website - I'm wondering what it was that actually happened to him. He was a character and although acerbic in the extreme made some interesting points.
