> The greatest performance improvement of all is when a system goes from not-working to working
Except when speed is part of the definition of working. For example deep learning. The correct implementation was not enough until the hardware was fast enough.
> The correct implementation was not enough until the hardware was fast enough.
This doesn't refute his argument, which I read as being, essentially, against performance optimization of software during initial construction. Knuth famously bemoaned premature optimizations, as well.
Speed isn't in the definition of "working" for deep learning. Speed is an implementation detail. Perhaps a very critical one, but still an implementation detail. One can evaluate the success of a deep learning program (e.g. "is it working?") without knowing how much processing power it used.
Except when speed is part of the definition of working. For example deep learning. The correct implementation was not enough until the hardware was fast enough.