I feel like we're just trading one bottleneck for another here. So instead of slow storage, we now have a system that's hyper-sensitive to any interruption and probably requires a dedicated power plant to run.
Cool experiment, but is this actually a practical path forward or just a dead end with a great headline? Someone convince me I'm wrong...
Sandia National Labs is one of the few places in the country (on the planet?) doing blue-sky research. My first thought was similar to yours--If it doesn't have storage, what can I realistically even do with it!?
But sometimes you just have to let the academics cook for a few decades and then something fantastical pops out the other end. If we ever make something that is truely AGI, its architecture is probably going to look more like this SpiNNaker machine than anything we are currently using.
It doesn't have built-in storage, but that doesn't mean it can't connect to external storage, or that its memory cannot be retrieved from a front-end computer.
Think of L2 cache (small access time, small capacity) vs. memory modules (larger access time, large capacity) on a motherboard. You get the large capacity and an access time somewhere in between, depending on the hit rate.
Yes, a different usage pattern could give you the worse of both worlds.
That's a very deep insight from queue theory and similar topics, I don't know if you wrote what you wrote with that in mind (I'm not being an ass, I just want to highlight this observation).
A system's behavior is highly dependent, to the point of it being fundamental, on how it is going to be used.
HPC jobs generally don't stream data to disk in the first place. They write out (huge) snapshots periodically. So mount a network filesystem and be done with it. I don't see the issue.
Cool experiment, but is this actually a practical path forward or just a dead end with a great headline? Someone convince me I'm wrong...