Quite true, but far out speculation about the future of computer architectures doesn't really seem germane to talking about a new form of fast non-volatile memory.
It's just a popular sweet-nothing people have grown fond of parroting whenever memristors come up. It is impossible to really refute, because it speculates about the possibility of an architecture we haven't yet imagined. That plus how cool "brand new architectures, completely new ways of computing!" sounds to the layman, means you hear it every time the word "memristor" hits the headlines.
The joke, of course, is that (if memory serves) modern computers actually use the Harvard architecture- not Von Neumann.
When someone writes about the end of the Von Neumann architecture, I take it as dreaming that the poster's favorite language will someday be faster than C.
"You do realize that John von Neumann spent the last 10 years of his life singlehandedly developing a theory of computing based on cellular automata? The computer you're reading this blog rant on was his frigging prototype! He was going to throw it out and make a better one! And then he died of cancer, just like my brother Dave did, just like so many people with so much more to give and so much more life to live. And we're not making headway on cancer, either, because our computers and languages are such miserable crap."
With memristors it is not even remotely "far out". First, we can substantially modify the von neumann architecture by dramatically improving the capabilities of reconfigurable FPGA type devices, allowing them to run at the speeds of custom ASICs and reconfigure with a delay of only a few clock ticks. That alone is pretty astounding. Second, memristors could be used directly as logic and memory, making it possible to programatically transform blocks of memory into circuitry and back to memory, which would have implications beyond our current reckoning. And all of this is possible in the near-term within the next few decades.