Everyone's right to celebrate the success of RISC-V, but part of me thinks it's a shame that there's relatively little architectural diversity (edit I should have said ISA diversity) in modern CPUs. MIPS, Alpha, and Super-H, have all but faded away. Power/PowerPC is still out there somewhere though. Apparently they're still working on SPARC, too. [0]
At least we'll always have the PS2. ...until the last one breaks, I guess.
I wish the barriers to using new architectures were lower.
For instance, suppose binaries were typically distributed in a platform-agnostic format, like LLVM intermediate representation or something equivalent. When you run your program the first time, it's compiled to native code for your architecture and cached for later use.
I realize I've sort of just re-invented Javascript. But what if we just did away with native binaries entirely, except as ephemeral objects that get cached and then thrown away and regenerated when needed? It seems like this would solve a lot of problems. You could deprecate CPU instructions without worrying about breaking backwards compatibility. If a particular instruction has security or data integrity issues, just patch the compiler not to emit that instruction. As new side-channel speculation vulnerabilities are discovered, we can add compiler workarounds whenever possible. If you're a CPU architect and want to add a new instruction for a particular weird use-case, you just have to add it to your design and patch the compiler, and everyone can start using your new instruction right away, even on old software. You'd be able to trust that your old software would at least be compatible with future instruction architectures. Processors would be able to compete directly with each other without regard to vendor-lock-in.
That's how IBM implemented the AS/400 platform. Everything compiled down to a processor-agnostic bytecode that was the "binary" format. That IR was translated to native code for the underlying processor architecture as the final step. And objects contained both the IR and the native code. If you moved a binary to another host CPU, it would be retranslated and run automatically. The migration to POWER as the underlying processor was almost entirely transparent to the user and programming environment.
> That's how IBM implemented the AS/400 platform. Everything compiled down to a processor-agnostic bytecode that was the "binary" format
Originally, AS/400 used its own bytecode called MI (or TIMI or OMI). A descendant of the System/38's bytecode. That was compiled to CISC IMPI machine code, and then after the RISC transition to POWER instructions.
However, around the same time as the CISC-to-RISC transition, IBM introduced a new virtual execution environment – ILE (Integrated Language Environment). The original virtual execution environment was called OPM (Original Program Model). ILE came with a new bytecode, W-code aka NMI. While IBM publicly documented the original OPM bytecode, the new W-code bytecode is only available under NDA. OPM programs have their OMI bytecode translated internally to NMI which then in turn is translated to POWER instructions.
The interesting thing about this, is while OMI was originally invented for the System/38, W-code has a quite different heritage. W-code is actually the intermediate representation used by IBM's compilers (VisualAge, XL C, etc). It is fundamentally the same as what IBM compilers use on other platforms such as AIX or Linux, and already existed on AIX before it was ever used on OS/400. There are some OS/400-specific extensions, and it plays a quite more central architectural role in OS/400 than in AIX. But W-code is conceptually equivalent to LLVM IR/bitcode. So here we may see something in common with what Apple does with asking for LLVM bitcode uploads for the App Store.
> And objects contained both the IR and the native code. If you moved a binary to another host CPU, it would be retranslated and run automatically
Not always true. The object contains two sections – the MI bytecode and the native machine code. It is possible to remove the MI bytecode section (that's called removing "observability") leaving only the native machine code section. If you do that, you lose the ability to migrate the software to a new architecture, unless you recompile from source. I think, most people kept observability intact for in-house software, but it was commonly removed in software shipped by IBM and ISVs.
App uploads to the iOS store in the LLVM's own Bitcode format is a distant echo of the CPU ISA agnostic IR approach IBM employed at the time. Bitcode is transpiled down to the underlying CPU instructions via the static binary translation and optimisation, and the translation between Bitcode -> x86 or Bitcode -> ARM has been possible to do for some time: https://www.highcaffeinecontent.com/blog/20190518-Translatin...
Rosetta 2 AOT, whilst not being exactly the same thing as the ISA agnostic IR solution, is another example of the static binary translation. Theoretically, Apple could start requiring OS X app submissions to the app store in the Bitcode format as well, so they could be transpiled and optimised at the app download time and perform efficiently on M3, M4, M5 etc CPU's in the future. However, with their habit of obsoleting certain things fast, it is not clear whether they will choose to go down the Bitcode path for the OS X apps.
This is really fascinating. Is there a reason why we haven't seen more of this approach as it seems like it was pretty successful for IBM. Is there a practical reason that would prevent an open source project from building something similar?
> For instance, suppose binaries were typically distributed in a platform-agnostic format, like LLVM intermediate representation or something equivalent.
The Mill does something like this, but only for their own chips. "Binaries" are bitcode that's not specialized to any particular Mill CPU, and get run through the "specializer" which knows the real belt width and other properties to make a final CPU-specific version.
> I realize I've sort of just re-invented Javascript.
Or one of several bytecodes that get JIT or AOT compiled.
WASM in particular has my interest these days, thanks to native browser support and being relatively lean and more friendly towards "native" code, whereas JVM and CLR are fairly heavyweight, and their bytecodes assume you're going to be using a garbage collector (something that e.g. wasmtime manages to avoid.)
Non-web use cases of WASM in practice seem more focused on isolation, sandboxing, and security rather than architecture independence - stuff like "edge computing" - and I haven't read about anyone using it for AOT compilation. But perhaps it has some potential there too?
I think the short answer is that the performance penalty is so significant that it doesn't make sense to use WASM unless you're running untrusted code.
Ah yes, it even rates a mention on that Wikipedia page. Thank you for pointing this out. 1966! Funny how all these supposedly 'new' concepts really go back all the way to the beginning.
Come to Nix and Nixpkgs, where we can cross compile most things in myriad ways. I think the barriers to new hardware ISAs on the software side have never been lower.
Even if we get an ARM RISC-V monoculture, at least we are getting diverse co-processors again, which present the same portability challenges/opportunities in a different guise.
> For instance, suppose binaries were typically distributed in a platform-agnostic format, like LLVM intermediate representation or something equivalent. When you run your program the first time, it's compiled to native code for your architecture and cached for later use.
IBM's OS/400 (originally for the AS/400 hardware, now branded as System i) did precisely this: Compile COBOL or RPG to a high-level bytecode, which gets compiled to machine code on first run, and save the machine code to disk; thereafter, the machine code is just run, until the bytecode on disk is changed, whereupon it's replaced with newer machine code. IBM was able to transition its customers to a new CPU architecture just by having them move their bytecode (and, possibly, source code) from one machine to another that way.
Using ANDF you could produce portable binaries that would run on any UNIX system, regardless of CPU architecture. It was never commercially released though. I think while it is cool technology the market demand was never really there. For a software vendor, recompiling to support another UNIX isn't that hard; the real hard bit is all the compatibility testing to make sure the product actually works on the new UNIX. ANDF solved the easy part but did nothing about the hard bit. It possibly would even make things worse, because then customers might have just tried running some app on some other UNIX the vendor has never tested, and then complain when it only half worked.
Standards are always going to have implementation bugs, corner cases, ambiguities, undefined behaviour, feature gaps which force you to rely on proprietary extensions, etc. That's where the "hard bit" of portability comes from.
You've just reinvented bytecode. JVM/ART, WebAssembly, ActionScript, some versions of .net... You know, all the stuff that supposedly runs on everything.
Surely the barrier to using a new architecture is being able to boot a kernel and run (say) the GNU toolchain, as demonstrated with RISC-V. Then you just compile your code, assuming it doesn't contain assembler, or something. Whether or not you'll have the same sort of board support issues with RISC-V as with Arm, I don't know.
I don't know much about Android specifically. Is it still heavily Java-based?
There are a lot of universal-binary candidates both current and historical. Javascript, Java, SPirV, LLVM intermediate representation are some of the current ones. So, this isn't a new idea, it's just that most of the software I use regularly is compiled specifically for x86-64. Maybe it would be better if that were a rare exception rather than the norm.
> I wish the barriers to using new architectures were lower.
> For instance, suppose binaries were typically distributed in a platform-agnostic format, like LLVM intermediate representation or something equivalent.
We're doing a pretty good job on portability these days already. Well-written Unix applications in C/C++ will compile happily for any old ISA and run just the same. Safe high-level languages like JavaScript, Java, and Safe Rust are pretty much ISA-independent by definition, it's 'just' a matter of getting the compilers and runtimes ported across.
Adopting LLVM IR for portable distribution, probably isn't the way forward. I don't see that it adds much compared to compiling from source, and it's not what it's intended for. (LLVM may wish to change the representation in a subsequent major version, for instance.)
For programs which are architecture-sensitive by nature, such as certain parts of kernels, there are no shortcuts. Or, rather, I'm confident the major OSs already use all the practical shortcuts they can think up.
> When you run your program the first time, it's compiled to native code for your architecture and cached for later use.
Source-based package management systems already give us something a lot like this.
There are operating systems that take this approach, such as Inferno. [0] I like this HN comment on Inferno [1]: kernels are the wrong place for 'grand abstractions' of this sort.
> I realize I've sort of just re-invented Javascript
Don't be too harsh on yourself, JavaScript would be a terrible choice as a universal IR!
> [[ to the bulk of your second paragraph ]]
In the Free and Open Source world, we're already free to recompile the whole universe. The major distros do so as compiler technology improves.
> Processors would be able to compete directly with each other without regard to vendor-lock-in.
For most application-level code, we're already there. For example, your Java code will most likely run just as happily on one of Amazon's AArch64 instances as on an AMD64 machine. In the unlikely case you encounter a bug, well, that's pretty much always a risk, no matter which abstractions we use.
> "Adopting LLVM IR for portable distribution, probably isn't the way forward. I don't see that it adds much compared to compiling from source, and it's not what it's intended for. (LLVM may wish to change the representation in a subsequent major version, for instance.)"
Maybe, but PNaCl (unfortunately deprecated by Google) "defines a low-level stable portable intermediate representation (based on the IR used by the open-source LLVM compiler project) which is used as the wire format instead of x86 or ARM machine code"
https://www.chromium.org/nativeclient/pnacl/introduction-to-...
Sure. I'm not saying it's impossible to construct such an IR and get it to work, I'm saying I doubt it's the best way forward. See my other comment [0] where I mention Google Native Client.
It would be a poor fit for certain languages, there may be performance penalties depending on target platform, it would preclude legitimate platform-specific code such as SIMD assembly, it would preclude platform-specific build-time customization, etc.
The way toward painless portability is to move away from unsafe languages like C and C++, where you're never more than an expression away from undefined behaviour, and where programmers may be tempted to make silly mistakes like writing code sensitive to the endianness of the target architecture. [1] With C and C++, disciplined developers working carefully, can write portable code. With Safe Rust, code can be pretty close to 'portable by construction', like Java. If you feed Windows-style path strings to Linux, or vice versa, then things might go wrong, but for the most part you'll be on solid ground.
> Well-written Unix applications in C/C++ will compile happily for any old ISA and run just the same. Safe high-level languages like JavaScript, Java, and Safe Rust are pretty much ISA-independent by definition, it's 'just' a matter of getting the compilers and runtimes ported across.
That sort of works, but duplicating the proper development environment in the end-user's computer would take a lot of space and would be complicated by the enormous variety of programming languages and environments. Linux distros manage this with a lot of effort. I'm imagining something like a universal intermediate representation that can be compiled quickly (because a lot of the early language-specific part of compilation will have already been done by whoever you get your packages from) and in a uniform way because there's a common intermediate representation format that all the compiled languages use.
Universal binaries might also be acceptable for commercial, closed-source applications where source distribution would not.
> duplicating the proper development environment in the end-user's computer would take a lot of space
Most distros offer precompiled binaries, there are relatively few that use source-based distribution and expect the user to have all the necessary compilers installed.
> would be complicated by the enormous variety of programming languages and environments
That problem isn't effectively addressed by a universal IR. You can't have a single IR that works well for all languages, precisely because of the variety of languages.
> Linux distros manage this with a lot of effort.
Hopefully that should improve if the trend toward languages like Safe Rust continues. C and C++ are infamously full of footguns.
> I'm imagining something like a universal intermediate representation that can be compiled quickly (because a lot of the early language-specific part of compilation will have already been done by whoever you get your packages from) and in a uniform way because there's a common intermediate representation format that all the compiled languages use.
Again this can't be done effectively. There are good technical reasons why Java, Haskell, and JavaScript, don't generally use LLVM as their backend. The differences between languages aren't just skin deep, they extend right through the compiler stack.
To be more precise: it could be done, but there would be an unacceptable performance cost. After all, you could start distributing binaries for the SuperH SH-4, and just use emulation everywhere. The question is whether it could be done effectively.
I mentioned before that LLVM IR is not intended to be used this way, although the Google Native Client project took LLVM and turned it into what you're suggesting.
C and C++ are quite different from Java. The size of the int type varies between platforms, for instance. They also have a preprocessor which allows the programmer to conditionally compile platform-specific code, e.g. intrinsics, fragments of assembly code, or workarounds. The program might use system-specific macros that expand before compilation.
Languages like Haskell are very different from the sorts of languages that LLVM is built for. Even Java prefers to use its own backend, with tight integration with its GC.
There's also a package-management question, although this issue wouldn't be as significant. The C/C++ way is to have the build system (autotoools or CMake or whatever) detect what libraries are available on the system. If an optional library is missing, the C/C++ code is automatically adjusted by the build system, prior to compilation. It would be unusual to detect availability of libraries at runtime. This approach doesn't play nicely with a universal IR. This might not be an issue if the IR is treated as a surrogate for the native-code binary, but the IR wouldn't be a good surrogate for the source.
The C/C++ philosophy is to accommodate platform variations, in contrast with the JVM approach of mandating compliance to a virtual machine. With the JVM approach you forbid the sorts of variations that C and C++ permit (everything from int_fast32_t varying between platforms, to hand-written SIMD assembly).
Others have already mentioned WASM and Google Native Client, both of which are stable, but neither of which are going to become mainstream ways of distributing Unix application code.
This topic has turned up on HN before, but frustratingly I wasn't able to find the thread.
> Universal binaries might also be acceptable for commercial, closed-source applications where source distribution would not.
True, but I think modern Unix OSs do a pretty good job on ABI stability. If they want portability without releasing source (something I don't think GNU/Linux should aim to accommodate, incidentally) they already have other options, like Java. JavaFX doesn't get much attention but it pretty much 'just works' for portable GUI applications.
edit skissane has an interesting comment on ANDF, a solution I hadn't heard of before.
What RISC-V achieves is architectural diversity over the boring mov, add, mul instructions: the interesting part is in vector and matrix manipulation, and while RISC-V is working on a great solution, it allows for other accelerators to be added.
SPARC is well known to be different enough (big endian, register windowing of the stack, alignment, etc.) that it exposes a lot of bugs in code that would be missed in a little-endian, x86 derived monoculture.
I always found SPARC's stack handling to be very elegant and I write enough of low level code that these architectural details do from time to time impact me, but isn't it largely irrelevant for the industry at large?
After all MIPS's original insight was that machine code was now overwhelmingly written by compilers and not handwritten assembly, so they made an ISA for compilers. I think history proved them absolutely right, actually these days there are often a couple of layers between the code people write and the instructions fed into the CPU.
I guess my point is that nowadays I'm sure that many competent devs don't know what little-endian means and probably wouldn't have any idea of what "register windowing of the stack" is, and they're completely unaffected by these minute low level details.
Making it a bit easier for OpenBSD to find subtle bugs is certainly nice, but that seems like a rather weak argument for the vast amount of work required to support a distinct ISA in a kernel.
Honestly I'm not convinced by the argument for diversity here, as long as the ISA of choice is open source and not patent encumbered or anything like that. Preventing an x86 or ARM monoculture is worth it because you don't want to put all your eggs in Intel or Nvidia's basket, but if anybody is free to do whatever with the ISA I don't really see how that really prevents innovation. It's just a shared framework people can work with.
Who knows, maybe somebody will make a fork of RISC-V with register windows!
I had a neat experience a long time ago when I wrote a Perl XS module in C, in my x86 monoculture mindset. When you deploy something to their package manager (CPAN), it's automatically tested on a lot of different platforms via a loose network of people that volunteer their equipment to test stuff...https://cpantesters.org.
So, I immediately saw it had issues on a variety of different platforms, including an endianess problem. Cpantesters.org lets you drill down and see what went wrong in pretty good detail, so I was able to fix the problems pretty quickly.
It used to have a ton of different platforms like HPUX/PA-RISC, Sun/Sparc, IRIX/MIPS and so on, but the diversity is down pretty far now. Still lots of OS's, but few different CPUs.
MIPS and Berkeley RISC started an entire revolution. They appear "not unique" only because other ISAs copied them so thoroughly. I think it's safe to say that Alpha, ARM, POWER, PA-RISC, etc wouldn't have been designed as they were without MIPS.
Even today, comparing modern MIPS64 and ARM aarch64, I find ARM's new ISA to be perhaps more similar to MIPS than to ARMv7.
> What problem did MIPS solve in a unique way that others didn't?
The MIPS R2000 was debatably the first commercial RISC chip. It solved whatever problem you needed a really fast CPU for in 1985. The alternatives on the market were the Intel 386 and the Motorola 68000. The Intel 386 at 16 MHz did about 2 MIPS (heh - millions of instructions per second) with 32 bit integer math. At 16 MHz, the R2000 did about 10 MIPS. Even accounting for RISC code bloat, that's 3 - 4x faster.
Note how there were only two competitors selling 32-bit designs in the market they entered. I think that's probably the biggest impact of MIPS. They actually sold the chip! They wanted companies to design their own computer systems around it. Use it in an embedded device. Whatever. That was not the norm c. 1985 - 1988 for high-end silicon.
There were machines faster than the 386 or 68020 at that time. You could buy one of the fast microprocessor-based VAXes recently introduced. Or if not too squeezed for office space and with a blank cheque, one of the super-minis like a real VAX or IBM's new "mini-mainframe". After '86, maybe you'd buy one of the other RISC options, like SPARC or PA-RISC.
Whatever you bought, it would be the whole system. Take it or leave it. DEC would not sell you something like a CVAX processor all by itself just so you can build it into a product that will compete against them. (Well, they would sell you one, just not at a price you could afford if you aren't a defence contractor.)
Both DEC and SGI would use MIPS processors in their workstations of the late 80s, as did some less well-known names. The embarrassment of having to use a competitor's processor to sell a decently fast and affordable UNIX workstation would inspire DEC to create the Alpha. In this vein of "we'll sell it to whoever wants to buy it!" MIPS was also doing ARM-style core IP licensing, before ARM did. That's probably part of why MIPS was so prominent as an embedded architecture in the late 90s and early 2000s, in everything from handhelds to routers to satellites.
As far as I understand, it's not that MIPS is the best at embedded, it's just that it's cheaper to sell as the license cost is non-existing and good support already exists in kernels and so on.
If MIPS offered adequate performance and features, good performance-per-watt, and a competitive licence fee, and if none of its competitors could beat it, doesn't that count as 'best'?
> it's not that MIPS is the best at embedded, it's just that it's cheaper to sell
That sounds a lot like MIPS being the best at embedded. Not high-end, sure, but a lot of embedded is "what is the cheapest processor that can run Linux?"
ISA diversity can surface bugs in code, that only shows up in ISAs with different approaches. For example code that works on some arches but is slow due to alignment issues will just fail to run on other arches.
I remember CS 61C at Berkeley used to use MIPS to teach assembly language programming and a bit about computer architecture, using the original MIPS version of Patterson and Hennessy's Computer Organization and Design. Now that book is available in both MIPS and RISC-V versions, with, I've assumed, much more effort going into the RISC-V version...
I do think the simplicity of MIPS was a big plus there, including simplicity of simulating it (http://spimsimulator.sourceforge.net/). I suppose a lot of students may appreciate being taught on something that is or is about to be very widely used, even if it's more complicated in various ways -- and the fact that one of the textbook authors was a main RISC-V designer makes me assume that educational aspects are not at all neglected in the RISC-V world.
More on topic, though, RISC-V seems to really be designed in a way that makes it easy to teach. This is partially why I have doubts that it can be made very performant, but the focus of a prettier design over a more practical one is probably going to help it be more accessible to students.
> part of me thinks it's a shame that there's relatively little architectural diversity
Perhaps CPU diversity is in decline but it seems to me that the industry as a whole is moving towards more diversity. It's gotten significantly cheaper to roll your own chips to the point that we've seen entirely new processors emerging (e.g. GPUs, TPUs, etc) and becoming commonplace if not essential.
Isn't the point of RISC-V that the CPU is simple and augmented or complimented by any number of custom co-processors? If this is the general tend in the industry then the CPU itself might become a commodity part to be easily swapped out as something better emerges. Particularly if it can be abstracted away from the ISA.
Academic RISC was designed by Patterson and Hennessy. Hennessy went off and was one of the founders of MIPS, Patterson is one of the instrumental leaders in the RISC-V space.
Patterson and Hennessy were in competition with each other, at different universities. It's only much later they wrote text books together.
Patterson says RISC-V is derived from RISC-I and RISC-II. I think this doesn't really old water -- at least no more so than any other RISC.
RISC-I and RISC-II had condition codes and register windows, like SPARC. RISC-V doesn't have either, like MIPS. The RISC-V assembly language is also very similar to MIPS.
RISC-II had both 16 and 32 bit opcodes (as did IBM 801, and Cray designs) and RISC-V has inherited this (but also following the great success of ARM Thumb2).
Unfortunately it isn't gaining any traction. From a Long term Cost perspective it is actually cheaper choosing ARM even if OpenPOWER is free. And ARM is already inexpensive.
MIPSr6, aarch64 and riscv are siblings born from MIPSr5 plus the best of other RISC architectures.
ppc64le and 32-bit Arm are part of the wider family, but have some notable differences that make them slightly less RISC-like. Both include e.g. more complex condition code handling and instructions that operate on more than three registers.
Since you mentioned ppc64le, there's also aarch64eb (arm64 in big endian mode). I saw that NetBSD supports it. It seems like support for other operating systems is limited mostly because of issues around booting...not the actual kernel or userland itself.
There's a lot of diversity in software. Windows is quite different from GNU/Linux, for instance. There's quite a lot of diversity in the major programming languages.
If you want something really different, whether in operating systems or programming languages, you have it. KolibriOS and Haskell, say.
At least we'll always have the PS2. ...until the last one breaks, I guess.
[0] https://en.wikipedia.org/wiki/SPARC