the elixir community is quite credulous. particularly when it comes to circa 200X era wisdom. people talk about the vm being the key to elixir/erlang and talk up things like lightweight green threads, message passing and the garbage collector but the truth is these are all of fairly low quality compared to other competing languages/implementations
the real key for erlang and later elixir's success was pervasive async io. this was a genuine advantage during erlang's peak but languages, runtimes and libraries like go, node and nio have caught up and surpassed the erlang vm
the truth is without that advantage almost everything in the erlang/elixir world is worse than more mainstream alternatives. there's some exceptions -- ecto is pretty good because it has one of the best written connection pooling implementations i've ever seen. i think both languages are pretty good and i still write the occasional thing in erlang for my own satisfaction but the world has moved on and elixir and especially erlang haven't innovated enough to have kept up
almost every language has lightweight cooperative threading (or green threads) available these days. go calls them goroutines, c# and ruby fibres (altho i think ruby removed them, ultimately?), python has threadless, rust has tokio, julia has tasks, the jvm has like 4 competing implementations in akka, kiilim, quasar and project loom. windows and linux both have built in os level support for cooperative multitasking (the fibre api and the context api, respectively) that any language can use
these all have slightly different semantics and characteristics, but it's simply not true that beam is doing anything unique here. pervasive links between processes are a somewhat interesting wrinkle of the beam implementation but even that is achievable in other languages with little work
what made green threads work in erlang (and later elixir) was async i/o. in other languages green threads would block on all i/o calls and had no opportunity to yield whereas on the erlang vm all i/o calls would effectively yield while waiting. today nearly every language has async i/o (in libraries if not pervasively) so green threads are much more accessible
i don't say this as an elixir hater or whatever. i genuinely respect erlang's place in history as a populizer of some of these concepts. when i say the elixir/erlang community is credulous and prone to exaggerating the differences between those languages and other more modern languages (particularly when it comes to implementation) i don't say it dismissively as a reason to abandon those languages. i say it because without an impetuous to keep improving elixir and erlang are going to become increasingly irrelevant. it would be a shame if the 'elixir is different and unique' attitude led to complacency and stagnation
I am not going to engage you on the details that you got wrong (Akka not covering 100% of OTP's guarantees is one example) but I'll just point out something else on a higher, more pragmatic level.
Of all 7-8 programming languages I worked actively with for my almost 20 years of career, only Elixir apps had predictable and stable latency even under load. So you know, at one point I stopped caring how does the BEAM do it or why aren't the other languages/runtimes doing it. I just started going to the technology that gives me this.
Is Go faster? Feck, absolutely yes! But its 95 percentile latency is spiking through the roof under load while a Phoenix/Ecto raises its median latency by no more than 20-30% (worst I've seen is +300% when from median latency 15ms the app went to 60ms and that's only in the 99th percentile of requests) even when the hardware is close to toppling over.
I feel that the raw muscle power of languages is vastly over-represented. I'd love to have a Rust with OTP's guarantees because at some places it's literally 1000x faster than Elixir, absolutely. But in a world where we have to choose raw power versus predictable performance (even if that performance is lower than what we can get in other languages) then I'll choose the latter any day.
And I am not alone in this. Many teams are choosing Elixir for exactly this reason.
---
One thing I'll agree with is that other languages have taken notice and are working hard to catch up with the OTP. I'd welcome them in the club once they are there because I hate language wars and I gauge technologies based on their merit. But they are still not there, sadly.
i'm not trying to convince you (or anyone, really) to stop using elixir. i am however encouraging you to engage deeper with the beam vm and it's actual properties and compare that honestly to what else is out there. what is it exactly about the beam vm and elixir that lets it achieve these latencies and why is this not achievable in other languages? simply saying the beam vm is better than other implementations isn't an answer to that question
Don't get me wrong, I'd LOVE doing that but my work time and employer priorities doesn't allow it yet (and might never). And I am starting to get really sick of extending my work time to my free time as well.
> c# and ruby fibres (altho i think ruby removed them, ultimately?
Ruby did not remove Fibers (in fact, they’ve been recently, in 3
0, enhanced to optionally be nonblocking—that is, automatically yielding on any operation that wpuld block.)
Ruby removed continuations from the core (moved them to stdlib) after adopting independent Fibers way back with 1.9; continuations were the previous mechanism for similar lightweight concurrency.
What about functions as a service, like AWS lambda?
If you use Node as an example, your code is JIT compiled to machine code, any single request can fail, and you can scale to any number of requests without thinking about the underlying OS or VM.
Async/await will allow you to do a “blocking receive” like Erlangs processes.
Nah, BEAM is not suited well for that. It has a big startup time. BEAM is designed for long-running daemons, not "wake up, do small amount of work, get shut down".
For something like this I'd choose Rust or OCaml due to their insanely fast cold startup time (if the program is a CLI tool).
Erlang/BEAM is not there yet and it might not soon be.
OP said he did not know any other "BEAM like environments", but AWS and GCP are "BEAM-like" systems in that they allow you to use distributed hardware computers to achieve scale and fault tolerance.
This sounds true on the surface and many people have argued that e.g. Kubernetes is "OTP but for distributed nodes" but I remain skeptical. The devil is always in the details and I haven't heard many people being very pleased with Kubernetes.
Admittedly Google's Cloud Run is very easy and nice to use though. And fairly cheap.
First of all, the other programming environment like it is Pony. Although that's barely a C-list language right now, very young, one to keep an eye on though.
I'd also cite the real technology I'd use today, which is a heterogenous set of services in whatever languages I'd like, hooked up by a high-quality message bus. This is the real technology that drives the at-scale Internet. You basically get Erlang's reliability out of that setup when used properly, and you don't need Erlang to do it. In fact you can get a touch more than Erlang's reliability because I find in practice 1-or-n delivery to be much more practical than Erlang's 0-or-1 delivery. It's basically the same enviromnent Erlang gives you integrated, except decoupled, and since all the pieces are decoupled, while Erlang has sat on the same effective point in this space that it picked out 25 years ago, all the decoupled components have been iterating and evolving over that time frame and are now better than what Erlang offers in its integrated package.
Second of all, if in 2021, around 25 years after Erlang and easily 15 years after Erlang has been generally known as a B-list language among language designers, almost nobody else has seen fit to copy it... maybe it isn't that great of an idea. Rust does something completely different, and in my opinion, strictly more useful, albeit at the cost to programmer complexity. I moved to Go from Erlang roughly 8 years ago, and I'm happier, because it turns out "general community good practices + channels" is fine, and also means I can go faster, and get a nicer language in the meantime. All the other modern languages are coming with some sort of concurrency story; it's table stakes for any language born in the last 10 years, if not the last 15.
For the mid 1990s, it was sheer genius. For 2021, it's a very brute-force, inelegant solution to the problem that nobody's very interested in copying. While in the 1990s concurrency was a nightmare and Erlang legitimately had a claim to a better solution, in 2021 there's a good 3 or 4 things I'd use before dropping back to Erlang as a solution. Concurrency is much less of a problem than it used to be, through a combination of various things, and the proposition of burning so much of a languages design budget on that problem is a lot less appealing than it was 30 years ago. Erlang really needs to adopt Go-like channels for some of what it's doing (not in replacement for the processes, but for some of the things they're not very good at), the ~10x slowdown for general logic is a real kick in the teeth in 2021, the lack of backpressure in the Erlang message model becomes a big problem at scale, and lots of other little problems I'd have if I had to go back to it. (Yes, I've been reading the release notes. If I weren't I'd have a couple more things to add.)
Erlang/Elixir/BEAM isn't leading the pack anymore. They're a cut behind in most ways now, but the community still thinks they are leading, ensuring that none of the lessons learned by other communities can filter back into the Erlang/Elixir/BEAM community.
Even if I disagreed with you on another comment I'll have to say that I find myself much more in agreement with you here.
Erlang / the BEAM did indeed make a lot of good innovations and I can only be angry at myself for being an idiot pressured by employers and never looking beyond it all for something better (until 5+ years ago anyway). But I agree that some of it is starting to show cracks.
In terms of language design, Erlang (and Elixir) aren't anything special. I can't fall in love with syntax anymore because I've literally never seen a language I completely like (LISP included, although it + OCaml are fairly close to ideal languages if you don't stray too much off of the beaten path and venture into their more arcane constructs, of which OCaml sadly has plenty).
To clarify, I believe Elixir is one of the most solid contenders for writing highly available and reasonably performant Web / GraphQL server apps but the lack of compiler apparatus tooling, tooling to modify AST and a few others are definitely starting to hurt it. Having standardized introspection in the language helps it reach higher levels, e.g. have tools that can manipulate an existing project a la like TreeSitter and/or SemGrep can modify/query language-specific constructs. Elixir doesn't have that and I am starting to get annoyed with it because of that.
RE: Using an external messaging bus makes sense but let me point out something important that seem to be often not said in discussions about Erlang / Elixir:
The BEAM gives you a lot of good training wheels and the truth is that at least 90% (if not 98%) of the commercial projects out there don't require much more than that. As shared in the other comment, I was able to get away with not using Redis for a long time and had zero trouble. I only yielded after we needed to share various message queues and events/streams with other apps (not written in a BEAM language).
So I'd say the BEAM ecosystem gives you a lot out of the box, plus the Elixir community is small but fairly dedicated and they have libraries of excellent quality. But, as you alluded to, when you need to throw those training wheels off, other much more dedicated and focused technologies like Redis do exist and we should reach for them after the circumstances change enough.
the real key for erlang and later elixir's success was pervasive async io. this was a genuine advantage during erlang's peak but languages, runtimes and libraries like go, node and nio have caught up and surpassed the erlang vm
the truth is without that advantage almost everything in the erlang/elixir world is worse than more mainstream alternatives. there's some exceptions -- ecto is pretty good because it has one of the best written connection pooling implementations i've ever seen. i think both languages are pretty good and i still write the occasional thing in erlang for my own satisfaction but the world has moved on and elixir and especially erlang haven't innovated enough to have kept up