Was about to say that we managed upwards of 4k servers worth of MySQL databases (as in 4k baremetal servers worth, not 4k small VMs each having a small MySQL) using "Orchestrator" at Booking.com ten years ago. I checked if it was still kicking before writing this and found that the GitHub repo was archived last year. The next Google hit I find is this article from three days ago:
Please do some additional research into the state of maintenance of that piece of tech before jumping on it. But it certainly did a lot of powerful things for us back in the day. The automatic promotion of followers was key to our deployment.
That, and patching MySQL in weird ways. I recall something about the filesystem layer being monkeyed with, because "who needs safe writes, it slows things down"
They make it easier, but just at a source code level. They're not a real (and certainly not full) abstraction. An example that'll be making it obvious: if you replace the underlying type with a floating point type, the semantics would change dramatically, fully visible to the user code.
With larger types that otherwise have similar semantics, you can still have breakage. A straightforward one would be padding in structs. Another one is that a lot of use cases convert pointers to integers and back, so if you change the underlying representation, that's guaranteed to break. Whether that's a good or not is another question, but it's certainly not uncommon.
(Edit: sibling comments make the same point much more succinctly: ABI compatibility!)
It's actually quite likely something else (unless it's just an excuse to reap short term savings): in a company large enough, deciding to shift staffing from one area to another is hard. As an executive with thousands of staff, you can tell your management team to each cough up a certain number or (somewhat) suitably qualified people. But again, if large enough, incentives diverge, so you don't necessarily end up with the top talent you thought you needed for your big new thing.
An "easy" solution is to do a layoff, then open roles elsewhere, allowing for selection.
It's commonly practiced across the large companies in the industry.
Former Perl language contributor here. A sibling comment to this already pointed out that you must use strict mode with Perl to retain your well being.
The two languages certainly both have their terrible warts. I think in the implicit conversion gotchas, JS is actually markedly worse. Perl has polymorphic values, but somewhat typed (for its basic types) operators (eg "eq" for strings, "==" for integers). JS has both implicit value type conversions and overloaded operators. That leads to an unholy level of indeterministic mess.
Many moons ago, I made a case for making strict mode the default in Perl. We settled on the current backwards compatibility compromise, which is that breaking changes are hidden behind a minimum version toggle:
Eg. putting "use v5.14.0;" or similar on top of your file (or compilation unit/scope) will indeed turn on strict mode for you, along with adding a number of features as well.
At the time, also auto-toggling warnings was considered unacceptable because technically, using the warnings pragma anywhere had some edge case action at a distance. This has been remedied in some later release after I wasn't involved in the language development anymore, and from some more recent version, warnings are also part of the standard import.
I imagine you (TheDauthi) already know that, though.
It's not an idle use case by the way. mhx wrote and maintains a library that provides important backwards compatibility for native (typically C based) extensions for Perl across decades of language releases.
See my response to a sibling of the comment you're responding to. The library had shocking code quality issues. It's unlikely that they're all peachy now.
The other side of this is that while Pieter's writing was marketing genius, it was also woefully understating the complexity of any practical use case. The way I tried to summarize that to folks who were keen to try zeromq then was that they should start at the back of the book with the most complex example, and that's by far the simplest setup that they could hope to end up with once they start thinking about putting something into production. And everything leading up to that - a book no less - was exclusively educational/toy use cases.
Zeromq will have changed a lot since then, but some time in the 2010s, I prototyped a system using it (which was going to be a major production system in a large tech company) and had weird unexpected blocking issues with it. To debug, I sat down to read a bunch of the zeromq code, just to realize that it was using assert() to handle wire protocol errors (unrelated to the blocking bug).
I've never dropped a piece of software as quickly as that.
More or less my experience as well. Asserting on bad user configuration, asserting on OS errors that weren't in a particular list. I followed their recommendation of having a "small, simple, reliable" broker and it kept crashing on asserts in the library at the worst times.
This is Google + Thales doing the 3rd party operator model, with the operator being a subsidiary of Thales and not Google.
(NB: I work for Google in the EU.)