Because security is not a priority for the industry. Most have no security, default authentication in the rare case that they have it, and they use protocols with no support for it. The field is decades behind in security practices (it's pretty much IoT) and won't improve unless forced to.
It's also difficult to update such devices in the field so even if they do fix such issues it's only for new units or a new product line which most customers won't bother with until forced to by regulations / incidents as it's expensive to replace them (you have to send someone out on the field as there are pretty much no OTA updates).
The field is decades behind best practice because these systems have multi-decade operational lives.
There's an absolute chasm between implementation intervals that can be achieved through pure software systems and those with distributed hardware components. Throw in a few layers of abstraction where those designing, purchasing, installing, operating, and maintaining those components are all unrelated parties with different (and potentially conflicting) motives and any sort of cohesive systems engineering is hard.
This doesn't excuse continued irresponsibilities in product security, because they absolute exist, but "impressively fragile yet surprisingly functional" is a completely logical Nash equilibrium to settle on given the surrounding non-technical components.
> The field is decades behind best practice because these systems have multi-decade operational lives.
This would be more convincing if not for the fact that smart meters are IIoT. They're a new thing. IIoT is kind of an unholy breed between those hardcore industrial engineers you talk about, designing hardware with multi-decade operational lives, and the people implementing the IoT part using webdev practices, trying to put Docker containers full of NPM modules onto the industrial devices (and if they can't fit there, then plugging them immediately upstream).
Now that latter group is (mis)using bleeding edge tools to develop greenfield solutions - and thus should very much be able to keep up with basic security practices developed in the last 20 years.
But we are not talking about them using too weak RSA keys from 2 decades ago, or even not about transmitting passwords unencrypted, so anyone with a right radio could glean that.
We are talking about a complete lack of any access control. Like two wires instead of an ignition lock. An electric box with a mechanical meter and switches would at least have a padlock on it.
Neither is long term functioning of the electric grid if you read the IEEE. Go read the IEEE journal where every few years someone writes an article warning that the electric grid will fail catastrophically when an 1859 level solar flare occurs that we can prevent with a relatively straightforward fix.
Technical debt exists in disciplines other than software development.
It's also difficult to update such devices in the field so even if they do fix such issues it's only for new units or a new product line which most customers won't bother with until forced to by regulations / incidents as it's expensive to replace them (you have to send someone out on the field as there are pretty much no OTA updates).