I generally agree with you - I don't think that you can survive well on just plants. At the same time I think that modern people eat way more meat than people used to eat in any other period of history, and probably much more than is necessary.
For example, the way I understand it, how people used to eat where I'm from (Greece) in the past is that they fed themselves very well on pulses, legumes, vegetables, fruit, plenty of fish, metric shittons of olive oil, and then only ate red meat in religious feasts, where a lamb or an ox would be sacrificed in ancient times, or just roasted on a spit and eaten whole, offal and all, in more modern times.
To be sure, I wasn't saying it was impossible to survive well on just plants (though it's certainly a bit more challenging and risky). It's just that naive comparisons of food efficiency don't tell the whole story. You need a lot of trace vitamins which you can only obtain by 1) taking a lot of a low-density source and concentrating it (like cooking an entire bag of spinach into a souffle to get iron), or 2) eating something which has done the concentrating for you, usually over a period of time. Either way, living off plants is considerably less efficient than "meat vs wheat" would suggest.
"Plenty of fish" is a problematic addition to the diet you describe - that's meat, plain and simple, with all of the bioefficiency loss and ecosystem disruption that entails. It's just less visible because it's underwater.
For example, the way I understand it, how people used to eat where I'm from (Greece) in the past is that they fed themselves very well on pulses, legumes, vegetables, fruit, plenty of fish, metric shittons of olive oil, and then only ate red meat in religious feasts, where a lamb or an ox would be sacrificed in ancient times, or just roasted on a spit and eaten whole, offal and all, in more modern times.