Correct me if I'm wrong, but I get the impression Kubernetes is a somewhat expensive solution. The only reason I'd go with Kubernetes is if I was a) on Google cloud, which has first class support for Kubernetes b) a dedicated ops team deploying on premise.
I run it as a 1 man dev team, spend maybe a few minutes per month on it. Save about 2x over running the same workload on bare VMs, and maybe 4x over the same workload on lambda.
But even on AWS.. all google is buying me is 1 "master" node for free (both running the node cost, and setting up the master cost). You can spin up a single reserved instance as your master node in about a day on AWS, and be good to go. So my "savings" by going google is like $50 and the setup of 1 master node. Outside of the master node, the experience is the same.
Bin packing is really nice. My app has something like 30 containers running. Using VMs, that would be 30 instances to fire up. In kubernetes, I fit it on 5 servers. Yes those 5 servers are a bit beefier, but it is still a pretty substantial savings.
