They're doing over 100rps if they're doing 10M requests a day. That's not a good use case for Lambda. If you're going to be that heavily utilized it makes more sense to run your API on EC2 or ECS/Fargate/etc.
Lambda is a good use case for when you have lots of not-often-used APIs. Lambda is a great solution for an API that's called a few times a day. It's also great for when you're first starting out and don't know when or where you'll need to scale.
But once you get to 100rps for an API, it's time to move to a more static setup with autoscaling.
I've always found Troy Hunt's tech stack of haveibeenpwned.com interesting. The API does 5M requests a day with Azure Functions and Cloudflare caching. Ultimately only costing him 2.6c per day.
I think the problem is that moving from Lambda/FaaS to a container-centric alternative (ECS and friends) requires a complete re-architect of your stack. Whereas starting with a simple, single container solution and organically evolving that into container-orchestration is much simpler - because the fundamental building block hasn't changed. It's all just containers.
Personally I'd like to see the industry coalesce on "serverless" containers rather than FaaS which are organised around functions being the fundamental blocks. Please just run my Docker container(s) like a magic block box that is always available, scales as necessary and dies when no longer needed.
Aren't there abstractions to reduce the impedance mismatch between serverless offerings, e.g. Serverless (javascript framework)[1], which should allow easier portability to self-hosted solutions - openfaas or openwhisk etc - including running in containers on more traditional infrastructure, which is cheaper for this particular scale and use-case?
Sure, they're still FaaS which seems to be the unit of deployment for the serverless movement. For (hidden) managed server container deployment, Fargate is the offered solution I believe.
Not necessarily. The reason I decided to try this was exactly because I found a tutorial showing you could easily host a bog-standard ASP.NET web app on Lambda with the serverless framework. I had to add a small startup class, and one config file to our existing app and I was up and running.
Recently we had to test a posibility of our core application being hosted in contenerized environment. It was really hard, because some tweaks we used are not really documented well. Kernel Tweaks had to be tested separately because we were not sure whether they work or not, recource management had to be done from zero level. JVM optimalizarion from zero level. Service Discovery configuration from zero level.
---
Sure it works from the go for punny small apps. But when it comes to huge corporations running extrordinary workloads - no its not that easy.
I think the problem is that moving from Lambda/FaaS to a container-centric alternative (ECS and friends) requires a complete re-architect of your stack.
Not really. I converted a Node/Express API running in lambda using the proxy integration to a Docker/Fargate implementation in less than a couple of hours by following a Fargate tutorial. Most of that time was spent learning enough Docker to do it.
The only difference between the Docker implementation of the API and the lambda implementation was calling a different startup module.
There is nothing magical about any lambda, from the programming standpoint you just add one function that accepts a JSON request and a lambda context.
Converting it to a standalone service (outside of APIs) is usual a matter of wrapping your lambda in something that runs all of the time and routing whatever event you’re using to trigger to a queue.
Agree with you, Lambda would make a lot more sense if it was kind of a scaffold for your application.
Say you only have 1k users and don't want to spend too much time on infrastructure, lambda is a perfect fit for it. Your application is taking off and now you have 100k: just click a button and migrate it to a Fargate/ECS. That would be the perfect world.
AFAIK the only framework that supports this kind of mindset is Zappa (https://www.zappa.io). I use it in production but never had to migrate out of AWS Lambda so I'm not sure about the pain points related to it.
Is there any fundamental reason for this, apart from AWS' pricing model? It seems to me that ideally, serverless should scale from extremely small to very big without too many problems.
You're basically hiring an AWS devops position since you don't need to manage anything yourself. Great for the small startup but not so great for the already established enterprise that has some devops guys anyway.
An AWS DevOps should be able to manage a lot more than one company's lambdas. That's one of the big reasons the cloud can save money. I don't see why it would be different for serverless setups.
"It's also great for when you're first starting out and don't know when or where you'll need to scale."
To me this is probably the most significant benefit, and one that many folks in this discussion strangely seem to be ignoring.
If you launch a startup and it has some success, it's likely you'll run into scaling problems. This is a big, stressful distraction and a serious threat to your customers' confidence when reliability and uptime suffer. Avoiding all that so you can focus on your product and your business is worth paying a premium for.
Infrastructure costs aren't going to bankrupt you as a startup, but infrastructure that keeps falling over, requires constant fiddling, slows you down, and stresses you out just when you're starting to claw your way to early traction very well might.
I thought the point of Lambda wasn't for not so often used APIs but for APIs where you need instant autoscaling where you may need 100 servers in 2 minutes and only need them for 20 minutes.
Lambda is a good use case for when you have lots of not-often-used APIs. Lambda is a great solution for an API that's called a few times a day. It's also great for when you're first starting out and don't know when or where you'll need to scale.
But once you get to 100rps for an API, it's time to move to a more static setup with autoscaling.