> Are you saying that all of them are actually losing money?
Are you saying these companies are operating on anything resembling a sustainable business model?
I think not!
If OpenAI needs to impose 5 hour limits during a time where they are aggressively trying to grow market share, what do you think they (or whatever the winning provider is) will do in 5 years once they IPO and need to boost margins?
They won’t be as friendly with usage caps as they’re being now.
`needs to impose 5 hour limits ` I'm no OpenAI defender but that's like saying Apple is going bankrupt because they charge for their products. Why wouldn't they do everything they can to turn $20 users into $200 users?
Exactly. It is common knowledge that all the AI companies have terrible margins. IMO tokens are the cheapest they'll ever be as these companies compete for users.
To the extent that the 3rd party inference resellers are a viable market because of Chinese/certain US AI labs (current) willingness to absorb substantial compute/R&D/data harvesting costs while releasing weights on permissive licensing terms for free, then yes, current pricing is distorted.
Openrouter is not so open as well, right? They take the middle man cut by enforcing their own limit restrictions and 429s on top of the model provided limit restrictions and what not.
Now why would I pay a middle man anything if I can get the same hosted model without their own whimsical service limits?
I mean, they don’t train on your data unless you have the setting enabled.
Do you really think they are reading your prompts at all?
Free inference providers sure, but Anthropic?
But has this been thoroughly documented and are there solid libraries to achieve this?
My understanding is that this is not part of the spec and that the only way to achieve this is to sign/hash documents on clients and server to check for correctness
Well, it seems that the Apollo way of doing it now, via their paid GraphOS, is backwards of what I learned 8 years ago (there is always more than one way to do things in CS).
At build time, the server generates a random string resolver names that map onto queries, 1-1, fixed, because we know exactly what we need when we are shipping to production.
Clients can only call those random strings with some parameters, the graph is now locked down and the production server only responds to the random string resolver names
I mean yeah, in that Persisted Queries are absolutely documented and expected in production on the Relay side, and you’re a hop skip and jump away from disallowing arbitrary queries at that point if you want to
Though you still don’t need to and shouldn’t. Better to use the well defined tools to gate max depth/complexity.
I feel this would go down pretty bad considering the recent attempts to break E2E encryption on messaging.
Also a very tempting vector for hackers and governments to track user’s behaviour
You understand the complexities and risks involved in maintaining the setup you’ve described?
It’d be unreasonable for most folks who just want to sell regular products and deal with marketing and sales.
Those become the biggest tasks once they start getting traction (ask any relatively successful indie hacker).
Paying the processor’s fee is worth it for most
reply