We recently built our own sandbox environment backed by firecracker and go. It works great.
For data residency, i.e. making sure the service is EU bound, there is basically no other way. We can move the service anywhere we can get hardware virtualisation.
As for the situation with credentials, our method is to generate CLIs on the fly and expose them to the LLMs and then they can shell script them whichever way they want. The CLIs only contain scoped credentials to our API which handles oauth and other forms of authentication transparently. The agent does not need to know anything about this. All they know is that they can do
$ some-skillset search-gmail-messages -q "emails from Adrian"
In our own experiments we find that this approach works better and it just makes sense given most of the latest models are trained as coding assistants. They just love bash, so give them the tools.
We recently built our own sandbox environment backed by firecracker and go. It works great.
For data residency, i.e. making sure the service is EU bound, there is basically no other way. We can move the service anywhere we can get hardware virtualisation.
As for the situation with credentials, our method is to generate CLIs on the fly and expose them to the LLMs and then they can shell script them whichever way they want. The CLIs only contain scoped credentials to our API which handles oauth and other forms of authentication transparently. The agent does not need to know anything about this. All they know is that they can do
$ some-skillset search-gmail-messages -q "emails from Adrian"
In our own experiments we find that this approach works better and it just makes sense given most of the latest models are trained as coding assistants. They just love bash, so give them the tools.