Even with billing, GAE expects gradual ramp-up of traffic, as would happen in the real world. Your app will fail to scale if you hit it with an artificial blast of benchmarked requests because the back-end wouldn't know to fire up more instances.
In our extra tests we get 15k real user hits (get an image on a popular web-site front page, 3req/sec average), and then added 2 servers with load generators to get to 50-80 requests per second. We haven't noticed when it auto-scales (that was "proper" sharded counter application). Probably it never scales on non-billed account (but we were far from our limits).
