RedditPP Mascot

RedditPP

r/googlecloud • … • u/null_reference_user

Cloud run dropping requests for no apparent reason

Hello!

We have a Cloud Run service that runs containers for our backend instances. Our revisions are configured with a minimum scaling of 1, so there's always at least one instance ready to serve incoming requests.

For the past few days we've had events where a few requests are suddenly dropped because "there was no available instance". In one of these cases there were actually no instances running, which is clearly wrong given that the minimum scaling is set to 1, while in the other cases there was at least one instance and it was serving request perfectly fine, but then a few requests get dropped, a new instance is started and spun up while the existing is still correctly serving other requests!

The resource usage utilization graphs are all well below limits and there are no errors apart from the cloud run "no instances" HTTP 500 ones, we are clueless as to why this is happening.

Any help or tips is greatly appreciated!

— 0