usage
List In Flight
Requests the gateway is currently serving, longest-running first.
A usage row is written when a request settles, so the log alone cannot answer "is anything happening right now": on a slow backend, a 30-second local model call is invisible until it finishes. This reports what is in progress.
Read from an in-memory registry, so it describes the process that answers this call and not the deployment: behind a load balancer, consecutive polls reach different otari processes, and there is no deployment-wide total to ask for. total is the true in-flight count for the answering process even when requests is capped.
get/v1/usage/in-flight
Response
Successful Response