Billing support
Rate Limits and Quotas
Understand request, concurrency, and token limits and handle 429 with queues and backoff.
Limits protect platform and account stability. Values can vary by account, model, and service plan; use the limits displayed in the console.
Common limit dimensions
| Dimension | Meaning | Client response |
|---|---|---|
| Request rate | Requests allowed per time window. | Token buckets, leaky buckets, or queue shaping. |
| Concurrency | Simultaneous in-flight requests. | Semaphores and separate pools for long and short work. |
| Token rate | Input/output tokens processed per time window. | Trim context, cap output, and spread work over time. |
| Balance / budget | Available account spending or internal budget. | Alert before exhaustion interrupts service. |
Handling 429
- Stop immediate retries and read any available limit information in the response.
- Use exponential backoff, for example roughly 1, 2, then 4 seconds, with random jitter.
- Bound retry attempts and total waiting time.
- Reduce global concurrency or use a bounded queue rather than letting every task retry independently.
- For persistent limits, analyze peak traffic and request an appropriate quota.
Rate-limit response contract
Header names and values require platform confirmation. Published guidance must specify whether RPM, TPM, and concurrency apply to accounts, projects, API keys, or models and provide readable response headers.
| Information | Documentation requirement |
|---|---|
| Limits and remaining capacity | Header names for request, token, and concurrency limits and remaining quota. |
| Reset time | Window algorithm, time zone, precision, and whether reset values are seconds or timestamps. |
Retry-After | Whether it is returned and whether to parse seconds or an HTTP date. |
| 429 error object | Stable error types distinguishing request, token, concurrency, account-quota, and platform-capacity limits. |
| Quota increases | Required business data, review time, temporary capacity, and service-level options. |
Quota planning
- Estimate concurrency from average request duration, not RPM alone.
- Reserve capacity for releases, campaigns, and batch workloads.
- Separate online interaction and offline work into different queues and priorities.
- Apply per-tenant limits to prevent noisy neighbors.
- Provide peak concurrency, token distributions, target models, and business time windows when requesting capacity.