Adding “20 requests per minute” to an endpoint is easy. Deciding what should be counted, across which servers, and what the user experiences requires product judgment.
An AI streaming request may cost money and hold resources for seconds. A cached catalog read has a very different risk. They should not inherit one global number.
Choose the identity#
Authenticated endpoints usually limit by user. Public endpoints may use IP, accepting that shared networks and proxies make it imperfect. Some operations need both a per-user limit and a global capacity limit.
Never trust a caller-provided user header as the key. Use verified identity from authentication context.
Choose storage for the deployment#
In-memory counters are convenient for local development and a single process. With multiple workers or replicas, every process sees a different count. Redis or another shared atomic store provides a distributed limit.
If shared storage is unavailable, decide whether the endpoint fails open or closed. A login or expensive AI endpoint may make different choices.
Express the result honestly#
Use 429 when this caller exceeded its policy and include retry guidance such as Retry-After. Use 503 when the service as a whole cannot accept more work. Clients may present and retry these situations differently.
Return limits through the standard error contract:
{
"code": "rate_limited",
"message": "Please wait before trying again.",
"retryable": true
}
Layer limits with business quotas#
A requests-per-minute limiter protects traffic shape. A daily free-tier AI quota enforces product entitlement. Keep them separate so the user receives the right explanation and recovery path.
Monitor allowed, rejected, and storage-error outcomes using bounded dimensions. Test boundary timing, multiple identities, multiple app instances, and unavailable Redis.
Rate limiting is part capacity control, abuse prevention, cost protection, and product experience. The middleware implements the counter; the product defines the policy.