14.10 API Design, Versioning & Rate Limiting

1. What It Is in Plain Words

API design defines the conventions, resource hierarchies, and HTTP methods used to expose server capabilities cleanly to clients; rate limiting protects services from denial-of-service abuse by enforcing quotas on client request frequency.

---

2. How THIS Project Uses It

• API Design: Follows pragmatic REST conventions in [app.py](file:///Users/avishjain/avishjain9-repos/21-bis-compliance-engine/app.py):
  • GET /: Idempotent liveness status check.
  • POST /chat: Primary resource processing endpoint accepting JSON and returning audit determinations.
  • POST /audit/export: Action endpoint generating and streaming binary/text CSV attachments.
• API Versioning: Currently Unversioned (endpoints mounted at root /chat rather than /v1/chat). This is acceptable for an early-stage specialized microservice [CONFIRM WITH DEVELOPER].
• Rate Limiting: Not Implemented in Code. The service relies on network infrastructure boundaries for traffic control.

---

3. Comparison of Rate Limiting Algorithms

Algorithm	Mechanism	Burst Handling	Memory Cost	Accuracy	Fit for This Engine
Token Bucket	Tokens refill at fixed rate; spent on request	Allows configurable bursts	Very Low (2 integers per IP)	High	Optimal for AI inference
Leaky Bucket	Requests leak out of queue at fixed rate	Smooths bursts into steady stream	Moderate	High	Good for steady pacing
Fixed Window	Counts requests in fixed minute window	Allows 2x burst at window boundary	Very Low	Low	Prone to boundary spikes
Sliding Window Log	Timestamps stored per request in sorted set	Strict accuracy	High memory overhead	Highest	Overkill

---

4. Trade-offs Accepted

• Lack of Endpoint Versioning: Breaking changes to the /chat JSON schema would impact existing frontend clients without a transition period.
• Open Execution: Malicious scripts can loop over /chat with thousands of automated queries, causing CPU spikes.

---

5. What Breaks at Scale

• At 10x Load: Normal client usage does not trigger failures.
• At 100x Load: A single abusive scraper or botnet can monopolize all CPU threads, starving legitimate users.
• At 1000x Load: Server enters complete CPU starvation without rate limiting.

---

6. How a Senior Engineer Would Improve It

1. Mount routes under an explicit version prefix: /api/v1/chat and /api/v1/audit/export.
2. Add rate limiting using SlowAPI or an Nginx limit_req_zone:

   limiter = Limiter(key_func=get_remote_address)
   @app.post("/api/v1/chat")
   @limiter.limit("30/minute")
   def chat_endpoint(...): ...

---

7. 30-Second Spoken Answer

"We structured our API around clean REST conventions using Pydantic request models and dedicated endpoints for chat and CSV streaming. In production, we would prefix routes with /v1/ to support non-breaking contract changes and enforce a Token Bucket rate limit of thirty requests per minute per IP using SlowAPI or an Nginx gateway to prevent CPU starvation from automated scrapers."