SaaS rate limiting gateways introduce unwanted network latency, vendor lock-in, and unpredictable billing models. When an API endpoint comes under heavy brute-force attack or automated bot scraping, traditional Redis-backed counters frequently fail due to connection pool exhaustion and network roundtrip overhead.
The Architectural Flaw of Centralized Counters
Executing an out-of-process network call to Redis on every HTTP request adds 2ms to 15ms of latency under normal load. During a distributed denial-of-service or scraping attempt, Redis CPU spikes, socket queues fill up, and the security layer becomes the bottleneck that takes down your entire infrastructure.
Dynamic Tarpitting vs. Instant 429 Rejections
Standard rate limiters instantly return an HTTP 429 status code. This allows scrapers to immediately retry with rotated proxy IPs. Dynamic tarpitting solves this by progressively delaying responses for abusive clients—introducing artificial sleep delays (e.g., 500ms to 5,000ms) before returning an error. This exhausts the attacker's thread pool and connection budget while costing you virtually zero CPU resources.
import time
from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse
app = FastAPI()
class LocalTarpitGuard:
def __init__(self, max_requests: int = 60, window_sec: int = 60):
self.max_requests = max_requests
self.window_sec = window_sec
self.clients = {}
def inspect(self, client_ip: str) -> float:
now = time.time()
count, reset_at, delay = self.clients.get(client_ip, (0, now + self.window_sec, 0.0))
if now > reset_at:
count, reset_at, delay = 0, now + self.window_sec, 0.0
count += 1
if count > self.max_requests:
delay = min(delay + 0.5, 5.0) # Exponential tarpit growth
self.clients[client_ip] = (count, reset_at, delay)
return delay
self.clients[client_ip] = (count, reset_at, delay)
return 0.0
guard = LocalTarpitGuard(max_requests=50, window_sec=60)
@app.middleware("http")
async def apply_rateshield_tarpit(request: Request, call_next):
client_ip = request.client.host
delay = guard.inspect(client_ip)
if delay > 0:
time.sleep(delay) # Hold connection open to exhaust attacker pool
if delay >= 2.0:
return JSONResponse(status_code=429, content={"error": "Rate limit exceeded. Connection throttled."})
return await call_next(request)Deploying RateShield via Docker
RateShield combines ultra-low overhead Go primitives with a turnkey Docker setup. Positioned as a sidecar or edge proxy in front of Python microservices, it processes over 100,000 requests per second at sub-millisecond execution times without external state dependencies.