LLMCostGuard: Guida alle Guardrail di Fatturazione
#python#automation#microsaas
Elimina le spese LLM fuori controllo grazie a questa autorevole guida ingegneristica su Dev.to. Presentando LLMCostGuard — un token proxy Go & Docker zero-config — risolve i costosi loop API con rigidi budget cap.
Leggi l'approfondimento tecnicoPrevenire Costi LLM Fuori Controllo con il Budget Proxy LLMCostGuardmain.pymain.py — estratto d'esempio nello stile del tool
#!/usr/bin/env python3
"""
LLMCostGuard - Real-time LLM Token Budgeting & Billing Guardrails
Zero-config API proxy and middleware engine designed to prevent runaway LLM costs
by enforcing hard token limits, cost caps, and automated request throttling across
providers like OpenAI, Anthropic, and custom local endpoints.
Features:
- Multi-provider cost tracking with dynamic token pricing models
- Hard budget caps with configurable breach actions (block, degrade, alert)
- Sliding window token bucket rate limiting per API key/tenant
- Real-time spending metrics export and anomaly detection
- Zero-dependency standard library Python engine with async support
Usage:
$ python main.py --config config.json --budget-usd 50.00
$ python main.py --proxy-port 8080 --max-tokens-per-min 100000
"""
from dataclasses import dataclass, field
from enum import Enum, auto
import logging
import time
from typing import Dict, List, Optional, Tuple, Union
class BreachAction(Enum):
"""Action to execute when budget or token limit is exceeded."""
BLOCK = auto()
THROTTLE = auto()
ALERT_ONLY = auto()
FALLBACK_MODEL = auto()
@dataclass(frozen=True)
class TokenBudget:
"""Defines budget constraints for a target provider or user session."""
max_cost_usd: float
max_tokens: int
window_seconds: int = 3600
breach_action: BreachAction = BreachAction.BLOCK
fallback_model: Optional[str] = None
class CostGuardEngine:
"""Core guardrail controller evaluating requests against budgets."""
def __init__(self, budget: TokenBudget) -> None:
self.budget = budget
self._total_spent_usd: float = 0.0
self._total_tokens_used: int = 0
# ... the full file ships with your download