AI & Automation

TraceSift

Distributed tracing pipelines frequently ingest massive volumes of repetitive spans, inflating observability ingestion bills while making it difficult to detect anomalous latencies across complex microservice calls. TraceSift solves this by parsing distributed spans, establishing baseline latencies, triaging performance anomalies, and calculating sampling strategies to lower telemetry costs. ## Key features - **Flat JSON Span Normalization**: Reads flat JSON span records and maps common field names (trace_id/traceId/traceID, startTime/timestamp, duration in ms, µs or ns) with automatic timestamp and duration conversion. Nested OTLP, Jaeger or Zipkin export files need flattening first. - **Statistical Anomaly Detection**: Profiles non-error operations to compute p95 baselines and flags spans using duration z-scores and error states. - **Cost Optimization Modeling**: Measures noise ratios and volume shares to recommend sampling rates and estimate monthly cloud ingestion savings. - **Synthetic Dataset Generation**: Includes a mock topology generator producing multi-tier microservice traces with simulated latency spikes and cascading failures. ## Example ```bash python tracesift.py --help python tracesift.py --generate-mock --mock-count 200 --cost-per-gb 0.75 python tracesift.py --input traces.json --output report.json ``` ## Limitations TraceSift is an in-memory batch processing utility that analyzes local JSON data or synthetic traces rather than a live collector daemon capable of streaming network telemetry in real time.

python tracesift.py --help