Every call, with its receipt.
Prompt, tools, retrieval, streamed output, usage and cost — one span tree per request, searchable in under 400 ms across two billion spans a day. Instrument in four lines or point your existing OpenTelemetry exporter at us.
Four lines. Then every call is a trace.
The decorator wraps the function; the SDK patches your model client. Nested calls become child spans automatically. Nothing blocks the request thread — spans are batched and flushed off-thread as Arrow.
from tracewell import Tracewell, trace
tw = Tracewell(api_key="tw_live_…", project="support-copilot")
@trace(name="answer_ticket", capture=["prompt", "tools", "cost"])
def answer(ticket):
ctx = retrieve(ticket.text, k=12) # → retrieval span
orders = tools.search_orders(ticket.user_id) # → tool span
return llm.complete(render(ctx, orders)) # → llm.completion span
# ✓ trace tr_8f2a41c0 · 6 spans · 1,549 ms · $0.0041import { Tracewell, trace } from "@tracewell/sdk";
const tw = new Tracewell({ apiKey: process.env.TW_KEY!, project: "support-copilot" });
export const answer = trace("answer_ticket", async (ticket: Ticket) => {
const ctx = await retrieve(ticket.text, { k: 12 }); // retrieval span
const orders = await tools.searchOrders(ticket.userId); // tool span
return llm.complete(render(ctx, orders)); // llm.completion span
}, { capture: ["prompt", "tools", "cost"] });
// ✓ trace tr_8f2a41c0 · 6 spans · 1,549 ms · $0.0041curl https://api.tracewell.dev/v1/spans \
-H "Authorization: Bearer $TW_KEY" \
-H "Content-Type: application/json" \
-d '{
"trace_id": "tr_8f2a41c0",
"name": "llm.completion",
"model": "gpt-4.1",
"usage": { "in": 2118, "out": 412, "cached": 1536 },
"ttft_ms": 388, "duration_ms": 1204,
"attributes": { "tenant": "lumeo", "feature": "support-copilot" }
}'
# 202 Accepted · span_id sp_71c0a9 · ingest lag 0.8 s02 — Span tree
The whole request, nested.
A request is a tree, not a line in a log. Tracewell keeps the parent–child structure your code already has: the agent step, the tool it called, the retrieval that fed it, the completion that answered. Click any node for its payload.
- 01Payloads stored verbatim — prompt, arguments, raw tool response, streamed tokens with timestamps.
- 02Redaction in-process — email, card numbers, secrets and your regexes are masked before the span leaves your service.
- 03Time-to-first-token measured per completion, separate from total duration.
output[0:80] "Your order #41882 shipped on 12 Oct and is with the carrier. Tracking shows…"
03 — Tools & retrieval
See what the agent actually did.
Tool calls are first-class spans with arguments, result, status and retries. Retrieval spans keep every chunk, its score, its source and — after the completion — whether the answer used it. Loops, dead tools and ignored context stop being theories.
- 01Loop detection — same tool, same arguments, N times → flagged on the trace and in alerts.
- 02Chunk usage — attribution from output back to retrieved context, per chunk.
- 03Any vector store — pgvector, Pinecone, Weaviate, Qdrant, your own.
loop check · 1 call in trace · no repeat
04 — Cost attribution
Know which feature spent the money.
Every completion is priced at the provider's sheet the moment it happens, including cached tokens and batch discounts. Roll it up by feature, tenant, model, release or any attribute you attach. Export to your billing system nightly.
- 01Per customer — cost of goods per tenant, ready for your margin report.
- 02Per release — the prompt change that doubled tokens shows up as a step in the chart.
- 03Budgets — soft and hard caps per feature with alerts at 80%.
05 — Latency
p95 is a span, not a guess.
Latency breaks down per span type, so you learn whether the slow tail is the model, the vector store or your own tool. Compare p50 / p95 / p99 by model and release. Set a gate on it.
- 01TTFT vs total — streaming feels fast when first token is fast; we measure both.
- 02Provider vs you — queueing inside your service is separated from provider time.
- 03Histogram per model — HDR histograms, 0.1% precision, 13 months.
What happens in the 0.8 seconds after your call.
01 · SDK, in your process
Spans are built on the request thread in microseconds, redacted by your rules, then batched off-thread and shipped as Arrow every 200 ms. p95 overhead: 3 ms.
02 · Collector
Stateless OTLP endpoints in eu-central-1 and us-west-2. Backpressure, dedupe, schema validation. Tested to 61k spans/s per tenant.
03 · Store
Columnar storage on object store, indexed by tenant, feature, model and time. Payloads encrypted with per-tenant keys you can revoke.
04 · Query
The console and the API read the same engine: filters over 2.4B spans return in 388 ms at p95. Saved views, shareable trace links.
05 · Evals & gates
Judges score sampled traces asynchronously. Datasets grow from failures. CI gates compare the branch against main before merge.
The numbers you'll ask in the RFP.
Published, not negotiated. If a number here is wrong for you, the Enterprise plan changes it — and we'll say so in writing.
| Dimension | Developer | Team | Enterprise |
|---|---|---|---|
| Spans / month | 1M | 20M included · $8 / 1M after | unlimited · volume pricing |
| Retention · spans | 7 days | 30 days | 13 months |
| Retention · payloads | 7 days | 30 days | 13 months · BYO bucket |
| Max payload per span | 256 KB | 1 MB | 4 MB |
| Ingest rate | 500 spans/s | 5,000 spans/s | 61,000 spans/s tested |
| Query latency · p95 | 388 ms over 2.4B spans (30-day window, 6 filters) | ||
| SDK overhead · p95 | 3 ms wall-clock, 0 allocations on request thread | ||
| Regions | eu-central-1 | eu-central-1 · us-west-2 | + self-host (Helm) |
| Export | CSV | CSV · Parquet · webhook | + S3/GCS sync · warehouse |
| SSO · audit log | — | SAML · 90-day audit | SAML · SCIM · 13-month audit |
Your first trace is four lines away.
One million spans a month free. OpenTelemetry users: change one exporter URL and you're done.