LLM Datasets - Curate Production Traces for Evaluations
Two surfaces: Data Hub curates requests; Evaluation Datasets freeze them for runs. Sample, branch, and version with confidence.
Data Hub
Mine your own traffic
Turn real production requests into curated datasets with a few clicks — sampled, deduplicated, and ready to use. Your best test data is the traffic you already have.
- Sample live traffic by any filter
- Dedup and clean automatically
Evaluation Datasets
Freeze for truth
Lock a dataset into an immutable version so every evaluation is reproducible. What you test today is exactly what you test next month.
- Train / dev / test partitions
- Immutable frozen versions
- Annotate with human verdicts
Workflow & Export
From data to gate
Sample, freeze, run, and gate — then export anywhere your stack needs it, from OpenAI Batch to raw JSON. Your eval loop, your formats.
- Open formats: JSONL, CSV, JSON
- Import straight from production
- Feed directly into gated deploys
10/page • SearchNew Dataset ▾
support-eval • 342 rowsView Rows →
Avg Cost/Row
$0.002
Success Rate
98%
Random Sample 100From Template
v2 frozenv3 draft18 examples
All (18)train 10dev 5
Import from prod → token/time window → partition
Commit versionAnnotate
Sample→Freeze→Run→Gate
JSONL
OpenAI Batch
CSV
Spreadsheet
JSON
Raw
Training
Template
Curate your first dataset
Random sample or import a trace — then freeze a version.