Benchmarks
Scenarios with known answers. Ablations that show why each part is there.
Three synthetic scenarios have gold labels for event membership, lineage families, contradictions, and reused media. The pipeline runs on each under the default configuration and with one capability switched off. Every run is reproduced on this page at build time; nothing here is a stored number.
Synthetic fixtures · labelled · deterministic · generated 2026-03-17 11:12Z
S1
Derivative copies are not independent witnesses
A wire story is republished by three outlets. One eyewitness post and one seismic reading are independent. A second quake elsewhere must stay separate.
7 items · 2 gold events · 4 lineage families · 0 contradictions · 0 reused media sets
| Metric | default | no dependency discount | no time uncertainty | no cross-modal | Target |
|---|---|---|---|---|---|
| event pairwise f1 precision 1, recall 1 | 1 | 1 | 1 | 1 | ≥ 0.9 |
| lineage pairwise f1 precision 1, recall 1 | 1 | 1 | 1 | 1 | ≥ 0.9 |
| independent support:S1-E1 counted 2 independent of 6 items | 2 | 5 ✗ | 2 | 2 | ≤ 2 text families (items 6) |
| status:S1-E1 corroborated, index 0.632 | — | — | — | — | corroborated or disputed when evidence exists |
| independent support:S1-E2 counted 0 independent of 1 items | 0 | 0 | 0 | 0 | ≤ 0 text families (items 1) |
| status:S1-E2 candidate, index 0.485 | — | — | — | — | corroborated or disputed when evidence exists |
| traceability 19 claims | 1 | 1 | 1 | 1 | every claim resolves to a span |
- · independent_supporting for E1 is 2 or 3, not 5 or 6
- · E1 corroborated with instrument backing
- · E2 remains a separate candidate
S2
Same type, same day, different places
Two explosions on the same day in Beirut and Karachi must not merge. A ministry denial inside the Beirut event must surface as a contradiction, not as a separate event.
5 items · 2 gold events · 5 lineage families · 1 contradictions · 0 reused media sets
| Metric | default | no dependency discount | no time uncertainty | no cross-modal | Target |
|---|---|---|---|---|---|
| event pairwise f1 precision 1, recall 1 | 1 | 1 | 1 | 1 | ≥ 0.9 |
| lineage pairwise f1 precision 1, recall 1 | 1 | 1 | 1 | 1 | ≥ 0.9 |
| traceability 10 claims | 1 | 1 | 1 | 1 | every claim resolves to a span |
| independent support:S2-E1 counted 2 independent of 3 items | 2 | 2 | 2 | 2 | ≤ 3 text families (items 3) |
| status:S2-E1 disputed, index 0.203 | — | — | — | — | corroborated or disputed when evidence exists |
| independent support:S2-E2 counted 2 independent of 2 items | 2 | 2 | 2 | 2 | ≤ 2 text families (items 2) |
| status:S2-E2 corroborated, index 0.448 | — | — | — | — | corroborated or disputed when evidence exists |
| contradiction:occurrence status surfaced with claim anchors | 1 | 1 | 1 | 1 | detected |
- · Beirut and Karachi stay separate (cannot-link on place)
- · Beirut event shows an occurrence_status or casualties contradiction and needs review
- · Karachi event corroborated by two independent reports
S3
Recycled image and coordinated burst inside a real flood
A real flood with an official alert and one newsroom report. Four accounts post identical alarmist text within a minute. A viral post reuses a 2019 photograph.
8 items · 1 gold events · 5 lineage families · 0 contradictions · 1 reused media sets
| Metric | default | no dependency discount | no time uncertainty | no cross-modal | Target |
|---|---|---|---|---|---|
| event pairwise f1 precision 1, recall 1 | 1 | 1 | 0.833 ✗ | 1 | ≥ 0.9 |
| lineage pairwise f1 precision 1, recall 1 | 1 | 1 | 1 | 1 | ≥ 0.9 |
| traceability 9 claims | 1 | 1 | 1 | 1 | every claim resolves to a span |
| independent support:S3-E1 counted 3 independent of 7 items | 3 | 6 ✗ | 2 | 3 | ≤ 3 text families (items 7) |
| status:S3-E1 corroborated, index 0.394 | — | — | — | — | corroborated or disputed when evidence exists |
| recycled media first seen 2019-06-10; event marked needs_review | 1 | 1 | 1 | 1 | flagged and surfaced on the event |
| coordination signal shared-object burst detected | 1 | 1 | 1 | 1 | raised |
- · Burst posts collapse into one lineage group and raise a coordination signal
- · Reused photograph is flagged with its first-seen date
- · Flood corroborated by official alert plus independent newsroom
Live corpus
Operating measures on the current public snapshot.
Real data has no gold labels, so these are operating measures rather than accuracy. They are reported so a reader can see the scale and the behaviour of the independence discount on live material.
- Items
- 2,496
- Sources online
- 11
- Events
- 1182
- Corroborated
- 98
- Derivative items
- 483
- Cross-modal events
- 29
- Unknown place rate
- 35%
- Traceability
- 100%
Counted once in support.
Items whose place could not be resolved.
Claims whose span resolves to stored text.
Snapshot 2026-10-23 14:15Z · live store
What is not here: a calibrated probability, a machine-text detector score, or a synthetic-image verdict. The interfaces exist and the readiness table on the product page says where they stand.