Skip to main content
A start-here for academic and benchmarking use: how to reproduce every published number, the evaluation methodology behind the headline claims (including the honest caveats), how to cite the project, and where the academic comparisons live.

​
Cite it

The repository carries a CITATION.cff, so GitHub shows a “Cite this repository” button (sidebar → Cite this repository) that exports BibTeX / APA. Cite the headline package as GoldenMatch (Golden Suite), author Ben Severn, MIT-licensed, at github.com/benseverndev-oss/goldenmatch.

​
Reproduce the benchmarks

Every published headline number maps back to a committed runner (scripts/run_benchmarks.py) with a pinned environment and a stated tolerance. The canonical guide is docs/reproducing-benchmarks.md — it gives the dataset source, drop location, and expected output per number.
No-LLM numbers are deterministic given fixed seeds and a fixed dataset (allow sub-0.1 pp drift from rapidfuzz reduction order). --with-llm runs are not re-runnable targets — treat them as single observations.
GoldenMatch’s zero-tuning probabilistic (Fellegi-Sunter) auto-config is benchmarked against an expert hand-rolled Splink config on one shared evaluator across every dataset Splink scores. Full writeup, including per-dataset P/R and performance: docs/benchmarks/2026-06-09-splink-bakeoff.md. Read the caveats before quoting these:
  • These are pairwise F1 under one shared evaluator (evaluate.evaluate), so all engines are judged by identical code. The often-cited ~0.97 Splink number on historical_50k is a cluster/entity metric, not exhaustive within-cluster pairwise F1 — under this harness Splink scores ~0.757 pairwise (recall-bound; no single field exceeds 0.60 recall → ~0.93 pairwise ceiling for any engine). The honest claim is “matches/beats Splink on the same evaluator,” not “0.97 pairwise.”
  • Reproducible as of the determinism fix (#829). The earlier figures rested on a non-deterministic EM training-pair sample (one CI run gave 0.805 / 0.779 / 0.643 for the identical path); sorting blocks by a stable key before the seeded shuffle made it reproduce within 0.002.
  • Splink is faster (3–19× on these datasets) and retains distributed F-S at 1B+ rows on Spark plus the mature m/u comparison-viewer UI. GoldenMatch’s edge is zero tuning and accuracy parity.
  • Bibliographic data: the probabilistic path is weak on dblp_acm (F1 0.377, recall-bound); use the weighted path there (0.964), not probabilistic.

​
Evaluation methodology

The same primitives are available to you for your own measurements — pairwise precision / recall / F1 against a ground-truth pair set, cluster-level B³, and ground-truth-free run comparison (CCMS / Talburt-Wang Index). See the Evaluation guide for the evaluate, evaluate_clusters, compare_clusters, and goldenmatch evaluate --threshold-sweep APIs.

​
Research arcs (documented negative results)

Two exploratory ER designs are written up honestly, including where they didn’t pan out — useful as prior art:

Amortized Bayesian ER

A calibrated neural posterior over the partition with EIG-driven active labelling. Mechanisms validated, not accuracy-competitive.

Landscape-sculpting ER

ER reframed as sculpting an attractor landscape. Novel framing, but cosmetic — same partition as a calibrated discrete split/merge loop.

​
Compare against other tools

Vendor & tool comparison

GoldenMatch vs Splink, Dedupe, Senzing, and others — feature and accuracy axes.

Entity resolution concepts

Blocking, scoring, clustering, survivorship — the model the benchmarks exercise.