GoldenMatchConfig and every nested object), the enumerated
string vocabularies, and the GOLDENMATCH_* runtime environment knobs. It is
built for both humans and AI agents that need a single, complete, always-current
map of what is configurable.
The object reference, vocabularies, and env index below the line are
generated from code (
scripts/gen_config_matrix.py) and verified in CI, so
they can never silently drift from the engine. The combinations & outcomes
section above the line is hand-authored guidance. For prose walkthroughs see
Configuration (YAML fields), Scoring,
Blocking, and Tuning & opt-ins
(env-var semantics + defaults).How to read this page
- Combinations & outcomes (below) is the decision layer: which knob to reach for, and what happens when knobs interact.
- Config object reference is the structural layer: every field, its type, its default, and its allowed Literal values, per config object.
- Enumerated vocabularies lists the allowed values for the
str-typed fields (scorers, strategies, standardizers). - Environment variables indexes every
GOLDENMATCH_*knob by area; defaults and effects are in Tuning & opt-ins.
Combinations & intended outcomes
Matchkey type: exact vs weighted vs probabilistic
The single most consequential choice. Set per matchkey via MatchkeyConfig.type.
Missing values x scorer/type
MatchkeyConfig.missing (unobserved vs disagree) is probabilistic-only.
- Auto-config picks the mode from profiled null rates (
disagreewhen the worst comparison field is >= 20% null); override withMatchkeyConfig.missingor the globalGOLDENMATCH_FS_MISSING. - Native kernel caveat: the native FS kernel implements only
unobserved; adisagree-mode matchkey routes to the numpy path automatically (correctness over speed). - On weighted matchkeys there is no
missingknob — a null field is simply dropped from the weighted mean (renormalized by observed weight).
Blocking strategy x data scale
BlockingConfig.strategy selects candidate generation. Any weighted/probabilistic
matchkey requires a blocking config. A null block key means “cannot be
blocked”, never “blocks with every other null-key row” — null keys are filtered
across every strategy (no false mega-blocks).
Guardrails regardless of strategy:
max_block_size + skip_oversized, max_total_comparisons, and auto_suggest / auto_select to let auto-config choose keys/strategy.
Execution routing: in-memory vs bucket vs distributed vs scale-mode
Left mostly to the controller, but pinnable.
Env-level routing (see the env index):
GOLDENMATCH_BUCKET_DEFAULT (the default in-memory bucket FS route up to ~750K rows), GOLDENMATCH_MATCH_FUSED (fused block->score->cluster kill-switch; fires under RSS pressure), GOLDENMATCH_NATIVE / GOLDENMATCH_FRAME (native kernel + arrow/polars frame seam). The controller refuses a low-confidence (“RED”) auto-config at >= 100K rows unless allow_red_config is set.
Dedupe vs link
Not an enum — it is expressed byInputConfig: a single files list = dedupe
(find duplicates within one dataset); file_a + file_b = link (match across
two datasets, cross-source pairs only).
Survivorship: field rules, groups, conditional, correlated
golden_rules decides the surviving value per field after clustering.
field_rulesmap a column to astrategy(see the survivorship vocabulary). List form = conditional rules (when:predicates over already-resolved fields, first match wins, one when-less default last).field_groupspin >= 2 columns lock-step to one winning row (e.g.city/state/zipmust come from the same record) via a groupstrategy(most_complete/source_priority/most_recent/anchor).- Correlated / conditional / validated survivorship runs in-memory only — it is refused on the distributed/Sail paths.
quality_weighting,weak_cluster_threshold,auto_split+max_cluster_sizecontrol how weak/oversized clusters are downgraded or split before survivorship.
Multi-table / collective ER: propagation_mode
For graph/relational ER across entity types (graph.propagation_mode):
Native / frame kill-switches
Defaults are fast: arrow frames + native kernel where available.GOLDENMATCH_NATIVE=auto
falls back to pure Python silently if the compiled wheel is absent; =1 requires
it (raises), =0 forces Python. GOLDENMATCH_FRAME=polars restores the Polars
frame seam (needs the [polars] extra). Reach for these only to reproduce a
result across environments or to isolate a perf/parity question.
Config object reference
Every config object in the pydantic tree(s), generated from the package schema. Nested objects link by name.GoldenMatchConfig
InputConfig
OutputConfig
MatchSettingsConfig
MatchkeyConfig
A matchkey: rule for declaring two records ‘the same’ on a field/field-set.
BlockingConfig
GoldenRulesConfig
StandardizationConfig
ValidationConfig
QualityConfig
GoldenCheck integration config for enhanced data quality.
TransformConfig
GoldenFlow integration config for data transformation.
LLMScorerConfig
DomainConfig
DistributedRoutingConfig
Per-stage distributed-routing pins. auto lets the planner decide;
SemanticBlockingConfig
Opt-in semantic candidate-generation (recall-lever) config. Carries the
ThroughputConfig
Opt-in sketch-then-verify throughput tier (#1083).
MemoryConfig
Learning Memory configuration.
IdentityConfig
Identity Graph configuration.
InputFileConfig
MatchkeyField
NegativeEvidenceField
v1.11: a field whose disagreement subtracts from a weighted matchkey’s
BlockingKeyConfig
SortKeyField
CanopyConfig
LSHKeyConfig
MinHash/LSH blocking on a text column (#1081).
SimHashKeyConfig
SimHash/LSH blocking on a text column via dense embeddings (#1082).
PerceptualKeyConfig
Banded hamming-LSH blocking over a column of perceptual hashes (ADR 0022).
GoldenFieldRule
GoldenGroupRule
ValidationRuleConfig
BudgetConfig
LearningConfig
Learning Memory learning parameters.
ChannelStitchConfig
Cross-device / channel stitching configuration (#1110, epic #1108).
SurvivorshipConfig
Golden-record survivorship configuration (#1111, epic #1108).
StabilizationConfig
Cross-run entity stabilization — Identity v3 (#1112, epic #1108).
MediationConfig
Conflict mediation workflow — Identity v3 (#1113, epic #1108).
CLI
Every command and its options/arguments, generated from the Typer app.choice-typed options list their allowed values.
MCP tools
82 MCP tool(s) exposed bygoldenmatch.mcp.server — the programmatic / agent surface. Config-bearing tools take the same knobs as above.
Enumerated vocabularies
Allowed values for thestr-typed / registry-backed fields above.
Scorers
VALID_SCORERS — MatchkeyField.scorer / NegativeEvidenceField.scorer.
Blocking strategies
BlockingConfig.strategy — BlockingConfig.strategy.
Simple transforms
VALID_SIMPLE_TRANSFORMS — transforms chains.
Survivorship strategies
VALID_STRATEGIES — GoldenFieldRule.strategy.
Group survivorship strategies
_GROUP_STRATEGIES — GoldenGroupRule.strategy.
Standardizers
VALID_STANDARDIZERS — StandardizationConfig.rules.
Matchkey types
_VALID_MK_TYPES — MatchkeyConfig.type.
Backends
BackendName — GoldenMatchConfig.backend / --backend.
Clustering strategies
ClusteringStrategy — planner cluster route.
Planning efforts
_PLANNING_EFFORTS — planning_effort.
Environment variables (index)
166GOLDENMATCH_* runtime knob(s) read by the package, scanned from source so this is complete. Details in Tuning & opt-ins. Grouped by area:
- AGENT (1):
GOLDENMATCH_AGENT_TOKEN - ALLOWED (1):
GOLDENMATCH_ALLOWED_ROOT - ANALYTICS (1):
GOLDENMATCH_ANALYTICS - ANN (6):
GOLDENMATCH_ANN_BACKEND,GOLDENMATCH_ANN_HNSW_EF_CONSTRUCTION,GOLDENMATCH_ANN_HNSW_EF_SEARCH,GOLDENMATCH_ANN_HNSW_M,GOLDENMATCH_ANN_HNSW_MAX_K,GOLDENMATCH_ANN_HNSW_MIN - API (2):
GOLDENMATCH_API_CORS_ORIGINS,GOLDENMATCH_API_TOKEN - ATTRIBUTE (1):
GOLDENMATCH_ATTRIBUTE_DEMOTION - AUTO (1):
GOLDENMATCH_AUTO_SEMANTIC_BLOCKING - AUTOCONFIG (10):
GOLDENMATCH_AUTOCONFIG_ARROW_NATIVE,GOLDENMATCH_AUTOCONFIG_FORCE_EXCLUDE,GOLDENMATCH_AUTOCONFIG_FORCE_INCLUDE,GOLDENMATCH_AUTOCONFIG_INDICATOR_BUDGET,GOLDENMATCH_AUTOCONFIG_LLM,GOLDENMATCH_AUTOCONFIG_MEMORY,GOLDENMATCH_AUTOCONFIG_ROUTE_PROBABILISTIC,GOLDENMATCH_AUTOCONFIG_SAMPLE_STRATEGY,GOLDENMATCH_AUTOCONFIG_ZERO_LABEL_COMMIT,GOLDENMATCH_AUTOCONFIG_ZERO_LABEL_STABILITY - BASE (1):
GOLDENMATCH_BASE_STORE - BENCH (1):
GOLDENMATCH_BENCH_DUMP_PAIRS - BLOCKING (8):
GOLDENMATCH_BLOCKING_CARDINALITY_SCALER,GOLDENMATCH_BLOCKING_DEGENERATE_MAX_AVG_BLOCK_SIZE,GOLDENMATCH_BLOCKING_DEGENERATE_THRESHOLD,GOLDENMATCH_BLOCKING_MAX_RATIO,GOLDENMATCH_BLOCKING_MIN_RATIO,GOLDENMATCH_BLOCKING_PAIRS_PER_ROW,GOLDENMATCH_BLOCKING_PASS_MIN_WEAKPOS,GOLDENMATCH_BLOCKING_PRUNE_PASSES - BRIDGE (1):
GOLDENMATCH_BRIDGE_REQUIRE_PY - BUCKET (5):
GOLDENMATCH_BUCKET_DEBUG,GOLDENMATCH_BUCKET_DEFAULT,GOLDENMATCH_BUCKET_SLIM_PROJECTION,GOLDENMATCH_BUCKET_VEC_MAX,GOLDENMATCH_BUCKET_VEC_MIN - CLUSTER (1):
GOLDENMATCH_CLUSTER_SPLIT_EDGE_BUDGET - COLUMNAR (1):
GOLDENMATCH_COLUMNAR_PIPELINE - CONFIG (1):
GOLDENMATCH_CONFIG_LINT - DATABASE (1):
GOLDENMATCH_DATABASE_URL - DISCRIMINATIVE (2):
GOLDENMATCH_DISCRIMINATIVE_TAU,GOLDENMATCH_DISCRIMINATIVE_VETO - DISTRIBUTED (13):
GOLDENMATCH_DISTRIBUTED_BLOCK_SHUFFLE,GOLDENMATCH_DISTRIBUTED_CLUSTERING_THRESHOLD,GOLDENMATCH_DISTRIBUTED_FS_TRAIN_ROWS,GOLDENMATCH_DISTRIBUTED_GOLDEN_THRESHOLD,GOLDENMATCH_DISTRIBUTED_OP_RESERVATION,GOLDENMATCH_DISTRIBUTED_PIPELINE,GOLDENMATCH_DISTRIBUTED_SCORE_CONCURRENCY,GOLDENMATCH_DISTRIBUTED_SCORE_NUM_CPUS,GOLDENMATCH_DISTRIBUTED_SCORE_PROJECT,GOLDENMATCH_DISTRIBUTED_SHUFFLE_PARTS,GOLDENMATCH_DISTRIBUTED_WCC,GOLDENMATCH_DISTRIBUTED_WCC_SCRATCH,GOLDENMATCH_DISTRIBUTED_WCC_SEED - DUCKDB (1):
GOLDENMATCH_DUCKDB_SCORE_DB - EMBEDDING (1):
GOLDENMATCH_EMBEDDING_PROVIDER - ENABLE (1):
GOLDENMATCH_ENABLE_DISTRIBUTED_RAY - ENSEMBLE (1):
GOLDENMATCH_ENSEMBLE_KERNEL - FACILITY (1):
GOLDENMATCH_FACILITY_NAME_NE - FD (1):
GOLDENMATCH_FD_NEGATIVE_EVIDENCE - FIELD (1):
GOLDENMATCH_FIELD_GROUP_SURVIVORSHIP - FRAME (2):
GOLDENMATCH_FRAME,GOLDENMATCH_FRAME_LANE - FS (28):
GOLDENMATCH_FS_ARROW_STREAM,GOLDENMATCH_FS_AUTOCONFIG_V2,GOLDENMATCH_FS_BATCH_ROWS,GOLDENMATCH_FS_BLOCK_SOURCE,GOLDENMATCH_FS_BUCKET_NATIVE,GOLDENMATCH_FS_CALIBRATED,GOLDENMATCH_FS_COLUMNAR_CLUSTER,GOLDENMATCH_FS_DEFAULT_BUCKET,GOLDENMATCH_FS_DOMAIN_COMPARATORS,GOLDENMATCH_FS_EM_AGG_BLOCKS,GOLDENMATCH_FS_EM_BLOCK_SLIM,GOLDENMATCH_FS_EM_SAMPLE_ROWS,GOLDENMATCH_FS_MAX_PASS_PAIRS,GOLDENMATCH_FS_MISSING,GOLDENMATCH_FS_MONOTONIC,GOLDENMATCH_FS_NATIVE,GOLDENMATCH_FS_OOC_ARROW_CLUSTER,GOLDENMATCH_FS_OOC_DEBUG,GOLDENMATCH_FS_OOC_WAVE_ROWS,GOLDENMATCH_FS_OUT_OF_CORE,GOLDENMATCH_FS_REQUIRE_POSITIVE_EVIDENCE,GOLDENMATCH_FS_ROUTE_MIN_ROWS,GOLDENMATCH_FS_SCORED_PAIRS_MAX,GOLDENMATCH_FS_STRIP_HONORIFICS,GOLDENMATCH_FS_TF_ADJUSTMENT,GOLDENMATCH_FS_VECTORIZED,GOLDENMATCH_FS_VEC_MAX_ELEMS,GOLDENMATCH_FS_WORKERS - FUSED (5):
GOLDENMATCH_FUSED_BLOCK_CONCURRENCY,GOLDENMATCH_FUSED_BYTES_PER_CELL,GOLDENMATCH_FUSED_BYTES_PER_PAIR,GOLDENMATCH_FUSED_PRESSURE_FRACTION,GOLDENMATCH_FUSED_RSS_SCALE - GOLDEN (4):
GOLDENMATCH_GOLDEN_FUSED,GOLDENMATCH_GOLDEN_SLIM_MULTIDF,GOLDENMATCH_GOLDEN_STRATEGY_STRICT,GOLDENMATCH_GOLDEN_TUNER_MIN_CORRECTIONS - GPU (3):
GOLDENMATCH_GPU_API_KEY,GOLDENMATCH_GPU_ENDPOINT,GOLDENMATCH_GPU_MODE - HEAL (2):
GOLDENMATCH_HEAL_MIN_HEALTH_GAIN,GOLDENMATCH_HEAL_STEP_CAP - IDENTITY (3):
GOLDENMATCH_IDENTITY_BATCH_FINGERPRINT,GOLDENMATCH_IDENTITY_DSN,GOLDENMATCH_IDENTITY_WRITE_PIPELINE - INHOUSE (1):
GOLDENMATCH_INHOUSE_MODEL - LLAMA (1):
GOLDENMATCH_LLAMA_GGUF - LLM (2):
GOLDENMATCH_LLM_BASE_URL,GOLDENMATCH_LLM_MODEL - MATCH (2):
GOLDENMATCH_MATCH_FUSED,GOLDENMATCH_MATCH_LOG_FLUSH_PAIRS - MCP (6):
GOLDENMATCH_MCP_ALLOW_PUBLIC,GOLDENMATCH_MCP_MAX_UPLOAD_BYTES,GOLDENMATCH_MCP_SESSION_MAX,GOLDENMATCH_MCP_SESSION_TTL,GOLDENMATCH_MCP_TOKEN,GOLDENMATCH_MCP_UPLOAD_TTL - MULTISOURCE (1):
GOLDENMATCH_MULTISOURCE_AUTOCONFIG - NATIVE (5):
GOLDENMATCH_NATIVE,GOLDENMATCH_NATIVE_ADDRESS_NORMALIZE,GOLDENMATCH_NATIVE_RAYON_MIN_BLOOM_ROWS,GOLDENMATCH_NATIVE_RAYON_MIN_PAIRS,GOLDENMATCH_NATIVE_SKETCH_RAYON_MIN_ROWS - NE (1):
GOLDENMATCH_NE_TUNER_MIN_CORRECTIONS - NOISE (2):
GOLDENMATCH_NOISE_AWARE_SCORERS,GOLDENMATCH_NOISE_AWARE_TARGET - PERCEPTUAL (1):
GOLDENMATCH_PERCEPTUAL_AUTOCONFIG - PLANNER (1):
GOLDENMATCH_PLANNER_BUCKET - PLANNING (1):
GOLDENMATCH_PLANNING_EFFORT - PREP (1):
GOLDENMATCH_PREP_STAGED_COLLECT - PREPARED (2):
GOLDENMATCH_PREPARED_RECORD_STORE_DIR,GOLDENMATCH_PREPARED_RECORD_STORE_PERSIST - QUALITY (2):
GOLDENMATCH_QUALITY_AWARE_BLOCKING,GOLDENMATCH_QUALITY_GATED_REVIEW - SAIL (1):
GOLDENMATCH_SAIL_IDENTITY_ID_SCHEME - SEMANTIC (1):
GOLDENMATCH_SEMANTIC_BLOCKING_THRESHOLD - SKIP (1):
GOLDENMATCH_SKIP_MATCH_LOG - STANDARDIZE (1):
GOLDENMATCH_STANDARDIZE_STAGED - STREAMING (1):
GOLDENMATCH_STREAMING_BLOCK_WORKERS - SUGGEST (10):
GOLDENMATCH_SUGGEST_COHESION,GOLDENMATCH_SUGGEST_COVERAGE_CAP,GOLDENMATCH_SUGGEST_FULL_DIST,GOLDENMATCH_SUGGEST_HEALTH,GOLDENMATCH_SUGGEST_MAX_VERIFY,GOLDENMATCH_SUGGEST_ON_DEDUPE,GOLDENMATCH_SUGGEST_RECALL_COH_ABS,GOLDENMATCH_SUGGEST_RECALL_MIN_SHED,GOLDENMATCH_SUGGEST_RECALL_RATIO,GOLDENMATCH_SUGGEST_VERIFY - SYNC (1):
GOLDENMATCH_SYNC_STREAMING_THRESHOLD - TF (1):
GOLDENMATCH_TF_NAME_WEIGHTING - THROUGHPUT (3):
GOLDENMATCH_THROUGHPUT,GOLDENMATCH_THROUGHPUT_RECALL,GOLDENMATCH_THROUGHPUT_SIMILARITY - UDF (1):
GOLDENMATCH_UDF_IMPORTS - VECTOR (1):
GOLDENMATCH_VECTOR_INDEX_DIR - WEAKNESS (2):
GOLDENMATCH_WEAKNESS_LLM,GOLDENMATCH_WEAKNESS_LLM_MODEL - WEB (1):
GOLDENMATCH_WEB_TOKEN
See also
- Configuration — YAML field walkthroughs with examples.
- Tuning & opt-ins —
GOLDENMATCH_*defaults, valid values, and effects. - Scoring and Blocking — per-topic detail.
- Config linter — the pre-flight rules that flag risky config.