Skip to main content
This is the exhaustive reference for every GoldenMatch config knob: the declarative schema (GoldenMatchConfig and every nested object), the enumerated string vocabularies, and the GOLDENMATCH_* runtime environment knobs. It is built for both humans and AI agents that need a single, complete, always-current map of what is configurable.
The object reference, vocabularies, and env index below the line are generated from code (scripts/gen_config_matrix.py) and verified in CI, so they can never silently drift from the engine. The combinations & outcomes section above the line is hand-authored guidance. For prose walkthroughs see Configuration (YAML fields), Scoring, Blocking, and Tuning & opt-ins (env-var semantics + defaults).

How to read this page

  • Combinations & outcomes (below) is the decision layer: which knob to reach for, and what happens when knobs interact.
  • Config object reference is the structural layer: every field, its type, its default, and its allowed Literal values, per config object.
  • Enumerated vocabularies lists the allowed values for the str-typed fields (scorers, strategies, standardizers).
  • Environment variables indexes every GOLDENMATCH_* knob by area; defaults and effects are in Tuning & opt-ins.

Combinations & intended outcomes

Matchkey type: exact vs weighted vs probabilistic

The single most consequential choice. Set per matchkey via MatchkeyConfig.type.

Missing values x scorer/type

MatchkeyConfig.missing (unobserved vs disagree) is probabilistic-only.
  • Auto-config picks the mode from profiled null rates (disagree when the worst comparison field is >= 20% null); override with MatchkeyConfig.missing or the global GOLDENMATCH_FS_MISSING.
  • Native kernel caveat: the native FS kernel implements only unobserved; a disagree-mode matchkey routes to the numpy path automatically (correctness over speed).
  • On weighted matchkeys there is no missing knob — a null field is simply dropped from the weighted mean (renormalized by observed weight).

Blocking strategy x data scale

BlockingConfig.strategy selects candidate generation. Any weighted/probabilistic matchkey requires a blocking config. A null block key means “cannot be blocked”, never “blocks with every other null-key row” — null keys are filtered across every strategy (no false mega-blocks). Guardrails regardless of strategy: max_block_size + skip_oversized, max_total_comparisons, and auto_suggest / auto_select to let auto-config choose keys/strategy.

Execution routing: in-memory vs bucket vs distributed vs scale-mode

Left mostly to the controller, but pinnable. Env-level routing (see the env index): GOLDENMATCH_BUCKET_DEFAULT (the default in-memory bucket FS route up to ~750K rows), GOLDENMATCH_MATCH_FUSED (fused block->score->cluster kill-switch; fires under RSS pressure), GOLDENMATCH_NATIVE / GOLDENMATCH_FRAME (native kernel + arrow/polars frame seam). The controller refuses a low-confidence (“RED”) auto-config at >= 100K rows unless allow_red_config is set. Not an enum — it is expressed by InputConfig: a single files list = dedupe (find duplicates within one dataset); file_a + file_b = link (match across two datasets, cross-source pairs only).

Survivorship: field rules, groups, conditional, correlated

golden_rules decides the surviving value per field after clustering.
  • field_rules map a column to a strategy (see the survivorship vocabulary). List form = conditional rules (when: predicates over already-resolved fields, first match wins, one when-less default last).
  • field_groups pin >= 2 columns lock-step to one winning row (e.g. city/state/zip must come from the same record) via a group strategy (most_complete / source_priority / most_recent / anchor).
  • Correlated / conditional / validated survivorship runs in-memory only — it is refused on the distributed/Sail paths.
  • quality_weighting, weak_cluster_threshold, auto_split + max_cluster_size control how weak/oversized clusters are downgraded or split before survivorship.

Multi-table / collective ER: propagation_mode

For graph/relational ER across entity types (graph.propagation_mode):

Native / frame kill-switches

Defaults are fast: arrow frames + native kernel where available. GOLDENMATCH_NATIVE=auto falls back to pure Python silently if the compiled wheel is absent; =1 requires it (raises), =0 forces Python. GOLDENMATCH_FRAME=polars restores the Polars frame seam (needs the [polars] extra). Reach for these only to reproduce a result across environments or to isolate a perf/parity question.

Config object reference

Every config object in the pydantic tree(s), generated from the package schema. Nested objects link by name.

GoldenMatchConfig

InputConfig

OutputConfig

MatchSettingsConfig

MatchkeyConfig

A matchkey: rule for declaring two records ‘the same’ on a field/field-set.

BlockingConfig

GoldenRulesConfig

StandardizationConfig

ValidationConfig

QualityConfig

GoldenCheck integration config for enhanced data quality.

TransformConfig

GoldenFlow integration config for data transformation.

LLMScorerConfig

DomainConfig

DistributedRoutingConfig

Per-stage distributed-routing pins. auto lets the planner decide;

SemanticBlockingConfig

Opt-in semantic candidate-generation (recall-lever) config. Carries the

ThroughputConfig

Opt-in sketch-then-verify throughput tier (#1083).

MemoryConfig

Learning Memory configuration.

IdentityConfig

Identity Graph configuration.

InputFileConfig

MatchkeyField

NegativeEvidenceField

v1.11: a field whose disagreement subtracts from a weighted matchkey’s

BlockingKeyConfig

SortKeyField

CanopyConfig

LSHKeyConfig

MinHash/LSH blocking on a text column (#1081).

SimHashKeyConfig

SimHash/LSH blocking on a text column via dense embeddings (#1082).

PerceptualKeyConfig

Banded hamming-LSH blocking over a column of perceptual hashes (ADR 0022).

GoldenFieldRule

GoldenGroupRule

ValidationRuleConfig

BudgetConfig

LearningConfig

Learning Memory learning parameters.

ChannelStitchConfig

Cross-device / channel stitching configuration (#1110, epic #1108).

SurvivorshipConfig

Golden-record survivorship configuration (#1111, epic #1108).

StabilizationConfig

Cross-run entity stabilization — Identity v3 (#1112, epic #1108).

MediationConfig

Conflict mediation workflow — Identity v3 (#1113, epic #1108).

CLI

Every command and its options/arguments, generated from the Typer app. choice-typed options list their allowed values.

MCP tools

82 MCP tool(s) exposed by goldenmatch.mcp.server — the programmatic / agent surface. Config-bearing tools take the same knobs as above.

Enumerated vocabularies

Allowed values for the str-typed / registry-backed fields above.

Scorers

VALID_SCORERSMatchkeyField.scorer / NegativeEvidenceField.scorer.

Blocking strategies

BlockingConfig.strategyBlockingConfig.strategy.

Simple transforms

VALID_SIMPLE_TRANSFORMStransforms chains.

Survivorship strategies

VALID_STRATEGIESGoldenFieldRule.strategy.

Group survivorship strategies

_GROUP_STRATEGIESGoldenGroupRule.strategy.

Standardizers

VALID_STANDARDIZERSStandardizationConfig.rules.

Matchkey types

_VALID_MK_TYPESMatchkeyConfig.type.

Backends

BackendNameGoldenMatchConfig.backend / --backend.

Clustering strategies

ClusteringStrategy — planner cluster route.

Planning efforts

_PLANNING_EFFORTSplanning_effort.

Environment variables (index)

166 GOLDENMATCH_* runtime knob(s) read by the package, scanned from source so this is complete. Details in Tuning & opt-ins. Grouped by area:
  • AGENT (1): GOLDENMATCH_AGENT_TOKEN
  • ALLOWED (1): GOLDENMATCH_ALLOWED_ROOT
  • ANALYTICS (1): GOLDENMATCH_ANALYTICS
  • ANN (6): GOLDENMATCH_ANN_BACKEND, GOLDENMATCH_ANN_HNSW_EF_CONSTRUCTION, GOLDENMATCH_ANN_HNSW_EF_SEARCH, GOLDENMATCH_ANN_HNSW_M, GOLDENMATCH_ANN_HNSW_MAX_K, GOLDENMATCH_ANN_HNSW_MIN
  • API (2): GOLDENMATCH_API_CORS_ORIGINS, GOLDENMATCH_API_TOKEN
  • ATTRIBUTE (1): GOLDENMATCH_ATTRIBUTE_DEMOTION
  • AUTO (1): GOLDENMATCH_AUTO_SEMANTIC_BLOCKING
  • AUTOCONFIG (10): GOLDENMATCH_AUTOCONFIG_ARROW_NATIVE, GOLDENMATCH_AUTOCONFIG_FORCE_EXCLUDE, GOLDENMATCH_AUTOCONFIG_FORCE_INCLUDE, GOLDENMATCH_AUTOCONFIG_INDICATOR_BUDGET, GOLDENMATCH_AUTOCONFIG_LLM, GOLDENMATCH_AUTOCONFIG_MEMORY, GOLDENMATCH_AUTOCONFIG_ROUTE_PROBABILISTIC, GOLDENMATCH_AUTOCONFIG_SAMPLE_STRATEGY, GOLDENMATCH_AUTOCONFIG_ZERO_LABEL_COMMIT, GOLDENMATCH_AUTOCONFIG_ZERO_LABEL_STABILITY
  • BASE (1): GOLDENMATCH_BASE_STORE
  • BENCH (1): GOLDENMATCH_BENCH_DUMP_PAIRS
  • BLOCKING (8): GOLDENMATCH_BLOCKING_CARDINALITY_SCALER, GOLDENMATCH_BLOCKING_DEGENERATE_MAX_AVG_BLOCK_SIZE, GOLDENMATCH_BLOCKING_DEGENERATE_THRESHOLD, GOLDENMATCH_BLOCKING_MAX_RATIO, GOLDENMATCH_BLOCKING_MIN_RATIO, GOLDENMATCH_BLOCKING_PAIRS_PER_ROW, GOLDENMATCH_BLOCKING_PASS_MIN_WEAKPOS, GOLDENMATCH_BLOCKING_PRUNE_PASSES
  • BRIDGE (1): GOLDENMATCH_BRIDGE_REQUIRE_PY
  • BUCKET (5): GOLDENMATCH_BUCKET_DEBUG, GOLDENMATCH_BUCKET_DEFAULT, GOLDENMATCH_BUCKET_SLIM_PROJECTION, GOLDENMATCH_BUCKET_VEC_MAX, GOLDENMATCH_BUCKET_VEC_MIN
  • CLUSTER (1): GOLDENMATCH_CLUSTER_SPLIT_EDGE_BUDGET
  • COLUMNAR (1): GOLDENMATCH_COLUMNAR_PIPELINE
  • CONFIG (1): GOLDENMATCH_CONFIG_LINT
  • DATABASE (1): GOLDENMATCH_DATABASE_URL
  • DISCRIMINATIVE (2): GOLDENMATCH_DISCRIMINATIVE_TAU, GOLDENMATCH_DISCRIMINATIVE_VETO
  • DISTRIBUTED (13): GOLDENMATCH_DISTRIBUTED_BLOCK_SHUFFLE, GOLDENMATCH_DISTRIBUTED_CLUSTERING_THRESHOLD, GOLDENMATCH_DISTRIBUTED_FS_TRAIN_ROWS, GOLDENMATCH_DISTRIBUTED_GOLDEN_THRESHOLD, GOLDENMATCH_DISTRIBUTED_OP_RESERVATION, GOLDENMATCH_DISTRIBUTED_PIPELINE, GOLDENMATCH_DISTRIBUTED_SCORE_CONCURRENCY, GOLDENMATCH_DISTRIBUTED_SCORE_NUM_CPUS, GOLDENMATCH_DISTRIBUTED_SCORE_PROJECT, GOLDENMATCH_DISTRIBUTED_SHUFFLE_PARTS, GOLDENMATCH_DISTRIBUTED_WCC, GOLDENMATCH_DISTRIBUTED_WCC_SCRATCH, GOLDENMATCH_DISTRIBUTED_WCC_SEED
  • DUCKDB (1): GOLDENMATCH_DUCKDB_SCORE_DB
  • EMBEDDING (1): GOLDENMATCH_EMBEDDING_PROVIDER
  • ENABLE (1): GOLDENMATCH_ENABLE_DISTRIBUTED_RAY
  • ENSEMBLE (1): GOLDENMATCH_ENSEMBLE_KERNEL
  • FACILITY (1): GOLDENMATCH_FACILITY_NAME_NE
  • FD (1): GOLDENMATCH_FD_NEGATIVE_EVIDENCE
  • FIELD (1): GOLDENMATCH_FIELD_GROUP_SURVIVORSHIP
  • FRAME (2): GOLDENMATCH_FRAME, GOLDENMATCH_FRAME_LANE
  • FS (28): GOLDENMATCH_FS_ARROW_STREAM, GOLDENMATCH_FS_AUTOCONFIG_V2, GOLDENMATCH_FS_BATCH_ROWS, GOLDENMATCH_FS_BLOCK_SOURCE, GOLDENMATCH_FS_BUCKET_NATIVE, GOLDENMATCH_FS_CALIBRATED, GOLDENMATCH_FS_COLUMNAR_CLUSTER, GOLDENMATCH_FS_DEFAULT_BUCKET, GOLDENMATCH_FS_DOMAIN_COMPARATORS, GOLDENMATCH_FS_EM_AGG_BLOCKS, GOLDENMATCH_FS_EM_BLOCK_SLIM, GOLDENMATCH_FS_EM_SAMPLE_ROWS, GOLDENMATCH_FS_MAX_PASS_PAIRS, GOLDENMATCH_FS_MISSING, GOLDENMATCH_FS_MONOTONIC, GOLDENMATCH_FS_NATIVE, GOLDENMATCH_FS_OOC_ARROW_CLUSTER, GOLDENMATCH_FS_OOC_DEBUG, GOLDENMATCH_FS_OOC_WAVE_ROWS, GOLDENMATCH_FS_OUT_OF_CORE, GOLDENMATCH_FS_REQUIRE_POSITIVE_EVIDENCE, GOLDENMATCH_FS_ROUTE_MIN_ROWS, GOLDENMATCH_FS_SCORED_PAIRS_MAX, GOLDENMATCH_FS_STRIP_HONORIFICS, GOLDENMATCH_FS_TF_ADJUSTMENT, GOLDENMATCH_FS_VECTORIZED, GOLDENMATCH_FS_VEC_MAX_ELEMS, GOLDENMATCH_FS_WORKERS
  • FUSED (5): GOLDENMATCH_FUSED_BLOCK_CONCURRENCY, GOLDENMATCH_FUSED_BYTES_PER_CELL, GOLDENMATCH_FUSED_BYTES_PER_PAIR, GOLDENMATCH_FUSED_PRESSURE_FRACTION, GOLDENMATCH_FUSED_RSS_SCALE
  • GOLDEN (4): GOLDENMATCH_GOLDEN_FUSED, GOLDENMATCH_GOLDEN_SLIM_MULTIDF, GOLDENMATCH_GOLDEN_STRATEGY_STRICT, GOLDENMATCH_GOLDEN_TUNER_MIN_CORRECTIONS
  • GPU (3): GOLDENMATCH_GPU_API_KEY, GOLDENMATCH_GPU_ENDPOINT, GOLDENMATCH_GPU_MODE
  • HEAL (2): GOLDENMATCH_HEAL_MIN_HEALTH_GAIN, GOLDENMATCH_HEAL_STEP_CAP
  • IDENTITY (3): GOLDENMATCH_IDENTITY_BATCH_FINGERPRINT, GOLDENMATCH_IDENTITY_DSN, GOLDENMATCH_IDENTITY_WRITE_PIPELINE
  • INHOUSE (1): GOLDENMATCH_INHOUSE_MODEL
  • LLAMA (1): GOLDENMATCH_LLAMA_GGUF
  • LLM (2): GOLDENMATCH_LLM_BASE_URL, GOLDENMATCH_LLM_MODEL
  • MATCH (2): GOLDENMATCH_MATCH_FUSED, GOLDENMATCH_MATCH_LOG_FLUSH_PAIRS
  • MCP (6): GOLDENMATCH_MCP_ALLOW_PUBLIC, GOLDENMATCH_MCP_MAX_UPLOAD_BYTES, GOLDENMATCH_MCP_SESSION_MAX, GOLDENMATCH_MCP_SESSION_TTL, GOLDENMATCH_MCP_TOKEN, GOLDENMATCH_MCP_UPLOAD_TTL
  • MULTISOURCE (1): GOLDENMATCH_MULTISOURCE_AUTOCONFIG
  • NATIVE (5): GOLDENMATCH_NATIVE, GOLDENMATCH_NATIVE_ADDRESS_NORMALIZE, GOLDENMATCH_NATIVE_RAYON_MIN_BLOOM_ROWS, GOLDENMATCH_NATIVE_RAYON_MIN_PAIRS, GOLDENMATCH_NATIVE_SKETCH_RAYON_MIN_ROWS
  • NE (1): GOLDENMATCH_NE_TUNER_MIN_CORRECTIONS
  • NOISE (2): GOLDENMATCH_NOISE_AWARE_SCORERS, GOLDENMATCH_NOISE_AWARE_TARGET
  • PERCEPTUAL (1): GOLDENMATCH_PERCEPTUAL_AUTOCONFIG
  • PLANNER (1): GOLDENMATCH_PLANNER_BUCKET
  • PLANNING (1): GOLDENMATCH_PLANNING_EFFORT
  • PREP (1): GOLDENMATCH_PREP_STAGED_COLLECT
  • PREPARED (2): GOLDENMATCH_PREPARED_RECORD_STORE_DIR, GOLDENMATCH_PREPARED_RECORD_STORE_PERSIST
  • QUALITY (2): GOLDENMATCH_QUALITY_AWARE_BLOCKING, GOLDENMATCH_QUALITY_GATED_REVIEW
  • SAIL (1): GOLDENMATCH_SAIL_IDENTITY_ID_SCHEME
  • SEMANTIC (1): GOLDENMATCH_SEMANTIC_BLOCKING_THRESHOLD
  • SKIP (1): GOLDENMATCH_SKIP_MATCH_LOG
  • STANDARDIZE (1): GOLDENMATCH_STANDARDIZE_STAGED
  • STREAMING (1): GOLDENMATCH_STREAMING_BLOCK_WORKERS
  • SUGGEST (10): GOLDENMATCH_SUGGEST_COHESION, GOLDENMATCH_SUGGEST_COVERAGE_CAP, GOLDENMATCH_SUGGEST_FULL_DIST, GOLDENMATCH_SUGGEST_HEALTH, GOLDENMATCH_SUGGEST_MAX_VERIFY, GOLDENMATCH_SUGGEST_ON_DEDUPE, GOLDENMATCH_SUGGEST_RECALL_COH_ABS, GOLDENMATCH_SUGGEST_RECALL_MIN_SHED, GOLDENMATCH_SUGGEST_RECALL_RATIO, GOLDENMATCH_SUGGEST_VERIFY
  • SYNC (1): GOLDENMATCH_SYNC_STREAMING_THRESHOLD
  • TF (1): GOLDENMATCH_TF_NAME_WEIGHTING
  • THROUGHPUT (3): GOLDENMATCH_THROUGHPUT, GOLDENMATCH_THROUGHPUT_RECALL, GOLDENMATCH_THROUGHPUT_SIMILARITY
  • UDF (1): GOLDENMATCH_UDF_IMPORTS
  • VECTOR (1): GOLDENMATCH_VECTOR_INDEX_DIR
  • WEAKNESS (2): GOLDENMATCH_WEAKNESS_LLM, GOLDENMATCH_WEAKNESS_LLM_MODEL
  • WEB (1): GOLDENMATCH_WEB_TOKEN

See also