Skip to main content

AgentSession.autoconfigure() — run controller; return config + telemetry

dedupe_df() — DataFrame deduplication

DedupeResult fields

StandardizationConfig - use rules dict, NOT keyword args

StandardizationConfig has a single rules: dict[str, list[str]] field with a model validator. Keyword args will raise a Pydantic validation error.

BlockingConfig multi_pass requires keys or passes

MatchkeyConfig requires name field

Extracting pairs from clusters (correct way)

Multi-pass blocking for catching different dupe types

Available scorers

  • exact: 1.0 if equal, 0.0 otherwise
  • jaro_winkler: best for short strings (names)
  • levenshtein: normalized edit distance
  • token_sort: handles word reordering
  • ensemble: weighted combination of jaro_winkler + levenshtein + token_sort + dice (best for names)
  • dice, jaccard: set-based similarity
  • soundex_match: phonetic matching
  • embedding: sentence-transformer cosine similarity

Available transforms (applied at matchkey time)

lowercase, uppercase, strip, strip_all, soundex, metaphone, digits_only, alpha_only, normalize_whitespace, token_sort, first_token, last_token, substring:start:end, qgram:n

LLM Scorer for borderline pairs

Common Mistakes

  • Using exact=["email"] as sole matchkey - creates oversized clusters with common emails
  • Using auto_configure() on synthetic data - it may produce poor configs
  • Not setting name= on MatchkeyConfig - it’s required
  • Not providing keys= or passes= on a multi_pass BlockingConfig - it needs at least one of them (keys OR passes)
  • Extracting pairs from dupes DataFrame directly instead of using result.clusters