Config matrix

The exhaustive, always-current reference for GoldenFlow configuration: the GoldenFlowConfig schema, the full transform-op vocabulary, the domain packs, and every GOLDENFLOW_* runtime knob. Built for humans and AI agents.

Everything below the line is generated from code (scripts/gen_config_matrix.py) and verified in CI. The guidance above the line is hand-authored. See Overview and CLI for walkthroughs.

How GoldenFlow is configured

A GoldenFlow config (GoldenFlowConfig, loadable from YAML or learned from a sample via learn_config) is a declarative transform plan:

transforms — the core: a list of TransformSpec{column, ops}, where ops is an ordered chain of transform-op names drawn from the Transform ops vocabulary below (e.g. ["strip", "lowercase", "email_normalize"]). The op vocabulary is the real config surface; the model shape is thin.
splits / filters / renames / drop / dedup / mappings — column reshaping around the transform chains.
Domain packs (learn_config(domain=...)) seed sensible op chains for a known domain (see the Domain packs vocabulary).

canonicalize is a first-class op with its own CanonicalizeKind set.

Config object reference

Every config object in the pydantic tree(s), generated from the package schema. Nested objects link by name.

`GoldenFlowConfig`

Field	Type	Default	Choices / notes
`source`	str \| None	`None`	Path or identifier of the input dataset to read records from.
`output`	str \| None	`None`	Path or identifier where the transformed dataset is written.
`transforms`	list[TransformSpec]	`[]`	`TransformSpec` — Per-column transform specs that standardize values by applying ordered ops.
`splits`	list[SplitSpec]	`[]`	`SplitSpec` — Split specs that break one column into several target columns.
`renames`	dict[str, str]	`{}`	Mapping of existing column names to new names to rename them to.
`drop`	list[str]	`[]`	Names of columns to remove from the output.
`filters`	list[FilterSpec]	`[]`	`FilterSpec` — Filter specs that keep only rows satisfying each condition.
`dedup`	DedupSpec \| None	`None`	`DedupSpec` — Optional deduplication spec that collapses duplicate rows by key columns.
`mappings`	list[MappingSpec]	`[]`	`MappingSpec` — Mapping specs that copy source columns to targets with optional transforms.

`TransformSpec`

Field	Type	Default	Choices / notes
`column`	str	required	Name of the column the transform operations are applied to.
`ops`	list[str]	required	Ordered list of transform-op names applied in sequence to the column.

`SplitSpec`

Field	Type	Default	Choices / notes
`source`	str	required	Name of the column whose values are split into multiple columns.
`target`	list[str]	required	Names of the output columns produced by the split.
`method`	str	required	Splitting method that determines how the source value is divided.

`FilterSpec`

Field	Type	Default	Choices / notes
`column`	str	required	Name of the column the filter condition is evaluated against.
`condition`	str	required	Predicate expression that rows must satisfy to be kept.

`DedupSpec`

Field	Type	Default	Choices / notes
`columns`	list[str]	required	Columns whose combined values define what counts as a duplicate row.
`keep`	Literal	`'first'`	`first`, `last` — Which duplicate to retain within each group, either the first or last occurrence.

`MappingSpec`

Field	Type	Default	Choices / notes
`source`	str	required	Name of the source column to read values from.
`target`	str \| list[str]	required	Name or names of the target column(s) the source values are written to.
`transform`	str \| list[str] \| None	`None`	Optional transform op or ordered list of ops applied while mapping source to target.

CLI

Every command and its options/arguments, generated from the Typer app. choice-typed options list their allowed values.

Command	Option	Type	Default	Notes
`agent-serve`	`--port`	integer	`8150`	Port the A2A agent server listens on.
`demo`	`--output-dir`	path	`Path('.')`	Directory where the sample data and config files are written.
`diff`	`before`	path	required	Before file
`diff`	`after`	path	required	After file
`history`	`--limit`	integer	`20`	Number of recent runs to show
`init`	`data`	path	`None`	Data file to profile
`init`	`--output`	path	`'goldenflow.yaml'`	Path where the generated config file is written.
`interactive`	`path`	path	`None`	Input data file
`learn`	`path`	path	required	Input data file
`learn`	`--output`	path	`'goldenflow.yaml'`	Output config path
`map`	`--source`	path	required	Source data file
`map`	`--target`	path	required	Target data file or schema
`map`	`--config`	path	`None`	Mapping config
`map`	`--output`	path	`None`	Save mapping config
`mcp-serve`	`--transport`	text	`'stdio'`	Transport: stdio or http
`mcp-serve`	`--host`	text	`'0.0.0.0'`	Host for HTTP transport
`mcp-serve`	`--port`	integer	`8150`	Port for HTTP transport
`profile`	`path`	path	required	Input data file
`schedule`	`path`	path	required	Data file to transform
`schedule`	`--every`	text	`'1h'`	Interval (e.g., 5m, 1h, 30s)
`schedule`	`--config`	path	`None`	YAML config applied on each scheduled run.
`schedule`	`--output-dir`	path	`None`	Directory where transformed files are written.
`serve`	`--host`	text	`'0.0.0.0'`	Network interface the REST API server binds to.
`serve`	`--port`	integer	`8000`	Port the REST API server listens on.
`stream`	`path`	path	required	Input data file
`stream`	`--chunk-size`	integer	`10000`	Rows per batch
`stream`	`--config`	path	`None`	YAML config applied to each streamed batch.
`stream`	`--output-dir`	path	`None`	Directory where the combined transformed file is written.
`transform`	`path`	path	required	Input data file
`transform`	`--config`	path	`None`	YAML config file
`transform`	`--output-dir`	path	`None`	Output directory
`transform`	`--domain`	text	`None`	Domain pack to use
`transform`	`--from-findings`	boolean	`False`	Read findings from stdin
`transform`	`--llm`	boolean	`False`	Enable LLM-enhanced transforms
`transform`	`--strict`	boolean	`False`	Fail if any transform errors occur
`validate`	`path`	path	required	Input data file
`validate`	`--config`	path	`None`	YAML config file describing which transforms to apply.
`watch`	`path`	path	`'.'`	Directory to watch
`watch`	`--config`	path	`None`	YAML config applied to each new or changed file.
`watch`	`--output-dir`	path	`None`	Directory where transformed files are written.
`watch`	`--interval`	float	`2.0`	Poll interval in seconds

MCP tools

10 MCP tool(s) exposed by goldenflow.mcp.server — the programmatic / agent surface. Config-bearing tools take the same knobs as above.

Tool	Description
`diff`	Compare two data files and show what changed (added, removed, modified rows).
`explain_transform`	Describe what a specific transform does, its mode, and input types.
`learn`	Generate a YAML config from data patterns.
`list_domains`	List available domain packs (e.g., people_hr, ecommerce, finance).
`list_transforms`	List all registered transforms with their modes, input types, and auto-apply status.
`map`	Auto-map schemas between source and target files.
`profile`	Profile a data file showing column types, nulls, and patterns.
`select_from_findings`	Map GoldenCheck findings to recommended GoldenFlow transforms. Bridge tool for Check-to-Flow handoff.
`transform`	Clean / normalize a data file with GoldenFlow transforms (phone, date, unicode, casing, categorical, and more) and write the transformed file. Zero-config auto-detects and applies transforms; pass a config to control them. See list_transforms for what’s available. Does not deduplicate or match — it only reshapes column values.
`validate`	Dry-run transform on a file. Shows what would change without writing output.

Enumerated vocabularies

Allowed values for the str-typed / registry-backed fields above.

Transform ops

list_transforms — TransformSpec.ops.

Value	Meaning
`aba_validate`	Validate a US ABA bank routing number: exactly 9 digits plus the
`abs_value`	Return the absolute value.
`account_mask`	Mask account numbers showing only last 4 digits.
`address_expand`	Replace street abbreviations (St, Ave…) with full forms.
`address_standardize`	Replace full street suffixes (Street, Avenue…) with abbreviations.
`age_from_dob`	Compute age in years from a date of birth.
`boolean_normalize`	Parse loose boolean-ish strings (yes/no/y/n/1/0/true/false/t/f).
`carceral_abbreviate`	Expand carceral facility-type abbreviations and state-complex aliases.
`carceral_name_normalize`	Full carceral name pipeline: org-strip + uppercase + punctuation strip
`carceral_org_strip`	Strip leading operator-org prefix from a carceral facility name.
`category_auto_correct`	Auto-correct categorical misspellings and case variants.
`category_from_file`	Load mapping from a CSV/YAML file and standardize values.
`category_llm_correct`	LLM-enhanced categorical correction.
`category_standardize`	Map variant values to canonical values. mapping: {canonical: [variant1, variant2, …]}
`cc_brand`	Detect the card brand (visa/mastercard/amex/…) or null. Native-first.
`cc_format`	Group a valid payment-card number (Amex 4-6-5, else 4-4-4-4…);
`cc_mask`	Mask a payment-card number to stars + last 4 digits.
`cc_validate`	Validate a payment-card number via the Luhn checksum.
`clamp`	Clip values into [min_val, max_val].
`collapse_whitespace`	Collapse runs of whitespace to single spaces.
`comma_decimal`	Convert European decimal format (1.234,56) to float (1234.56).
`company_extract_legal`	Extract the canonical legal-form token (`inc`/`llc`/…) or null.
`company_normalize`	Composite company dedup key: lowercase, drop leading ‘the’, strip legal
`company_strip_legal`	Strip trailing legal-form suffixes, preserving the core name’s case.
`country_standardize`	Normalize country names to ISO 3166-1 alpha-2 codes.
`currency_strip`	Strip currency symbols and thousand separators, return numeric.
`cusip_format`	Standardize CUSIP identifiers (9 chars, uppercase).
`cusip_validate`	Validate a CUSIP. Native-first over goldenflow-core.
`date_eu`	Parse a European (DD/MM/YYYY) date.
`date_iso8601`	Parse/format a date as ISO-8601.
`date_parse`	Auto-detect format and normalize to ISO 8601.
`date_shift`	Shift dates by a number of days (positive = forward, negative = backward).
`date_us`	Parse a US (MM/DD/YYYY) date.
`date_validate`	Validate if value is a parseable date. Returns True/False/None.
`datetime_iso8601`	Parse to ISO 8601 datetime (with time component).
`double_metaphone_alt`	Alternate Double Metaphone code. Native-first.
`double_metaphone_primary`	Primary Double Metaphone code (blocking key). Native-first.
`ean_validate`	Validate an EAN-8, UPC-A, or EAN-13 via its GTIN mod-10 checksum.
`ein_format`	Normalize EIN to XX-XXXXXXX format. Native-first.
`email_canonical`	Full dedup key: email_normalize + alias googlemail.com -> gmail.com so
`email_extract_domain`	Extract the lowercased domain from an email address.
`email_lowercase`	Lowercase the entire email address (trim + lowercase).
`email_mask`	PII mask: keep the first local char, star the rest, keep @domain
`email_normalize`	Normalize email: lowercase, strip +tags, strip dots from Gmail local part.
`email_validate`	Validate email format. Returns True/False/None.
`extract_day`	Extract the day of month as an integer (1-31).
`extract_day_of_week`	Extract the day of week name (Monday, Tuesday, etc.).
`extract_month`	Extract the month as an integer (1-12).
`extract_numbers`	Extract all numbers from text, joined by spaces.
`extract_quarter`	Extract the quarter (1-4) from a date.
`extract_year`	Extract the year as an integer.
`fill_zero`	Replace null values with 0.
`fix_mojibake`	Fix common UTF-8/Latin-1 mojibake by re-encoding.
`fraction_to_decimal`	Parse a fraction or mixed number (1/2, 3 3/4) to a float. Native-first.
`gender_standardize`	Standardize gender strings to `M`/`F`; anything else passes
`iban_format`	Group a valid IBAN into 4-char blocks; `null` for invalid input.
`iban_validate`	Validate an IBAN via structural checks + the ISO 7064 mod-97 check.
`icd10_format`	Standardize ICD-10 codes (uppercase, insert dot after 3rd char).
`imei_validate`	Validate an IMEI: exactly 15 digits plus the Luhn checksum.
`initial_expand`	Returns (series, flagged_rows). Values with initials are unchanged but flagged.
`isbn_normalize`	Canonicalize a valid ISBN-10/13 to its 13-digit form; `null` for
`isbn_validate`	Validate an ISBN-10 or ISBN-13 via its checksum.
`isin_validate`	Validate an ISIN (ISO 6166). Native-first over goldenflow-core.
`latlng_pack`	Pack `lat` + `lng` into a single `latlng` column shaped
`lowercase`	Lowercase the value.
`luhn_validate`	Generic Luhn check-digit validation. Native-first over goldenflow-core.
`merge_name`	Merge first_name and last_name columns into a full_name column.
`mls_normalize`	Normalize MLS listing IDs (uppercase, strip whitespace).
`name_initials`	Initials of each whitespace token (letter-leading tokens only).
`name_proper`	Proper-case a name (title-case + Mc/O’ fixups).
`name_script`	Detect the dominant Unicode script in a name: `Unknown` for empty
`name_transliterate`	ASCII-fold a name via an explicit curated diacritic map. Non-ASCII
`nickname_standardize`	Map common nicknames to formal first names.
`normalize_line_endings`	Normalize \r\n and \r to \n.
`normalize_quotes`	Replace smart/curly quotes with straight quotes.
`normalize_unicode`	Normalize to Unicode NFC form.
`npi_validate`	Validate a US NPI. Native-first over goldenflow-core.
`null_standardize`	Map null-sentinel strings (n/a, null, none, na, nil, nan, -, empty) to
`ordinal_to_int`	Parse an English ordinal (1st/2nd/3rd) to its integer. Native-first.
`pad_left`	Left-pad strings to a fixed width.
`pad_right`	Right-pad strings to a fixed width.
`percentage_normalize`	Strip trailing %, parse to float, divide by 100.
`phone_country_code`	Extract the country calling code as an integer.
`phone_digits`	Keep only the phone number’s digits.
`phone_e164`	Format a phone number as E.164.
`phone_national`	Format a phone number in national format.
`phone_validate`	Flag whether the phone number is valid.
`remove_digits`	Remove all digit characters from text.
`remove_emojis`	Remove emoji characters from text.
`remove_html_tags`	Strip HTML tags from text.
`remove_punctuation`	Strip punctuation characters.
`remove_urls`	Strip URLs (http/https) from text.
`roman_to_int`	Parse a Roman numeral to its integer (canonical forms only, 1..=3999).
`round`	Round to n decimal places (round-half-away-from-zero, see the
`scientific_to_decimal`	Convert scientific notation (1.5e3) to decimal (1500.0).
`sku_normalize`	Normalize SKU identifiers (uppercase, strip whitespace, remove special chars).
`soundex`	Soundex phonetic key. Native-first over goldenflow-core.
`split_address`	Parse ‘street, city, state zip’ into separate columns.
`split_name`	Split ‘First Last’ into first_name and last_name columns.
`split_name_reverse`	Split ‘Last, First’ into first_name and last_name columns.
`ssn_format`	Normalize SSN to XXX-XX-XXXX format. Native-first over goldenflow-core.
`ssn_mask`	Mask SSN to *--XXXX (last 4 visible). Native-first.
`ssn_validate`	Flag whether the value is a valid US Social Security Number.
`state_abbreviate`	Normalize state name to a 2-letter abbreviation (unmatched -> original).
`state_expand`	Expand a 2-letter state abbreviation to its full name (unmatched -> original).
`strip`	Trim leading/trailing whitespace.
`strip_middle`	Keep only the first and last whitespace tokens (drop the middle).
`strip_suffixes`	Strip trailing professional suffixes (Jr/Sr/II/MD/PhD/etc.) from names.
`strip_titles`	Strip leading personal titles (Mr/Mrs/Ms/Dr/Prof/etc.) from names.
`swift_format`	Normalize a valid SWIFT/BIC code to uppercase (spaces stripped);
`swift_validate`	Validate a SWIFT/BIC code via structural checks (length 8/11 +
`title_case`	Title-case the value.
`to_integer`	Parse string to integer, truncating any decimal part.
`truncate`	Truncate string to the first n characters.
`unit_normalize`	Normalize unit/apartment/suite designations.
`uppercase`	Uppercase the value.
`url_canonical`	Composite dedup key: ensure scheme, lowercase scheme+host, strip www.,
`url_extract_domain`	Extract domain from a URL.
`url_normalize`	Normalize URLs: ensure scheme, lowercase domain, strip trailing slash.
`url_strip_tracking`	Remove tracking query params (utm_*, gclid, fbclid, …), preserving the
`url_strip_www`	Strip a leading `www.` label from the host, preserving scheme, path,
`vat_format`	Normalize a valid EU VAT number to its compact uppercase form (prefix
`vat_validate`	Validate an EU VAT number: structural check (country prefix + length +
`zip_normalize`	Normalize a US ZIP to 5-digit form (strip +4, zero-pad all-digit, preserve

Domain packs

_DOMAINS — learn_config(domain=...).

Value	Meaning
`carceral`	Corrections / justice field pack.
`ecommerce`	E-commerce / retail field pack.
`finance`	Financial-services field pack.
`healthcare`	Healthcare field pack.
`people_hr`	People / HR field pack.
`real_estate`	Real-estate field pack.

Canonicalize kinds

CanonicalizeKind — canonicalize op.

Value	Meaning
`email`	Canonical email form.
`name`	Canonical person-name form.
`phone`	Canonical phone form.
`postal`	Canonical postal-address form.

Environment variables (index)

5 GOLDENFLOW_* runtime knob(s) read by the package, scanned from source so this is complete. Grouped by area:

ENGINE (1): GOLDENFLOW_ENGINE
FUSED (1): GOLDENFLOW_FUSED_APPLY
LLM (1): GOLDENFLOW_LLM
NATIVE (2): GOLDENFLOW_NATIVE, GOLDENFLOW_NATIVE_CSV_PARALLEL_MIN_BYTES

Get started

Concepts

GoldenMatch

GoldenCheck

GoldenFlow

GoldenPipe

GoldenAnalysis

InferMap

SQL extensions

Reference

Research

How GoldenFlow is configured

Config object reference

`GoldenFlowConfig`

`TransformSpec`

`SplitSpec`

`FilterSpec`

`DedupSpec`

`MappingSpec`

CLI

MCP tools

Enumerated vocabularies

Transform ops

Domain packs

Canonicalize kinds

Environment variables (index)

See also

​How GoldenFlow is configured

​Config object reference

​GoldenFlowConfig

​TransformSpec

​SplitSpec

​FilterSpec

​DedupSpec

​MappingSpec

​CLI

​MCP tools

​Enumerated vocabularies

​Transform ops

​Domain packs

​Canonicalize kinds

​Environment variables (index)

​See also

How GoldenFlow is configured

Config object reference

`GoldenFlowConfig`

`TransformSpec`

`SplitSpec`

`FilterSpec`

`DedupSpec`

`MappingSpec`

CLI

MCP tools

Enumerated vocabularies

Transform ops

Domain packs

Canonicalize kinds

Environment variables (index)

See also