Install
Document ingest is an optional extra (addspymupdf + Pillow for PDF rasterization and
image loading):
gpt-4o). Set an API key — the personal
key is preferred so a work-scoped key is never used by accident:
import goldenmatch.documents and the MCP server both degrade gracefully when the extra or
the key is absent — nothing else in GoldenMatch requires them.
The two-step workflow
You own the target schema. The recommended flow is: let a vision model propose a schema from one representative document, review it, then ingest the whole batch against it.Python
_source_file, _source_page, and
_extract_confidence. Pass return_report=True to ingest_documents to also get an
IngestReport (n_files, n_rows, and per-file errors — a bad file is recorded and the
batch continues rather than aborting).
You can define a schema by hand instead of suggesting one:
kind (text | email | phone | address | date | number) and an optional hint
guide the extractor. Fields absent from a given document come back null.
CLI
run writes .csv or .parquet. Both commands take --backend (default vlm) and
--model (default gpt-4o).
Other surfaces
The same capability is exposed everywhere GoldenMatch runs:
MCP, A2A, and REST are path-based (the caller passes server-accessible file paths or, for
REST/Web, uploads); the Web UI handles browser uploads.
How it works
- VLM-first, one call per page. A single vision call classifies the document and extracts
fields in one shot. The backend is a pluggable
Extractorseam behind an injectable transport, so the stack is fully offline-testable (swap in aFakeExtractor). - Rust kernel = single source of truth. Schema validation, response parsing, prompt
building, and record normalization live in the
documents-coreRust crate, bound to Python (native) and TypeScript (WASM). Pure-Python is a byte-identical fallback. - Tables fan out. A document containing several records (a roster, a directory) yields one row per record; a single-entity document (a card, a form) yields one row.
Notes and limits
- Extraction quality tracks the vision model. On clean documents it is near-perfect and never fabricates a value for a field that is absent; on genuinely hard inputs (unusual proper nouns, heavy scan degradation) it can misread — those errors are the model’s ceiling, not the pipeline’s.
kind: date/kind: numberare advisory hints to the extractor, not post-hoc coercion.- Each page is one vision API call; cost and latency scale with page count. Ingest is parallel-safe across files.