Pipeline recipes

The config matrix lists every stage. This page is the task-shaped shortcut: a few complete pipelines that wire the suite packages together. Every config below is a complete, valid file (CI validates each against the live schema).

goldenpipe run recipe.yaml

Clean and dedupe end to end

You have a raw file. You want to scan it, standardize it, and dedupe it in one declarative run. Stages run in order; each use names a registered stage (see the Stages table).

pipeline: clean_and_dedupe
source: raw.csv
stages:
  - name: scan
    use: goldencheck.scan
  - name: standardize
    use: goldenflow.transform
  - name: dedupe
    use: goldenmatch.dedupe

Profile, then report

You have a dataset you want a quality read on. You want a scan followed by a cross-cutting report, no matching.

pipeline: profile_and_report
source: data.csv
stages:
  - name: infer
    use: infer_schema
  - name: scan
    use: goldencheck.scan
  - name: report
    use: goldenanalysis.report

Dedupe, then resolve durable identities

You have records that arrive over time. You want to dedupe and then assign stable entity IDs that survive across runs.

pipeline: dedupe_and_resolve
source: contacts.csv
stages:
  - name: standardize
    use: goldenflow.transform
  - name: dedupe
    use: goldenmatch.dedupe
  - name: identities
    use: goldenmatch.identity_resolve

Config matrix GoldenAnalysis overview

Get started

Concepts

GoldenMatch

GoldenCheck

GoldenFlow

GoldenPipe

GoldenAnalysis

InferMap

SQL extensions

Reference

Research

Clean and dedupe end to end

Profile, then report

Dedupe, then resolve durable identities

​Clean and dedupe end to end

​Profile, then report

​Dedupe, then resolve durable identities

Clean and dedupe end to end

Profile, then report

Dedupe, then resolve durable identities