> ## Documentation Index
> Fetch the complete documentation index at: https://docs.bensevern.dev/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Native accelerator

> An optional Rust kernel for the heavy aggregation primitives, gated only after a measured wall-clock win.

The heavy aggregation primitives — `histogram` and `quantile`, run over the score and cluster-size arrays of large results — have an optional Rust kernel, gated exactly like `goldenmatch[native]` / `goldencheck[native]`.

Those two are the **measured performance wins**. The rest of the aggregation surface is now backed by the same `analysis-core` crate too — the frame-stat kernels (`null_ratio_per_column` / `duplicate_row_ratio` / `distinct_count`), the numeric reductions (`mean` / `min` / `max`), and the discrete `cluster_size_histogram` — but those were cut over for **one source of truth across Python / native / WASM**, not for speed (they're already cheap). Each is parity-gated byte-identical to the pure reference; none is claimed as a perf win.

```bash theme={null}
pip install goldenanalysis[native]   # pulls the separate goldenanalysis-native wheel
```

The pure-Python path stays the **default and the byte-identical reference**. The compiled kernel (`analysis-core`, a pyo3-free crate, plus the `analysis-native` abi3 wheel) mirrors `core/aggregate.py` value-for-value, reading input as a Float64 Arrow array (zero-copy). With the wheel installed, `GOLDENANALYSIS_NATIVE=auto` (the default) uses the native path automatically; `GOLDENANALYSIS_NATIVE=0` forces pure.

## Gate on the measured wall, not "it's Rust"

In reference mode a primitive runs native wherever its kernel symbol is present; `_native_loader._GATED_ON` records which primitives cleared the **two**-gate sign-off: a parity test proves byte-identical output, **and** the wall-clock is measured to actually move on the target environment. The pure loops are tight, and the native path pays a list→Arrow conversion cost, so "it's compiled" is never enough — the goldencheck composite-key kernel was once 2.5x *slower* than its baseline, and the gate caught it.

`histogram` and `quantile` cleared both gates. The A/B harness (`bench-analysis-native.yml`, on Linux x86\_64) measured the **realistic dispatch** — a Python list converted to Arrow, then the kernel, exactly what the call sites pay:

| Primitive   | 1M rows     | 10M rows    |
| ----------- | ----------- | ----------- |
| `histogram` | 9.9x faster | 9.2x faster |
| `quantile`  | 5.8x faster | 6.6x faster |

The Arrow conversion turns out to be far cheaper than the pure-Python loops, so the native path wins decisively even including it.

## How it stays honest

* **Two crates, mirrored.** `analysis-core` (pyo3-free) holds the kernel logic so it can later back a SQL surface; `analysis-native` is the thin abi3 PyO3 + Arrow shim over it.
* **Parity is a CI gate.** A dedicated lane builds the wheel and runs the parity suite under `GOLDENANALYSIS_NATIVE=1`, including end-to-end tests that the public dispatch for **every** covered primitive (histogram, quantile, the frame-stat kernels, mean/min/max, and cluster\_size\_histogram) is byte-identical to the pure path. Analyzers that stay pure on both surfaces are locked by shared cross-surface fixtures instead — the frame-kernel equality semantics (`-0.0`/`NaN`/null), `cluster.distribution`, `quality.rollup`, and the regression decision logic each have a byte-identical Python↔TS fixture.
* **No behavior change without the wheel.** With no native extension installed, `auto` resolves to the pure path; results are identical whether or not native is present.

In-tree dev build (no maturin needed):

```bash theme={null}
uv run python scripts/build_analysis_native.py
```

## TypeScript: opt-in WASM (browser / edge / Node)

The TypeScript port (`goldenanalysis` on npm) reaches the **same `analysis-core` crate** via WebAssembly — the TS analogue of the Python native wheel. Same posture: pure-TS is the default and the byte-identical fallback; WASM is an explicit upgrade for large columns.

```ts theme={null}
import { aggregate, enableAnalysisWasm, disableAnalysisWasm } from "goldenanalysis";

await enableAnalysisWasm();                  // async: loads the .wasm once
const h = aggregate.histogram(values, 256);  // now runs in WASM (Float64 array crosses zero-copy)
const q = aggregate.quantile(values, 0.9);   // the sort runs in Rust
disableAnalysisWasm();                        // back to pure-TS
```

* **Covered ops:** `histogram`, `quantile`, `mean`, `min`, `max`, and `cluster_size_histogram` — the numeric + cluster kernels, each crossing as a `Float64Array` zero-copy (the quantile sort runs in Rust). The frame-stat kernels (`duplicate_row_ratio` / `distinct_count` / `null_ratio`) stay **pure-TS**: they need value-interning that has no clean WASM boundary, so a browser dedup surface is consciously deferred. Everything else in `aggregate` stays pure-TS too.
* **Opt-in, edge-safe, parity-gated** exactly like the scorer WASM backend in `goldenmatch`: pure-TS is the default + fallback, the `.wasm` is built in CI (never committed), and a CI lane asserts WASM ≈ pure-TS to 4 decimals. The plumbing is shared with `goldenmatch` via the `goldenmatch-wasm-runtime` package.
* **Build locally:** `bash packages/rust/extensions/analysis-wasm/build_wasm.sh` (needs the rustup `wasm32-unknown-unknown` target + `wasm-bindgen-cli`).