Serpaix benchmarks: measured, dated, reproducible

Median end-to-end wall time for one approximate-nearest-neighbour query, by candidate count and corpus geometry, on the shipped production executable with the natural delivery path. 10,000-row corpus, median of two iterations per cell.

Milliseconds. A stress leg at 5,000 candidates measured 1,381.2 ms. Growth is close to linear at roughly 0.25–0.28 ms per candidate and is almost geometry-independent — the signature of a transport-bound path, since scoring and parsing together account for about 13 ms of a 675 ms query.

Component timers from the same production run: 6,000-row cleanroom corpus, eight queries, k = 80. End-to-end median was 593.3 ms (p95 603 ms), reproduced to ±1 ms across runs.

A benchmark-harness stringify of 12.7 ms is excluded above because it is not production cost; the true production figure is therefore nearer 580 ms.

Inside the 471 ms vector fetch

The dominant cost is moving one ~14.3 MB payload of double-encoded text vectors per query, at roughly 35 MB/s across the process boundary. What is not the problem: the IPC round-trip floor (2.2 ms × 10 calls = 22 ms, 3.7 %), the JSON parse (5.9 ms) and cosine scoring (0.15 ms) together stay under 5 % of end-to-end.

The Tauri configuration’s connect-src directive did not allow ipc: http://ipc.localhost, so the framework’s fast custom-protocol path was rejected by the content security policy on every call. The runtime then fell back to window.ipc.postMessage plus eval(JSON). Nothing errored visibly; search was simply slower than it needed to be.

It was proven with a timing-independent probe: fetch('http://ipc.localhost/__vt_probe__') was rejected with Failed to fetch, two CSP violations were recorded against ipc.localhost, and the detector reported the delivery path as postMessage-fallback.

This is the class of bug that instrumented benchmarks exist to catch, and it is the reason the numbers in section 1 are what they are. Per-query cost is 14,339,384 bytes received against 36,294 bytes sent, across 10 IPC calls, for a single search.

A Rust-side component split over the same database and the same rows, per 400-row IPC chunk, comparing today’s stored text (serialised as a JSON string inside the IPC body) against a versioned binary f32 wire:

With the honest counterweight: storage stays as text, so the binary command must decode it to emit floats — a cost of about 54.9 ms per chunk that the text path never pays. Whether the smaller payload wins overall is decided by the end-to-end comparison, which is in the pending section below rather than asserted here.

An isolated Rust plane benchmark doing the identical work — fetch, decode, serialise, score and rank 2,500 rows of 384 dimensions — with vectors stored as text versus binary f32. Release build, bundled SQLite.

The speed is only interesting because the ranking is unchanged. In the same run, decoded values were identical, the top-80 ordering was identical, and the maximum score delta was 0.000e0. Separately, a Rust f64 scorer was checked against the shipped TypeScript scorer: ranking order and result set matched in 20/20 cases across all eight test blocks with a maximum delta of zero. A f32 binary scorer kept top-1 identical in 160/160 cases; its only deviations were swaps inside a tie band roughly two orders of magnitude tighter than the 1×10⁻⁵ tolerance it is held to.

While measuring speed we also measured quality, on a 50,000-row corpus across four geometries. The truthful summary is that the index is perfect and the candidate set was mostly noise:

The mechanism: a radius-1 probe returns roughly 157 rows, which is below the 240-row threshold that triggers a recency-based top-up. That top-up then fills almost the entire candidate set with the most recently updated rows in the database — rows unrelated to the query — and the similarity floor afterwards discards the genuine neighbours that did arrive, because their true similarity sits below it. On dense corpora, where the probe ball is large enough, the same pipeline reaches 54.0 % candidate recall. The conclusion recorded in the report is a structural one: inside the current parameter space, no tuning fixes non-dense recall.

These numbers come from an end-to-end run of the offline product against a realistic 65-file (0.53 MiB) business corpus with 18 labelled questions, with inference on the CPU only.

The practical reading: a small local model answers in half a minute on this class of machine, and a 3-billion-parameter model does not answer at all inside the client’s own timeout. That is why the product lets you choose the model rather than picking one for you, and why “runs on your machine” is a claim we only make after measuring the machine in question.

What we got wrong

These were our priors before instrumenting, and what the measurements did to them. They are recorded here because the value of first-party data is mostly in the corrections.

Listed so the gaps are visible rather than implied away. Work in this section has a protocol and a harness but no recorded result, and any performance figure derived from it is a projection from measured components, not a measurement.

For the sake of completeness, the projections that follow from the components already measured are a search path around 190–210 ms if the vector payload were the smaller binary encoding, and around 30–50 ms if scoring moved fully into Rust and the vectors stopped crossing the boundary. Both are projected from measured parts and neither should be quoted as a shipped result.

Method

1. One variable at a time. Within a comparison run, the queries, the candidate rows, the JavaScript cosine function and the database are identical across legs. Only the transport changes, which is what makes the result causal rather than correlational.
2. Cleanroom isolation. Each run executes against a throwaway profile with fresh application-data directories, and the harness asserts the database path it opened is inside that sandbox. Your own database is never touched by a benchmark.
3. Production builds only. Latency is measured against the release executable produced by the project build shim, not a development build — after a development binary was mistakenly measured once and produced a meaningless result.
4. Synthetic corpora, labelled. Recall is measured on generated geometries (dense, clustered, sparse, uniform, near-duplicate) at 6,000, 10,000 and 50,000 rows. No user data is used, and recall percentages are therefore properties of the algorithm on those geometries, not a claim about your library.
5. Correctness gates before speed. Ranking equivalence is a hard gate: a top-k change halts the benchmark before it scales, and the run fails closed. Unit-level state at the time of writing was 555 Rust tests passed with 0 failures (1 intentionally ignored benchmark), 68/68 local-store tests and 337/337 in the shared brain package.
6. Verdict labels. Measured is read from an instrumented run. Derived is an arithmetic residual between measured parts. Projected is an extrapolation with no gated implementation behind it. Pending means the protocol exists and no result does.

Evidence artifacts

Every figure above maps to one of these files in the Serpaix monorepo. The repository opens publicly under the MIT License on January 1, 2027, at which point these become independently reproducible rather than merely stated.

Corrections and contact

If a number here looks wrong, contradictory, or measured in a way that cannot support the claim, say so — that is a bug report, and it is the most useful kind. Write to hello@serpaix.xyz or open an issue once the source is public. Corrections are recorded by updating the review date above rather than by silently editing the text.