Arkeion ἀρχεῖον
Cloud
Release v0.13

Speed, with receipts

v0.12 put vector search inside the file. v0.13 makes it fast enough that you stop thinking about it — same recall, same guarantees, three times the throughput. This post is the release note, but it is also a small essay on how we got there, because half of the story is the optimizations we refused to ship.

The numbers first

SIFT-1M — the standard ANN benchmark: one million 128-dimensional vectors, 10,000 real queries, ground truth included. Same machine, same configuration, only the engine changed. And because Arkeion’s index build is deterministic, both versions train the exact same clusters: recall is byte-identical at every point of the curve. What you gain is pure speed:

nprobe recall@10 v0.12 v0.13
10 0.850 2.73 ms 1.10 ms 2.5×
20 0.938 4.09 ms 1.39 ms 2.9×
50 0.988 9.12 ms 2.48 ms 3.7×
100 0.996 15.8 ms 3.93 ms 4.0×
400 0.997 58.9 ms 11.7 ms 5.0×

Concurrent throughput at the recall@10 = 0.988 operating point goes from 375 to 1,381 queries per second (8 threads, 4 cores). The exact (non-indexed) KNN scan drops from 1.34 s to 322 ms per query. The vector index shrinks from 38.8 MB to 25.4 MB (−35%).

Milliseconds per query — lower is better v0.13 v0.12 recall 0.94 recall 0.988 recall 0.996 1.39 ms 4.09 ms 2.48 ms 9.12 ms 3.93 ms 15.8 ms 2.9× faster 3.7× faster 4.0× faster 0 5 ms 10 ms 15 ms
SIFT-1M, single thread, query at a time — same clusters, same recall, only the engine changed.

Nothing about the semantics moved. Deterministic builds, versioned history, AS OF over search — all intact. Ties in a shortlist now even resolve deterministically (by rowid), independent of scan order or thread count.

The FAISS question

The comparison everyone actually wants: FAISS, the reference ANN library. Same box, same SIFT-1M, identical configuration on both sides — IVF with 1,000 lists, PQ16 codes, exact rerank over a ×32 shortlist (IVF1000,PQ16,RFlat with k_factor=32 in FAISS terms). Query at a time, one thread:

recall@10 FAISS Arkeion v0.13 v0.12, for scale
0.85 0.42 ms 1.10 ms 2.73 ms
0.94 0.60 ms 1.39 ms 4.09 ms
0.99 1.32 ms 2.48 ms 9.12 ms
0.999 5.62 ms 11.7 ms 58.9 ms
Recall@10 vs latency — log scale v0.13 FAISS v0.12 1.00 0.90 0.80 0.70 recall 0.99 FAISS v0.13 v0.12 0.5 1 2 5 10 20 50 milliseconds per query (log)
The same climb to recall 0.99: FAISS gets there at ~1.3 ms in RAM, v0.13 at ~2.5 ms from a durable file, v0.12 needed ~9 ms.

FAISS holds everything in RAM and promises nothing: no durability, no transactions, no versioning, no encryption, no SQL — if the process dies, you retrain. Arkeion serves the same recall from an encrypted, versioned, crash-safe file, two milliseconds instead of one. Before v0.13 that gap was 7–10×; now it is the price of being a database, and we think it is the right price. (One methodological note: FAISS without the exact-rerank stage caps at recall 0.56 on this config — if you have seen suspiciously fast IVFPQ numbers somewhere, check for that.)

What shipped

Postings in blocks. The old index stored one b-tree cell per vector: every candidate paid a full cell visit — decode, key check, callback — before its distance was even computed. In v0.13 each cluster stores columnar blocks of codes and rowids (~3 KB per cell), so the distance kernel walks contiguous memory and the b-tree toll amortizes across a block. This is the format change behind both the speed and the smaller index. Your existing vector indexes keep working untouched in the old format; REBUILD VECTOR INDEX migrates them when you choose to.

Filtered KNN uses the index. The most common real query — WHERE tenant_id = ? ORDER BY distance LIMIT k — used to fall back to an exact full scan, because a WHERE clause disabled the vector plan entirely. Now the planner oversamples the shortlist, filters row by row, escalates to every cluster if the filter is hungry, and only falls back to the exact scan when even that cannot guarantee k survivors. Correct by construction; fast in the common case.

A knob that was missing. CREATE VECTOR INDEX … FACTOR n controls how much the index oversamples before the exact rerank (default 32, the old hardcoded value). Good PQ codebooks are happy at 8 — a quarter of the row fetches; hard data can ask for more. It is a per-index decision now, not ours.

The quiet ones. The SQL path caches decoded centroids the way the API always did (~2 ms of every query, gone). B-tree cursors descend in-page instead of materializing every inner node — that alone took the cluster scan from 2.9 ms to 1.2 ms, and sped up exact KNN as a side effect. Page-cache hits no longer touch the global pager lock, which is where the concurrent scaling was hiding. A flat ADC lookup table, a bounded candidate heap, a parallel rerank for the latency-sensitive path.

Where a query spends its time now index metadata centroids + PQ codebooks decoded once, cached was ~2 ms on every SQL query · now 0 cluster scan block postings · flat LUT bounded heap · radius pruning in-page b-tree descents 2.9 → 1.2 ms exact rerank k × FACTOR candidates vector column only fetched in rowid order parallel, opt-in 9.12 ms → 2.48 ms end to end SIFT-1M · recall@10 = 0.988 · single thread
Every stage kept its guarantees: deterministic build, exact rerank, versioned history.

What we refused to ship

We profile before we optimize, and we measure before we merge. Three ideas died that way, and we think the graveyard is worth publishing:

  • A shared-cursor multi-get for the rerank fetch: elegant, and 20× slower on scattered rowids. Diagnosing why led straight to the in-page descent fix above — the prototype paid for itself, then got deleted.
  • An integer SIMD kernel for int8 scans: 1.01×. LLVM already autovectorizes the float path; the extra quantization error bought nothing.
  • FAISS-style 4-bit fastscan: a real AVX2 prototype with shuffle_epi8 measured 1.9× — on a kernel that is now ~15% of the scan. A format change plus an unsafe-code exemption for ~8% end to end is a bad trade. Our scalar ADC runs at 8 ns per candidate; we’ll revisit when the rest of the scan catches up to it.
  • Full FAISS parity. We know how to close the remaining 2×: keep a flat copy of every vector inside the index (what FAISS calls RFlat) so the exact rerank never touches the row tree. It works — and it costs +512 MB per million vectors at 128 dimensions, +3 GB per million at 768. We decided a millisecond is cheaper than a gigabyte: the 25 MB index stays, the extra millisecond stays, and your disk stays yours. If real workloads ever disagree, it can ship later as an opt-in — never as the default.
The graveyard, measured — end-to-end gain no real gain works, too costly 1.0× = no change shared-cursor multi-get integer int8 kernel 4-bit fastscan flat rerank companion 0.19× fetch 20× slower on scattered rowids 1.01× — LLVM had already vectorized it 1.08× — a 1.9× kernel on 15% of the scan 1.9× · +0.5 GB per M vectors 0× 0.5× 1× 1.5× 2× bars start at 1.0× — left of the line is a regression
Four ideas, four measurements, one survivor on merit — killed by its own disk bill.

A database that promises you provable history should also prove its performance claims. Every number above comes from a reproducible benchmark, and every rejected optimization comes with the measurement that killed it.

Upgrading

cargo update -p arkeion to 0.13.0. Existing files open as always; existing vector indexes serve queries in their current format. Run REBUILD VECTOR INDEX <name> per index when you want the block format and the new speed — it is a discrete, deterministic event, like every build in Arkeion.