Rust API

The target ergonomics: someone coming from rusqlite should feel at home within five minutes, and the versioning capabilities — time-travel, branches, auditing — should be first-class, not an afterthought. Arkeion is an embedded engine: it links into your program as a library and reads and writes a single file. There is no server to run, no port to open, and no network hop between your code and your data.

The crate is arkeion on crates.io — the current release is v0.12.

Main types

#![forbid(unsafe_code)]

pub struct Database;            // shared, cloneable handle (internal Arc)
pub struct Connection;          // view over a branch (defaults to "main")
pub struct Transaction<'conn>;  // explicit multi-statement write
pub struct Rows;                // result iterator
pub struct Row;

#[derive(Clone, Debug, PartialEq)]
pub enum Value { Null, Bool(bool), Integer(i64), Real(f64), Text(String), Blob(Vec<u8>) }

#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
pub struct Version(pub u64);

#[derive(Clone, Copy)]
pub enum AsOf { Head, Version(u64), Timestamp(std::time::SystemTime) }

pub struct Options {
    pub create_if_missing: bool,   // true by default (zero-config)
    pub key: Option<Key>,          // Some(_) => AES-256-GCM at rest
    pub compress: bool,            // page compression on create (off by default)
    pub ecc_nsym: u8,              // Reed-Solomon parity per block on create (0 = off)
    pub cache_bytes: usize,        // page cache cap (default 64 MiB)
}
// builders: .create_if_missing(b) .with_key(k) .compress(b) .ecc(n) .cache_bytes(n)

pub struct Key([u8; 32]);          // raw key; Drop => zeroize. Deriving it from a
                                   // passphrase (a KDF) is the caller's responsibility.

#[non_exhaustive]
pub enum Error { Io(..), Corrupt{..}, ChainBroken{at: Version, ..}, WrongKey,
                 Sql{msg: String, pos: usize}, Conflict(MergeConflicts),
                 BranchNotFound(String), VersionNotFound(AsOf), Busy, .. }
pub type Result<T> = std::result::Result<T, Error>;

Two things to notice up front. Value is a closed set of six SQL types, so there is no dynamic typing surprise. And errors are a typed, #[non_exhaustive] enum — a broken hash chain, a wrong key, or corruption on disk are values you handle, never silently-bad data returned as if it were fine.

Database — lifecycle, branches, auditing

The Database is a cheap, cloneable handle to the file. Cloning it is just an Arc bump; share it across threads freely.

impl Database {
    pub fn open(path: impl AsRef<Path>, opts: Options) -> Result<Database>;

    pub fn connect(&self) -> Result<Connection>;                  // "main" branch
    pub fn connect_branch(&self, name: &str) -> Result<Connection>;

    pub fn create_branch(&self, name: &str, from: AsOf) -> Result<()>;
    pub fn drop_branch(&self, name: &str) -> Result<()>;          // deletes the ref, not the pages
    pub fn branches(&self) -> Result<Vec<BranchInfo>>;            // {name, head: Version, created}

    pub fn diff(&self, from: &str, to: &str) -> Result<Diff>;     // O(changes), not O(data)
    pub fn merge(&self, from: &str, into: &str, policy: MergePolicy) -> Result<MergeReport>;

    pub fn verify(&self) -> Result<AuditReport>;                  // walks the entire hash chain
    pub fn verify_anchor(&self, anchor: &AuditAnchor) -> Result<AuditReport>; // detects truncation/rewriting

    pub fn history(&self) -> Result<Vec<Revision>>;               // "git log": timeline of versions
    pub fn diff_versions(&self, from: u64, to: u64) -> Result<Diff>;          // "git diff" between versions

    pub fn vacuum(&self, retention: Retention) -> Result<VacuumReport>;       // compacts + atomic rename
    pub fn vacuum_rekey(&self, retention: Retention, new_key: Option<Key>)    // compacts and rotates the key
        -> Result<VacuumReport>;
}

pub enum MergePolicy { FailOnConflict }                 // v1; future: Theirs, Ours, resolver
pub enum Retention   { KeepAll, KeepLast(u64), KeepSince(SystemTime) }

pub struct Diff { pub tables: Vec<TableDiff> }          // additions/deletions/modifications by rowid + schema diffs
pub struct AuditReport { pub head: u64, pub commits: u64, pub chain_ok: bool, pub chain_hash: [u8; 32] }
pub struct AuditAnchor { pub version: u64, pub chain_hash: [u8; 32] }   // AuditReport::anchor() creates it
pub struct Revision { pub version: u64, pub timestamp: SystemTime, pub parent: u64 }
pub struct VacuumReport {
    pub kept_from: u64, pub head: u64, pub reclaimed_versions: u64,
    pub pages_before: u64, pub pages_after: u64,
}

branches, diff, and merge are the version-control surface: a branch is a named pointer at a version, diff is O(changes) because it walks the two commit roots rather than the data, and merge replays one branch onto another under a policy. verify() walks the whole hash chain and tells you it is intact; verify_anchor() compares it against an anchor you saved earlier, which is how you detect that history was truncated or rewritten out from under you. vacuum(Retention) is the one operation that forgets: it drops versions outside the retention window and rewrites the live set into a fresh file with an atomic rename.

Connection — SQL, transactions, time-travel

impl Connection {
    pub fn execute(&self, sql: &str, params: &[Value]) -> Result<usize>;   // rows affected
    pub fn query(&self, sql: &str, params: &[Value]) -> Result<Rows>;
    pub fn prepare(&self, sql: &str) -> Result<Statement>;                 // parses once

    /// Bulk load: all rows in ONE transaction (one commit, one fdatasync),
    /// without a per-row SQL executor; index entries are inserted in bulk
    /// (UNIQUE verified). Autocommit only: either the whole batch or nothing.
    pub fn bulk_insert<I, R>(&self, table: &str, rows: I) -> Result<usize>
    where I: IntoIterator<Item = R>, R: AsRef<[Value]>;

    pub fn begin(&self) -> Result<Transaction<'_>>;     // acquires the single writer

    pub fn snapshot(&self, at: AsOf) -> Result<Connection>;  // pinned READ-ONLY connection
    pub fn version(&self) -> Version;                   // current head of the branch
    pub fn branch(&self) -> &str;
}

impl Transaction<'_> {
    pub fn execute(&self, sql: &str, params: &[Value]) -> Result<usize>;
    pub fn query(&self, sql: &str, params: &[Value]) -> Result<Rows>;      // reads its own writes
    pub fn commit(self) -> Result<Version>;
    pub fn rollback(self) -> Result<()>;                // Drop without commit => implicit rollback
}

execute/query outside a transaction are autocommit — one transaction per statement. Inside begin(), statements accumulate and commit() returns the new Version they produced.

A simple SELECT — a column projection or * with no WHERE/JOIN/aggregate/ ORDER BY — is served in streaming fashion: Rows owns its snapshot and decodes each row as it iterates, touching only the projected columns and never materialising the whole result. Every other query goes through the full executor; the result is indistinguishable except in cost.

Snapshots and time-travel

snapshot(at) gives you a read-only Connection pinned to a past version. Because history is append-only, that snapshot is stable no matter how many commits land afterward — readers pin a version, the single writer appends new ones, and the two never contend.

Engine API (no SQL)

Typed row access that skips the SQL parser, planner, and executor and goes straight to the catalogue and the b-tree — preserving every guarantee (versioning, indexes, encryption, and the audit chain). Connection::table takes a consistent snapshot when created; it is a stable read that does not see open transactions. Writing at the engine level is bulk_insert.

impl Connection {
    pub fn table(&self, name: &str) -> Result<TableReader>;  // snapshot on creation
}

impl TableReader {
    pub fn get(&self, rowid: i64) -> Result<Option<Vec<Value>>>;      // point lookup by PK
    pub fn scan(&self) -> Result<impl Iterator<Item = Result<(i64, Vec<Value>)>>>;
    pub fn scan_columns(&self, cols: &[usize]) -> Result<ProjectedScan>; // projected, no alloc/row
    pub fn count(&self) -> Result<u64>;
    pub fn column_index(&self, name: &str) -> Option<usize>;
    pub fn version(&self) -> u64;
}

impl ProjectedScan { // borrowing iterator: next() borrows &[Value] until the next call
    pub fn next(&mut self) -> Result<Option<&[Value]>>;
}

What it’s for: throughput and control when SQL isn’t needed — embedded ledgers, event stores, custom index maintenance. Measured on the same engine and data, a point lookup by PK is about 3.7× faster than SELECT … WHERE id = ?, because it saves the per-call parse, plan, and validation. A full scan gains only about 1.1×, because the SQL scan path already uses the same projected streaming route — the cost there is per-row record decoding, not the SQL layer. In short, the engine API’s advantage is in random and point access, not scanning.

Rows and parameters

impl Rows { /* Iterator<Item = Result<Row>> */ }

impl Row {
    pub fn get<T: FromValue>(&self, col: impl ColIndex) -> Result<T>;  // by index or name
}

// FromValue for: i64, f64, String, Vec<u8>, bool, Option<T>, Value
// Into<Value> for the same, via a convenience macro:
let n = conn.execute(
    "INSERT INTO clients (name, created) VALUES (?1, ?2)",
    &params!["Acme GmbH", 1718000000_i64],
)?;

Full example

use arkeion::{Database, Options, AsOf, MergePolicy, params};

let db = Database::open("tenant-42.arkeion", Options::default().with_key(key))?;
let conn = db.connect()?;

conn.execute("CREATE TABLE invoices (id INTEGER PRIMARY KEY, total REAL, status TEXT)", &[])?;

let tx = conn.begin()?;
tx.execute("INSERT INTO invoices (total, status) VALUES (?1, ?2)", &params![120.0, "draft"])?;
let v1 = tx.commit()?;

// — time-travel —
conn.execute("UPDATE invoices SET status = 'issued' WHERE id = 1", &[])?;
let before = conn.snapshot(AsOf::Version(v1.0))?;
let status: String = before.query("SELECT status FROM invoices WHERE id = 1", &[])?
                           .next().unwrap()?.get(0)?;          // "draft"

// — branching for a migration —
db.create_branch("vat-migration", AsOf::Head)?;
let mig = db.connect_branch("vat-migration")?;
mig.execute("UPDATE invoices SET total = total * 1.21", &[])?;
let diff = db.diff("main", "vat-migration")?;                  // review before merging
db.merge("vat-migration", "main", MergePolicy::FailOnConflict)?;

// — audit —
assert!(db.verify()?.chain_ok);

API guarantees

• Database: Send + Sync and cheap to clone; a Connection is per-thread (Send, not Sync).
• Reads never block a write, and are never blocked by one.
• Single writer. Writes are serialised. Autocommit (execute or bulk_insert outside a transaction) queues: under contention it waits its turn — which lasts microseconds, because the commit releases the writer before its fdatasync and concurrent commits’ syncs are batched together by group commit — and only returns Busy if the writer is still held past a safety threshold. An explicit transaction (begin() / BEGIN) instead returns Busy immediately if another write is in progress, since it may hold the writer for an unbounded time and must not hang other callers.
• snapshot() and branch connections share the same Database page cache.
• Every read validates its authentication tag; Corrupt, ChainBroken, and WrongKey are typed errors, never silently bad data.

Network access

The API above is embedded and in-process. For network access, the client / server layer exposes it over a native protocol — one branch per session, with AS OF and verify as first-class operations — in separate crates:

• arkeion-client (Client, MIT/Apache, separate repo): connect, use_branch, execute, query / query_as_of, verify. It mirrors the subset of Connection that travels over the wire.
• arkeiond (the server, proprietary): the daemon, thread-per-connection, that serves each session over a branch of the same Database.

The protocol is hand-rolled, without serde, reusing the engine’s own varint and Value encoding.