Sub-millisecond billing: how Finno stays invisible on the hot path

Developers choose your API for latency and reliability. They abandon it when billing middleware adds jitter, timeouts, or mysterious 503s from a “usage service” sitting in the critical path.

The hardest requirement for revenue infrastructure is paradoxical: it must be financially strict and operationally invisible. If consumers feel your billing layer, they feel friction — and they route around you.

Finno’s design goal is simple to state and hard to execute: the entire billing cycle (authenticate, resolve funding, reserve, meter, finalize, record) must stay below human-perceptible latency while still committing every charge durably before the HTTP response returns.

At p99, that budget is under 250 microseconds.

Where the milliseconds actually go

In a typical metered API call, latency is dominated by factors you do not control:

Stage	Typical cost
Client ↔ your edge	20–200 ms
Your gateway ↔ upstream provider	10–500 ms
Model inference / heavy compute	200 ms – 30 s

Adding even 5 ms of billing overhead is noticeable in SLAs. Adding 50 ms changes product economics for real-time use cases. Calling Postgres or Redis on every request — or waiting for a billing microservice — is a non-starter.

Finno’s answer is architectural: nothing on the hot path waits on network I/O to external stores.

The hot path lives entirely in memory

When a request hits the gateway:

1. Route matching — compiled matchit table behind a lock-free atomic pointer. No per-request allocation.
2. Authentication — bearer token hashed; consumer resolved from an in-memory table loaded at boot and refreshed via LISTEN/NOTIFY.
3. Funding resolution — walk quota → wallet → credit from atomic ledger state in a sharded in-memory structure.
4. Pricing — worst-case cost computed from the consumer’s package (flat, graduated, or volume tiers) using integer micros throughout.
5. Reserve — propose a Raft command; followers re-derive sufficiency deterministically.

No database round trip. No cache miss to Redis. No HTTP call to a billing SaaS.

Representative microbenchmarks from production-shaped workloads:

Operation	Cost
Auth lookup	< 1 µs
Balance / quota check	< 1 µs
Route matching	~5 µs
Request transform (compiled jaq bytecode)	10–50 µs
Reserve + Finalize (Raft group-commit)	20–100 µs
Response transform	10–50 µs

A datacenter round trip is ~1 ms — four orders of magnitude larger than Finno’s overhead. Your upstream provider is the slow part. Finno is a rounding error on the invoice, not on the stopwatch.

Durability without blocking the response

“But wait,” you say — “if everything is in memory, how is it durable?”

Finno uses Raft for linearizable command replication. The trick is when you pay for fsync.

In group WAL mode, Reserve is enqueued at the start of the request filter, but durability is awaited in the logging phase — after the upstream call returns. The ~100 µs fsync runs concurrently with the upstream round trip that already takes milliseconds.

Timeline:
|-- Reserve (queued) --|-- upstream 250 ms --|-- Finalize + fsync 100 µs --|
                       ↑                                    ↑
                  overlap window                    response ready

Durability is not skipped. It is scheduled into dead time you were already spending waiting on the provider.

On leader failure, followers hold a warm replica and elect within 150–300 ms. The load balancer health-checks route only to the leader. Your API stays up; your ledger stays consistent.

Cold path vs hot path

Not everything belongs in memory — and Finno does not pretend otherwise.

Concern	Path	Mechanism
Per-request charge	Hot	In-memory ledger + Raft
Deposits, invoice payments	Cold	Leader-driven reconcile from Postgres
Subscription renewals	Cold	Subscribe / period roll commands
Config changes	Warm	ArcSwap reload; route tables recompiled

Postgres remains the configuration and reporting store. The Raft log is the source of truth for balances and charges. Dashboards and financial reports read from the mirror — but the request path never blocks on them.

This separation is why Finno can be both fast and finance-grade. The hot path does one thing: apply commands deterministically. The cold path does everything else.

Why Rust — and why integers

Performance is not only about microseconds. It is about predictability.

• No GC pauses mid-request. A charging layer that stalls the world for 50 ms every few seconds is indistinguishable from an outage to latency-sensitive clients.
• Memory safety without sacrificing speed. Use-after-free in a billing system is not a crash — it is corrupted balances.
• Money as i64 micros everywhere. No floating-point tiers. No “0.1 + 0.2” invoice surprises. Pricing converts units → micros with integer arithmetic only.

Combined with mimalloc, CPU pinning, and Linux tuning on gateway nodes, the runtime behaves like infrastructure — not an app server hoping the GC cooperates.

What this means for product teams

When billing is invisible:

• You can put every endpoint on metered pricing without fearing SLA regressions.
• Token metering from JSON responses (via compiled jaq paths) does not require a separate observability pipeline.
• 402 Payment Required is a first-class response — consumers get immediate feedback instead of surprise invoices.

You should not have to choose between “correct books” and “fast API.” The same process that proxies is the process that posts to the ledger — in less time than it takes light to cross a rack.

---

Your upstream is the slow part. Finno’s job is to make revenue as cheap as a route lookup — and as durable as a bank ledger.