Engineering

Metering compute you can't see: pricing on-VM AI and heavy jobs fairly

FastYoke Engineering · 8 min read · Aug 3, 2026

Architecture
Billing
Metering

The problem: your requests are not the same size

Seat-based pricing has one underrated property: it hides compute entirely. Ten users cost ten times one user, whether they click twice a day or hammer the thing all afternoon. CPU cost is smeared across the subscription and nobody has to think about it.

Usage-based pricing removes the smear, and the moment it does you have a measurement problem you can't defer — because the requests your customers make are wildly non-uniform. A workflow transition (check a guard, write a row, append an event) is microseconds of CPU. A route optimization over a few hundred stops with time windows is thirty seconds of a core doing nothing else. An embedding pass over a 90-page attachment is somewhere in between and depends entirely on the document. These are all "one request."

Charge the same for each and you've picked one of two failures. Price for the cheap request and the customer running solves is subsidized by the customer clicking buttons — your margin inverts exactly as your best customers grow. Price for the expensive one and you're charging a small business rent on microseconds. Neither is a strategy; both are a refusal to measure.

This sharpens when the expensive work runs on your hardware. If every AI call is a proxied request to a model vendor, your cost is itemized for you — someone else metered it, in tokens, and invoiced you. Run inference on your own VM, as we do for retrieval, and there is no invoice. There's just a core that was busy, and a question about who pays for it.

Why it matters now

Two things changed at once. Heavy compute stopped being exotic — route optimization, document parsing, embedding generation, semantic search used to be someone else's SaaS with its own bill, and now they're features inside your platform, triggered by a button a customer clicks without any sense that they just spent real money.

And much of that compute moved back on-VM for reasons unrelated to cost: data residency, not shipping customer records to a third party for embedding, latency. Those are good reasons — on-VM retrieval is a real architectural win — but they hand back the metering problem the API vendor used to solve for you as a side effect of charging you.

So you need a unit of account: fair, predictable to the buyer, and cheap enough to measure that measuring isn't itself a cost centre.

The candidate units, honestly

There are about five options, each wrong in a specific way you should know before you pick.

Wall-clock time. Trivially easy — timestamp, timestamp, subtract. It's also the unit that bills your customer for your operational problems. If the box is loaded, a neighbour's job is thrashing, or a disk stalls, the same work costs them more, and they can't see any of it, so from where they sit the price is random. Fine observability metric, bad billing unit.

CPU time. Fairer: it charges for work done, not time elapsed waiting. The difficulty is attribution. Getting per-request CPU time out of a multi-threaded server, split correctly across async tasks belonging to different tenants, is real engineering — and the answer still moves with the hardware. Same job, faster core, different bill.

Work units metered inside a sandbox. If the expensive logic runs inside a WebAssembly host with a fuel budget, the runtime is already counting — a deterministic count of executed work, attributed to exactly one guest execution, no sampling and no thread accounting. Two runs of the same script over the same input produce the same number on any hardware. This is why sandboxed execution and metering pair so naturally: containment and measurement are the same mechanism. The caps that stop runaway logic — bounded fuel, bounded memory — are the same counters that tell you what to bill.

Tokens. Meaningful for external LLM calls, meaningless for everything else. If you're passing through a model vendor's cost, pass through their unit — the customer can compare it to a public rate card, which is worth a lot. Don't invent a token count for work that has no tokens.

Per-job flat rates. "A route solve costs X." Wonderful for the buyer: budgetable, decidable before clicking. Risky for you, because you eat the tail. The p50 and p99 solve differ by more than you'd like, and the customer with pathological inputs runs the most of them. Capping the work itself bounds the damage — do that anyway — but you're taking variance onto your balance sheet on purpose.

Determinism beats precision

If you take one thing from this: a customer can accept a price they can predict. They cannot accept a bill that varies threefold for what looks like the same job.

That reorders the trade-offs above. A unit that sits somewhat off true cost but holds steady beats one that tracks cost to the cent and swings with load. Precision you can't explain reads as arbitrariness, and arbitrariness in a bill generates disputes and churn even when every number is correct.

So prefer a unit the customer can reason about before they click — bounded work per job, a stated cap, a count they could have predicted from the size of their own input. If the only way to know the price is to run the job, you've built a slot machine.

And measure at a boundary you control. Inside the sandbox, every unit belongs to one execution for one tenant. At the VM level, once several tenants share a box, the number is a shared resource you're allocating by heuristic — and heuristics are what customers dispute.

Last, the honesty test: if you can't explain the unit in one sentence to a non-engineer buyer, it's the wrong unit. "You pay per document embedded" passes. "You pay per normalized compute second adjusted for instance class" does not.

How we handle it: soft debit, hard enforcement

FastYoke's model separates two things most billing systems fuse.

Metering is a soft debit. Every metered event debits a prepaid wallet balance as it happens and appends to an immutable usage ledger. The wallet is allowed to go negative — we will not fail a legitimate workflow transition because the balance ran dry mid-flight. That's a deliberate principle, not a gap: a billing edge case must never become an operational outage inside someone's business process. Ledger and credits are append-only, so what happened and what was paid is auditable after the fact rather than enforced in the hot path.

Enforcement is a separate sweep. Because metering never blocks, the consequences live somewhere else. A low-balance alert fires while there's still runway — once, re-arming only after the balance recovers. A zero balance auto-suspends a standard organization's tenants, reason recorded. Crediting the wallet auto-unsuspends. Enforcement is coarse, out-of-band, and reversible, which is the right shape for something that can take a customer offline.

Now the most-misunderstood part, stated flatly: the cap is a prepaid wallet floor, not a monthly spend ceiling you type into a box. There is no "stop this customer at $500 this month" field. Usage draws a balance down; a balance at zero suspends. That bounds exposure, but it's a funding bound, not a budget. If you're expecting a spend-limit field — most people are, because card-based platforms condition you to expect one — reset that model now, not after the first surprise.

The rest follows. Apps cost nothing and are not metered or licensed per app; you own them. What you pay for is platform usage above a free tier. On-VM retrieval is a metered add-on on Pay-as-you-go and bundled into Enterprise Platform alongside region pinning, SSO, dedicated compute, and an uptime SLA — because at that scale the buyer wants a fixed number, not a meter.

What to watch out for

Metering overhead that outweighs the work. If you're billing microsecond transitions, an instrumentation path costing microseconds per event has doubled the cost of what you're measuring. Batch the writes; sample cheap events and meter expensive ones precisely.

Retries and idempotency. A job that fails at second 29 of 30 and retries has consumed sixty seconds of your CPU. Bill one or two? Either answer is defensible — pick one, write it down, and make the meter implement what you wrote. Every retried job needs an identity, or you'll double-bill someone and hear about it from them.

Cold starts. The first invocation after an idle period pays for initialization nobody asked for. Amortize or absorb it; don't put it on the first caller's line item.

The free tier as a denial-of-wallet vector. A free tier with access to expensive compute is an invitation to burn your CPU on someone else's account. Bound the work — per-job caps, concurrency limits — not just the money. The wallet stops billing abuse; only resource caps stop resource abuse.

Unexplainable line items. Every metered unit should trace back to an action in the event log, with a timestamp and an actor. A row the customer can't map to something they did is a bad row even when it's arithmetically correct.

Where this goes next

The direction of travel is toward units bounded by construction rather than measured after the fact. A sandbox that refuses to exceed a work budget doesn't only protect the host — it makes the price knowable in advance, which is what customers actually want. Metering and containment converge because they answer the same question: how much of my machine is this allowed to use?

No unit is simultaneously perfectly fair, perfectly predictable, and free to measure. You're choosing which of the three to give up a little of, and for a platform the right answer is usually to give up precision, keep predictability, and be transparent about the trade.