Transactions and derived state

These patterns belong to the data layer. They describe deployment choices and proposed reusable mechanisms, not a claim that Reaktor supplies a complete transaction, saga or event-sourcing runtime. Current storage distinguishes present APIs; Tidemark consistency defines authority and transaction domains.

Durable admission and relay

A command often changes business state and requests an external effect. Commit the state change and an outbox/intent record together in the same local authority transaction. A relay publishes unsent intent and records broker acceptance. A crash after publish but before marking creates duplicate delivery, so consumers retain stable effect identity and deduplicate at their own authority. Two independent ObjectStore writes or a Queue send inside a database transaction do not make the two systems atomic.

A DO can atomically persist state and intent within its own storage and drive a relay with an alarm. It cannot atomically commit an external API call, Queue send or another database's transaction. A D1 outbox must share the applicable atomic transaction/batch with the business rows. Where the authoritative graph/provider cannot supply the needed transaction, choose an explicit recoverable admission/receipt protocol rather than silently adding a second authority.

Idempotent effects

Bind the key to tenant/application, operation revision and immutable input fingerprint. Concurrent duplicates need a uniqueness/conditional-write protocol. Commit state effect and dedup result in the same local transaction. For external APIs, use their idempotency protocol or reconcile unknown results; check-then-call alone is not sufficient. Retain receipts for the permitted retry, delay and redrive horizon, which can exceed provider history retention.

An effect can be qualified as occurring once within a stated transaction/idempotency scope. Queue transport remains at least once, and replay after expired dedup can repeat effects. See Queues and Workflows for delivery and execution admission.

1. Data and state patterns

1.1 Caching

Strategies: cache-aside (lazy; most common), read-through, write-through (always fresh, slower writes), write-behind (fast writes, crash-loss risk — treat its buffer as needing outbox-grade durability). Invalidation is the crux: TTL (bounded staleness, no coordination) vs explicit-on-write (precise, must catch every path). Stampede protection: single-flight / request coalescing, probabilistic early expiry, stale-while-revalidate. Reaktor mappings:

1.2 Batching

Best practice: coalesce many small calls into one (the DataLoader pattern — per-tick batching + per-request memoization; solves N+1); micro-batch writes by size/time; debounce/coalesce bursts. The knob is latency vs throughput (DDIA Ch. 11/12); bigger batches amortize overhead but raise tail latency and failure blast radius. Reaktor: a Channel + windowed collector or Flow (buffer/conflate) for coalescing; JDBC batch / R2DBC for writes; CF Queues batching is first-class; a DO accumulating on an alarm is a server-side DataLoader; D1 batch() for multi-statement round-trips.

1.3 The authority/cache topology

Selected by data contracts and qualified bindings: Postgres is the default relational/document home; vector and search are relational facets by default. Memgraph supplies graph storage, ClickHouse analytical serving, D1/DO/KV suitable edge state, R2 objects/archive, Pub/Sub event distribution and SQLite local state where qualified. Authority, projections, caches and conflict semantics are explicit per dataset/operation. Provider, runtime and consistency choices remain separate.

1.4 Ledger-style invariants for scarce or conserved things

Use a ledger when a system tracks a scarce, valuable, reservable, transferable, consumable, expiring, refundable, externally reconciled, or audit-sensitive thing. Literal double-entry is for money, but the generalized invariant is broader:

movement = immutable transfer between accounts
for every movement:
  source loses quantity
  destination gains quantity
total debits == total credits

Creation and destruction are still explicit movements through source/sink accounts (Supplier -> Available, Available -> Expired, User credits -> Consumed). That keeps the audit trail intact and makes retries, reversals, and reconciliation explainable.

Reaktor mapping: a ledger is an append-only fact stream plus projections. The write path is a command handler guarded by idempotency and optimistic concurrency; the durable fact is a ledger movement/event; the outbox publishes it; projectors build balances, dashboards, availability counts, and audit views. ActorNode/Durable Object aggregates are the natural single-writer authority for small hot ledgers; Postgres/D1 tables with uniqueness, version columns, and append-only entries are the durable store.

Do not force this pattern onto generic CRUD. Profiles, comments, settings, and static metadata may only need versioned updates and audit rows. Ledger modeling earns its cost when the object is scarce, valuable, externally reported, retried, corrected, or owned by multiple services.

1.5 Reconciliation, repair, and derived state

Production systems do not have one perfect truth. They have internal facts plus external observations: payment gateway reports, bank settlements, warehouse counts, courier scans, provider delivery receipts, cloud bills, tax invoices, analytics pipelines, and user-visible status. The data layer treats reconciliation as a first-class flow, not a spreadsheet afterthought.

expected state
observed external state
difference
reason code
resolution entry
audit trail

Reconciliation handlers are ordinary command handlers with stricter audit requirements: they read an external report, compare it with the authoritative ledger/state machine, post adjustment facts (LedgerCorrectionPosted, InventoryCountAdjusted, NotificationReceiptReconciled, OrderStateCorrected), and publish repair events through the outbox. Raw SQL repair stays outside the normal path; data repair becomes a controlled workflow with actor, authority, ticket/case link, before/after, and reason code.

Derived fields follow the same rule. canCancel, remainingCredits, currentOrderStatus, availableInventory, and merchantDashboard may be cached projections, but the source of truth is the durable facts and legal transitions that explain them. The cache can be rebuilt; the facts cannot be guessed later.

2. Transactional patterns

2.1 Command-shaped APIs, state machines, and temporal truth

Production APIs expose intent, not arbitrary mutation:

POST /orders/{id}/cancel
POST /orders/{id}/refund
POST /orders/{id}/ship
POST /orders/{id}/return-request

not:

PATCH /orders/{id} { "status": "REFUNDED" }

The command validates business rules, checks the current version, records a transition, posts any ledger movement, and schedules side effects. The transition record is as important as the current state:

entity_id
from_state
to_state
reason_code
actor
authority
correlation_id
effective_at
recorded_at
metadata

This prevents impossible boolean combinations (isPaid && isCancelled && isShipped) and gives support, finance, security, and ops the trail they need. Legal transitions belong in policy/config where they change often, but the handler owns enforcement. Invalid transitions fail before side effects.

Reservation / hold / commit. Scarce resources should move through explicit states: Available -> Reserved -> Committed -> Consumed, Available funds -> Authorized -> Captured -> Settled, Available seat -> Held -> Booked -> CheckedIn. The invariant is: do not consume a scarce resource until the workflow has earned the right to consume it. Holds get expiries and release paths; commits are idempotent; ambiguous external results trigger reconciliation before retry.

Optimistic concurrency and fencing. Commands update the state they observed (WHERE id = ? AND version = ?), then retry or fail on conflict. Distributed single-writer work uses leases plus fencing tokens; stale workers must not be able to commit newer work. This is the same discipline as ClusterActorRef fencing in the distributed-actors design, applied to every business state machine.

Temporal modeling. Store both when something was true and when the system learned it (effective_at, recorded_at, valid_from, valid_to). Pricing, tax, subscriptions, contracts, permissions, discounts, and refunds often depend on yesterday's truth, tomorrow's scheduled truth, or a corrected external fact.

2.2 Saga

Problem: a business transaction spans multiple services/DBs; 2PC is avoided (blocking, availability-killing). A saga is a sequence of local transactions, each triggering the next; on failure, completed steps are undone by compensating transactions (semantic undo). Sagas do not create a global atomic or isolated transaction. Guard intermediate states, make compensations idempotent and track failed or impossible compensation for reconciliation. A compensation is a new semantic effect, not erasure of history.

• Orchestration vs choreography. Orchestration = a central coordinator issues commands and decides compensation (explicit, monitorable; risk of a smart hub). Choreography = services react to events and emit their own (loosely coupled; emergent flow is hard to debug). Reaktor leans orchestration, modeled as a saga graph authored in the visual language and executed by a graph visitor / pass (reaktor_graph_visitors_design), with the orchestrator's state held by an actor / Durable Object (durable, single-writer, alarm-driven retries).
• JVM — Arrow Saga (in-process): saga { saga(action){ compensation } }.transact() — compensations run only if the whole saga fails. Great for composing local steps / sequencing remote calls with rollback inside one service; not a distributed durable engine.
• Long-running / cross-service / crash-surviving: durable orchestration state — a Durable Object saga orchestrator (alarms for step timeouts/retries, Queues for commands), or Cloudflare Workflows (durable execution). A JVM actor can hold coordination state, but durable transactions, ownership recovery and fencing must be implemented; an ObjectStore reference alone does not establish a crash-surviving saga engine.

// In-process composition with rollback (Arrow): order → payment → ship, compensate on failure
val checkout: Saga<Receipt> = saga {
    val o = saga({ orders.place(cart) })   { orders.cancel(it.id) }
    val p = saga({ payments.charge(o) })   { payments.refund(it.id) }
    saga({ shipping.dispatch(o, p) })      { shipping.recall(it.id) }
}
// Either.catch { checkout.transact() }     // cross-service/durable: a DO/Workflow orchestrator instead

The Bus design provides transport and correlation/causation contracts, not a shipped saga engine. Reusable graph-native saga execution remains proposed; CompanyOS has a provider-specific Workflow skeleton. See durable execution status.

2.3 Transactional outbox

See durable admission and relay — the dual-write fix; the foundation under sagas, CQRS, and event sourcing.

2.4 CQRS

Pattern: split the write model (commands, normalized, enforces invariants) from one or more read models (denormalized materialized views shaped per query), built by projecting write-side events (via outbox / event sourcing). Costs: eventual consistency between models (read-your-writes-eventually), operational complexity. When NOT to: simple CRUD, low scale, or when immediate read-your-writes consistency is required — don't default to it. Reaktor: write model in Postgres/D1/DO; a bus consumer (projector) updates read models idempotently (KV / D1 / a read-optimized table); the projector is a Queue consumer or a DO alarm; projections must be idempotent and replayable. A Reaktor-aligned analytical read model is ClickHouse (datastore families) — a columnar projection of the same events for funnels/cohorts/aggregates (eventually consistent; never a second product truth). Status: absent (document for completeness; adopt per-service, not globally).

2.5 Event sourcing

Pattern: persist state as an append-only log of immutable events (the log is truth); current state = fold/replay; projections derive read models (→ CQRS); snapshots bound replay cost; replay rebuilds/repairs projections and enables audit/time-travel. Event sourcing can make the committed event the delivery intent; reliable tailing, durable consumer positions and idempotent effects still require an explicit protocol. Costs: event versioning/upcasting is hard; eventual consistency; replay cost without snapshots. Reaktor: a Durable Object or ActorNode is a natural event-sourced aggregate — append events to DO SQLite / ObjectStore, keep folded state in memory, snapshot periodically, fan events to projectors over the bus; single-writer ownership still requires version checks for stale commands, and fencing/recovery when ownership can change. Status: absent; the ObjectStore DatabaseEvent stream (Put/Delete/Origin) is the nearest existing primitive, but it is change-tracking, not an event log.

Projection positions and repair

A projection records the source domain, epoch and applied position, or a vector of positions when multiple sources contribute. A wall-clock timestamp or HLC alone does not prove that a particular command has been applied. Read-your-writes requires comparing the authoritative commit token with the relevant applied position, waiting or reading authority when the projection lags. Federated observations do not imply one atomic snapshot.

Rebuild and replay preserve operation identity, scope and deletion fences. Record gaps and failed items rather than skipping them and advancing a misleading watermark. Queue, Workflow and operator repair receipts link to the exact input/definition revisions. Implementation gates require fault and convergence evidence.