The Data Graph

Earlier versions rejected membership in reaktor-graph because the old runtime graph coupled definition to activation. That implementation constraint must be addressed by the shared pure kernel. It is not a reason to create a permanently disconnected data graph. No data consumer should inherit the old runtime's dependency weight just to inspect a schema.

Memgraph is a possible store for product graph data and optionally a server-side index of graph projections. It is not a mandatory remote dependency for loading definitions. A phone can open SQLite while offline using a local projection. A large workspace can page/filter its graph; not every runtime downloads the entire catalogue or every data row.

Topology, product data and observations

The main graph carries three related representations with different owners and access patterns:

Executable connections join typed ports according to a protocol, including multi-party hyperedges. Semantic relations assert facts such as reads, derives, owns or replicates. Local facets annotate a subject with retention, residency, authority or execution requirements. A lineage relation does not by itself execute a transform; a connected port does not grant credentials.

Tag assertions by role: normative requirements/desired state, descriptive source/catalog facts, domain product facts, and observational runtime evidence. Each has source/provenance, subject revision, scope, extraction method and freshness. Preserve conflicts for inspection rather than merging all four through last-writer-wins. Definition revisions and observed activation epochs remain separate.

Physical URNs/locators such as SQL table addresses, graph labels or R2 object prefixes are binding attributes. They cannot be logical entity identity: replacing Postgres with TiDB must not rename every document and service dependency. Credentials and authority-bearing tokens are resolved outside portable snapshots. An opaque entity key does not grant access.

Schema IDs include revisioned generic structure, field identities and compatibility rules; language class names and HTTP route strings are insufficient. Canonical graph fingerprints use the kernel's specified normalized representation, including ordering, numeric encoding and schema version. Arbitrary serializer output or unordered JSON maps cannot stand in for canonicalization.

Observed physical schemas, authored desired schemas and inferred relationships remain separate claims. Reflection cannot infer business authority, ownership, merge laws or deletion intent from a foreign key alone. Conflicts, unknown coverage and stale evidence must remain visible.

Coverage and causal progress

Evidence reports the exact query/window, authority and permissions, returned scope, completeness, source revision/position and observation time. A bounded or permission-filtered query with no rows is not evidence that no rows exist. Confidence or a successful connection does not establish authorization.

A projection proves progress with its source domain, epoch and applied position, or a position vector across sources. A timestamp/HLC alone cannot prove read-your-writes. Federated reads retain each source's position and isolation scope; they do not invent one atomic global cut. Desired bindings, observed placement and evidence can disagree without being silently collapsed.

3. Nodes and typed ports

These are extensible vocabulary families, not a permanently sealed list of all possible data systems.

Services, actors, workflows, FFI functions, principals, runtimes and deployments use the definitions already present in the main graph. Data does not mint duplicate Process nodes for them. Their read/write/subscribe/publish ports connect to typed data/resource ports. Port payload schemas, interaction protocol and effects participate in compatibility; transport labels are only binding information.

Nested graphs express scope/composition. Multiplex edge families express distinct relationships between the same endpoints: a workflow may read, own, archive and authorize access to a dataset, with different laws on each edge. Data relationships are not reduced to one generic arrow.

4. Edges carry semantics

Older Projects/Mirrors names can be aliases for precise materialization/replication relationships during migration. An unqualified “mirror” is insufficient: a full recoverable replica and a lossy search index have different obligations. Symlink-like references must say whether they are aliases, copies or separately authorized external data.

Mappings are typed expression IR where possible, including codecs and key transformations. Native mapping code references a stable operation plus explicit provider requirements and declared dependencies. A string of SQL or arbitrary lambda does not become analyzable merely by being attached to an edge.

Intra-database and inter-database links

Inside Postgres, a References edge may compile to a foreign key/join; a relationship in Memgraph may compile to an edge/traversal. The graph records which native mechanism enforces integrity. JSON references and blob pointers may require application-level checks even within one logical model.

Across stores, use typed entity references plus batched lookup, maintained views or a qualified federation plan. No common transaction/snapshot is assumed. Specify referential lag, dangling-reference repair, deletion behavior and authorization at every hop. A result reports relevant source versions/freshness; a “current” result cannot silently combine a week-old graph projection with newly committed SQL records when the contract forbids it.

5. A worked composition

This is an illustrative descriptor structure, not an implemented Kotlin DSL:

entity Document@3
  identity: (tenant, documentId)
  fields: owner, title, bodyRef, permissions
  authoritative: documents.sql

representation documents.sql
  family: relational; facets: document-json, transaction, changefeed
  binding: postgres.primary@12

representation documents.links
  family: graph; binding: memgraph.knowledge@4
  derives: DocumentCommitted@2 through LinkProjector@5
  checkpoint: source-domain/epoch/position

representation documents.embedding
  facet: vector; binding: postgres.pgvector@2
  derives: bodyRef through Embed@7 using model@2026-08
  policy: authorization filter + declared freshness budget

operation SearchDocuments@4
  reads: permitted Document, documents.links, documents.embedding
  result: DocumentSummary@2
  constraints: bounded candidates/batches, tenant isolation, declared freshness

Embed@7 may bind to a synchronous C++ function locally or an asynchronous model service if its call contract allows that choice. Its data dependencies and model revision are unchanged by the implementation location. An E2EE body cannot be routed to an untrusted server simply because that server has the required vector capability.

Replacing the Postgres host changes a binding revision and qualification. Moving vectors to another engine adds a materialization/replication boundary if it was previously in the same transaction. The graph makes the new lag and failure mode visible before rollout. It does not classify this as a zero-cost connection-string change.

6. Planning, execution, and invalidation

Pure visitors inspect a scoped snapshot and return findings, proposed patches or plans. Executors own authority checks, locks/transactions, RPC, storage, and receipts. Follow the main graph's path:

snapshot/query → proposal → typed patch → normalize/validate
→ fingerprinted plan → authority/policy checks → execute → reconcile → receipt

A plan fingerprint includes relevant graph/schema/operation/binding revisions, authority scope and observed preconditions. Revalidate before applying if these changed. Plans can be stored and reviewed without opening a database connection or invoking a workflow. Execution failures do not mutate a static definition into an unqualified “healthy” claim.

Invalidation must account for query membership. A new row can change “latest 20 documents” without modifying any previously returned row. A permission change can invalidate an otherwise unchanged result. A new analyzer/model/transform revision can invalidate derived output without a source-row edit. Track these dependencies or conservatively invalidate; do not claim precise incremental maintenance from incomplete observations.

7. Producers, validation, and drift

Authored schemas/operations declare intent. Introspection reflects database schemas and provider capabilities. Runtime evidence supplies deployments, checkpoints, lag, failures and observed lineage. These producers contribute claims against shared identities with source ownership and reconciliation rules; one producer does not overwrite another's intent.

Build/deploy validation should catch incompatible schemas/ports, unresolved bindings, unsupported capabilities, missing authority, unsafe key mappings, illegal data movement, unbounded fanout, derivation cycles without a defined fixpoint, missing lifecycle coverage and unavailable target bindings. Capability plus authority plus policy plus effect are different checks.

Runtime drift includes removed indexes, changed schemas, unavailable extensions, missed CDC positions, expired credentials and out-of-band writes. There is no universal “degraded mode is never fatal.” If drift invalidates a safety requirement, the affected operation fails closed or becomes unavailable. Independent local reads may continue where their contract remains valid. Tooling reports the exact affected scope and recovery action.

8. Lifecycle and security

Graph traversal discovers declared obligations; it does not prove all copies have been discovered. Snapshot manifests record native positions, blob versions/checksums, schema and transform revisions, keys and dependency closure. Coordinated snapshots or replay protocols establish recovery consistency. HLC alone does not.

A delete plan distinguishes authoritative data, searchable derivatives, broker retention, caches, offline copies, archives and backups. Ownership policy determines cascade/restrict/detach. Execution receipts state completed, pending, retained, unsupported and unknown coverage. Deletion fences prevent old events/backups from republishing erased data. Do not treat dropping a metadata vertex as deleting actual data.

Data graph metadata can itself be sensitive. Apply scoped queries, redaction and permissions to schema names, relationship topology, entity identifiers, samples and runtime evidence. Encryption domains constrain where transforms may see plaintext; key references do not contain secret material. Policy revisions participate in cache and plan validity.

9. Size, overhead, and acceptance

The design requires no global metadata server on the request path. Compile/cache plans and bindings; retain scoped local projections; load detailed lineage on demand. Avoid one static node per application row or invocation. Large entity graphs, logs and metrics remain in suitable stores and expose qualified queries into the main graph view. Measure actual descriptor size and runtime overhead instead of promising a fixed kilobyte footprint.

Acceptance requires identity parity between direct module use and graph installation, deterministic serialization, typed mapping/port checks, bounded planner output, actual enforcement at runtime boundaries, and crash/recovery tests for the mechanisms consuming plans. Include out-of-band writes, stale schema evidence, tenant revocation, inserted-row cache invalidation, partial backup closure, and interrupted migrations.

The value of this graph is that the same semantic relationships can drive composition and operational plans throughout Reaktor. Its limit is the accuracy of those relationships and the mechanisms implementing them. A useful graph makes missing guarantees explicit before they become production assumptions.