Reaktor Analytics and Telemetry

Reaktor needs one signal model for framework telemetry, app product analytics, performance vitals, worker/server health, release correlation, and agent-visible insights. Reaktor apps like BestBuds should define product events in Kotlin, emit them through Reaktor runtime APIs, store them through provider-specific sinks, and surface them back in Reaktor Desktop as graph-linked facts.

• Owner Reaktor framework + app teams

• First app BestBuds

• Runtime modules — reaktor-telemetry reaktor-performance

• App modules analytics event taxonomy + sink wiring

• Current-direction reconciliation 2026-10-01; earlier catalog design retained below

• Product analytics — events / funnels / cohorts

• Runtime telemetry — traces / metrics / logs

• Performance — vitals / budgets / profiles

• Attribution — graph.id + node.id

• Storage — existing ClickHouse / OTel-Grafana; proposed WAE + R2/Iceberg for Shadow

• IDE — Reaktor Insights

• Agents — MCP queries

• Safety — schema + privacy lint

1. Architecture call

The goal is not to add another analytics SDK. The goal is a durable signal contract that spans Reaktor itself and every Reaktor app. A screen, service, actor, repository, worker, release, experiment, and cloud resource should be able to answer: what events can this node emit, what telemetry describes it, where does that data land, what is the retention policy, and which dashboard or MCP query should inspect it?

2. Analytics vs telemetry

Use precise words. "Analytics" and "telemetry" overlap operationally, but they answer different questions and need different governance.

BestBuds already has product analytics through BestBudsEvents, HttpAnalyticsSink, a sink adapter, and a Cloudflare Worker ingest path into ClickHouse. Reaktor has telemetry primitives in Feature.Telemetry, GraphTelemetry, OpenTelemetry-KMP and reaktor-performance, but the BestBuds app installs neither Feature.Telemetry nor GraphTelemetry: it propagates trace context on service calls, and its backends record the spans. The missing layer is a shared contract that makes these signals typed, queryable, graph-linked, and visible in Reaktor Desktop.

3. Current state

Reviewed against the current working tree on 1 October 2026. Operations & Reliability owns the complete operating map; earlier sections below retain product-catalog and schema design.

ClickHouse is retained for product analytics and Reaktor service evidence. OTel remains the runtime signal model. Product events keep their app-owned catalog; a Shadow observation/run/journal is an experiment record with state, assertion and coverage semantics, not a replacement telemetry framework. See storage and delivery contracts.

Firebase service boundaries — 7 October 2026

Product analytics use SinkAnalyticsAdapter → HttpAnalyticsSink → the analytics Worker → ClickHouse. Firebase Analytics dependencies, native adapter, iOS pod declarations and browser stub have been removed from maintained source. Ship rebuilt clients to complete the runtime cutover; existing installed builds keep their old SDKs.

FCM, App Distribution and Crashlytics remain the intended Firebase services. Crashlytics is retained through launch; Firebase Performance is a separate, qualified timing integration. Firebase Analytics stays removed, so breadcrumbs are explicit. Release acceptance covers Android R8/native symbols, iOS app/Kotlin-framework dSYMs, fatal/native and handled-error readback, grouping, environment isolation and tested alerts. Crashlytics release qualification defines the checks. The legacy debug Measure integration is pending removal; no hosted Measure installation or dual run is planned.

The target kernel joins Crashlytics evidence to Reaktor application sessions, graph, release and explicit operation traces. Minimum correlation and journal transport precede launch; full Shadow investigation follows it. Existing DevTools hooks persist managed exceptions for the next handshake; the iOS hook does not capture Objective-C exceptions or Unix signals. Its coexistence with the retained crash reporter needs a physical-device qualification. See capture fabric.

4. Unified signal model

This earlier envelope is retained as a product-catalog design sketch. Runtime traces/metrics/logs use OTel directly (§17.2); Shadow experiment records follow the paired-observation contract. Workbench Fact provenance relates these records without translating every signal into a new runtime envelope.

data class ReaktorSignal(
    val id: String,
    val name: String,
    val kind: SignalKind,        // ProductEvent, TraceSpan, Metric, Log, Vital, Audit
    val timestampMs: Long,
    val producer: SignalProducer, // app, worker, server, engine, tool, agent
    val context: SignalContext,
    val attributes: Map<String, SignalValue>,
    val privacy: PrivacyClass,
    val schemaVersion: Int,
)

data class SignalContext(
    val appId: String?,
    val environment: String,
    val graphId: String?,
    val nodeId: String?,
    val routePattern: String?,
    val releaseId: String?,
    val sessionId: String?,
    val traceId: String?,
    val userHash: String?,
)

Provider mapping is then deterministic:

5. Event catalog and schema governance

Every Reaktor app should own an analytics catalog in Kotlin. Constants are a good start, but not enough. The catalog should declare event names, parameters, privacy class, owner, lifecycle state, allowed screens/nodes, retention, and downstream purpose.

val bestbudsAnalytics = reaktorAnalytics("bestbuds") {
    event("screen_view") {
        owner("app-platform")
        purpose("navigation analytics and route health")
        retentionDays(90)
        param("screen", StringType, required = true, privacy = NonPii)
        param("session_id", StringType, required = true, privacy = Pseudonymous)
        mapsToNode("RouteNode")
    }

    event("chat_message_sent") {
        owner("messaging")
        purpose("engagement and reliability")
        forbid("message_body")
        param("length_bucket", EnumType("0", "1-9", "10-49", "50-199", "200-499", "500+"))
        param("has_media", BooleanType)
        param("message_kind", StringType)
    }
}

The catalog should materialize four things:

• Runtime constants and typed helper functions for app code.
• Validation for emitted events in debug builds and tests.
• Generated docs and Reaktor Desktop catalog panels.
• Warehouse schema, rollup definitions, dashboard seeds, and MCP schema.

6. Graph attribution

The differentiator is not that Reaktor can emit events. Every app can emit events. The differentiator is that signals attach to graph nodes.

RouteNode:ChatScreen
        |
        | emits product event
        v
analytics:screen_view(screen=chat)
        |
        | same session / trace / release
        v
ServiceNode:MessagingService --span--> Worker:messaging-service --query--> Postgres/Hyperdrive
        |
        | correlated by release and route
        v
Insight:chat_send_conversion_drop_after_release

Every graph node can expose a signal panel:

7. Reaktor runtime API

The runtime should stay small. Apps choose sinks for product analytics; Reaktor owns contracts and common emitters. For runtime and performance telemetry the provider-neutrality below is superseded by §17.2, which makes OpenTelemetry the internal model as well as the transport.

Core interfaces

Expected usage in an app

val analyticsSink = HttpAnalyticsSink(
    userProvider = { currentUserOrNull() },
)

Reaktor.start {
    Analytics = SinkAnalyticsAdapter(listOf(analyticsSink))
    Telemetry = TelemetryAdapter(
        controller = platformController,
        otel = createReaktorOpenTelemetry(bestbudsTelemetryConfig)
    )
}

TrackScreenView("chat", mapOf("chat_id" to chatId))

Provider code should not leak into every screen. Screens emit app-owned helpers. Runtime graph code emits graph telemetry. The catalog reconciles both.

8. BestBuds app implementation

BestBuds already demonstrates the desired app pattern.

BestBuds event families

9. Ingestion path

The Worker path inserts product events into ClickHouse. Android and iOS send events only through the HTTP sink; Firebase Analytics has been removed. The earlier Postgres-first diagram is superseded:

App event helper → Feature.Analytics
    └─ HttpAnalyticsSink → bounded pending/batch path
          → analyticsServer Worker → ClickHouse analytics_events

Client delivery and server ingestion must expose drops, retries, rejected batches, oldest pending age and last successful delivery. The ingest owner validates app/tenant/principal, catalog schema, size and rate limits. Product-event append and downstream rollups remain separate from recommendation scoring. Check the current application configuration for the actual endpoint rather than copying a historical URL.

Runtime OTel follows the existing shared signal pipeline; Shadow paired observations and replay journals follow their own evidence contracts. These paths share identity and governance without claiming one atomic write.

10. Storage, rollups, and recommendation data

ClickHouse already serves BestBuds analytics and stays in place. Crashlytics has its provider-owned lifecycle; Reaktor journals and Shadow evidence have separate schemas and retention. Historical Measure data is preserved during migration. These records must not share product analytics tables merely because a storage engine is available.

Materialized layers

Existing ClickHouse aggregation/retention facilities remain available through the Data Layer. WAE serves proposed hot Shadow aggregates, not arbitrary warehouse joins or the exact replay oracle. R2/Iceberg adds historical evidence; it does not require replacing ClickHouse or moving an existing consumer. Choose any additional serving path from measured query/cost needs, with its consistency and deletion contract explicit.

11. Privacy and safety model

Analytics becomes dangerous when it is easy to emit anything. Reaktor should make safe emission the default and unsafe emission hard to merge.

12. Reaktor Desktop and MCP

The Desktop Insights surface should become a live graph-linked analytics and telemetry workbench, not a dashboard iframe collection.

Desktop panels

MCP tool families

analytics_catalog(appId, includeDeprecated = false)
analytics_query(event, range, groupBy, filters, aggregation)
funnel_query(funnelId, range, segment)
telemetry_trace_query(nodeId, range, errorOnly = false)
signal_health(appId, sink, range)
privacy_lint(appId, changedFiles)
release_signal_diff(beforeRelease, afterRelease, nodes)

Agents should get aggregate analytics and trace summaries by default. Raw event reads, user-level exploration, retention changes, and sink mutations should be privileged operations with audit records.

13. Quality gates

Signal quality must be tested like code quality. Otherwise the catalog will drift and dashboards will become fiction.

14. Phased plan

Phase 1Catalog

Promote BestBudsEvents into a typed catalog

Add event metadata, param schema, owner, privacy class, retention, and graph-node mapping. Generate constants from the catalog or validate the existing constants against it.

Phase 2Validation

Make invalid analytics impossible to merge

Add lint/test gates for unknown events, unknown params, PII-like fields, high-cardinality labels, and missing retention. Wire the gate into Dagger and local Gradle checks.

Phase 3Telemetry

Standardize OTel attributes and graph links

Update GraphTelemetry, service interceptors, DB adapters, worker wrappers, and performance collectors to emit graph.id, node.id, release, environment, and trace context.

Phase 4Ingest

Harden BestBuds analytics ingest

Add multi-row insert, rate limits, request size limits, structured errors, worker metrics, dead-letter export, custom domain, and dashboarded queue health.

Phase 5Desktop

Make signals graph-visible in Reaktor Desktop

Build authenticated funnel, signal-health, trace, alert/SLO and privacy providers on the current source-derived catalog. Add node-level insight tabs and MCP aggregate query tools; keep unobserved health explicitly unknown.

Phase 6ML

Use analytics as a recommendation substrate

Build daily rollups, trait vectors from onboarding metadata, affinity vectors from behavior, campaign vectors, Discover ranking, R2 Parquet export, and quality dashboards.

15. Repo implementation map

16. Final shape

The target state is a signal-aware Reaktor IDE:

Kotlin analytics catalog
  -> constants / validators / docs / dashboards / MCP schema
        |
        v
Reaktor app runtime
  -> app ingest + OTel + performance collector
        |
        v
Storage backends
  -> ClickHouse / OTel-Grafana / proposed R2 evidence
        |
        v
Reaktor Engine graph
  -> node insights / funnels / trace links / privacy lint
        |
        v
Humans and agents
  -> aggregate query / compare release / diagnose regression / plan fix

17. Graph-native telemetry

Designed 13 September and reconciled 1 October 2026. This section supersedes the model in §4 and the provider-neutrality stance in §7 for runtime and performance telemetry. Product analytics keeps its sink-flexible design unchanged — the four signal classes in §2 are still four classes.

17.1 Telemetry is a derived layer over the graph

The load-bearing decision: no SpanNode, MetricNode or LogNode is ever created in the application graph. Millions of telemetry nodes would destroy the property that makes the graph useful — that it is a stable, comprehensible description of what can connect.

Three related structures instead:

              G — ARCHITECTURE GRAPH          "what can connect?"
              stable topology, stable ids
    ChatScreen.sendMessage ──────▶ ChatService.sendMessage

    ═══════ M(g, Δt) — TELEMETRY OVERLAY ═══════  "how has this behaved?"
        calls/s 42 · p95 84ms · errors 0.3%

    ═══════ Tᵣ — EXECUTION GRAPH ═══════════════  "what happened this time?"
        SendMessage → ChatScreen.send → ChatInteractor.send
          → ChatService.sendMessage → HTTP → Worker → Postgres

with φ : Tᵣ → G mapping every span back to the node, port or edge responsible for it.

Every Desktop view then becomes a projection rather than its own subsystem:

This is why the overlay belongs here and not in the workbench: five panes currently want their own metrics model, and they should all read one.

17.2 Decision — OpenTelemetry is the internal model

Reaktor fixes on OpenTelemetry for runtime and performance telemetry. Shadow manifests, journals and paired observations carry experiment semantics alongside OTel; they are not new trace/metric/log models. Existing ClickHouse consumers stay. See Graph evidence and Shared observability. There is no vendor-neutral internal signal model, no TelemetrySignal, no TelemetryContext, and no separate reaktor-telemetry-otel mapping module. OTel already specifies traces, metrics, logs, context propagation, semantic conventions, span links, baggage and the Collector pipeline; re-modelling those in Reaktor types buys portability nobody has asked for and costs a translation layer at every boundary.

What this deletes from the design as originally proposed:

Reaktor's contribution is a semantic convention. That is the smaller and more durable thing to own, and it leaves the data model to a specification with an ecosystem behind it.

Use opentelemetry-java where it reaches. On JVM and Android the reference implementation covers the SDK, OTLP exporters, W3C propagators, generated semantic conventions, views, aggregation and cardinality limits, plus JDBC instrumentation for D10. Adopt it directly there.

The risk, stated plainly. io.opentelemetry.kotlin is the KMP API and it is experimental — GraphTelemetry already carries @OptIn(ExperimentalApi::class). Fixing on OTel couples the framework's telemetry surface to that API's stability across Android, iOS, JS, Workers and native. This is accepted deliberately, and the exposure is smaller than it first appears: with opentelemetry-java serving JVM and Android, the experimental API only has to carry iOS, JS and Workers, where no alternative exists. The coupling stays confined to reaktor-telemetry, and the facet and visitor layer above it names no OTel type.

17.3 The reaktor.* semantic convention

Every span, metric and log carries the architectural identity that produced it, so a trace collected six months later still resolves against the graph:

reaktor.graph.id      = bestbuds
reaktor.node.id       = ChatService
reaktor.port.id       = sendMessage
reaktor.port.shape    = request
reaktor.edge.id       = ChatInteractor→ChatService
reaktor.contract.id   = chat.send-message.v2
reaktor.snapshot.id   = git:a41302
reaktor.activation.id = <runtime only, never a metric dimension>

The separation that makes this work is the one rgp/1 §3 already enforces: stable source and deployment identity is not runtime activation identity. reaktor.snapshot.id and reaktor.node.id identify the source but are not automatically bounded metric dimensions; reaktor.activation.id is per-process and belongs on spans only (§17.8).

17.4 Instrument the port call itself

The interceptor chain and typed attachments now exist. PortTelemetryInterceptor and TelemetryPlanVisitor use supported port semantics and install/uninstall owned hooks. The previous inline-call prerequisite is complete for supported lambda-form invocations; raw-reference handoffs still do not intercept every later call.

Current semantics include Request, Mailbox, Stream, State and Reference. Duration/outcome are observed at the intercepted boundary; queue depth, subscriber lag and other holder state require actual probes. Reference is an explicit coverage boundary. The old 154/76 call-site count was a September inventory, not a current coverage denominator.

Telemetry modes and capture scope remain bounded and versioned. Graph-selected journal capture additionally needs operation descriptors/codecs, policy, pre-state and ordering evidence. Multiple layers can observe an effect, but one boundary owns replay substitution.

17.5 TelemetryFacet declares policy, never data

The current telemetry module has a policy type with this role; the following earlier DSL sketch is design intent, not the exact shipped API.

data class TelemetryFacet(
    val tracePolicy: TracePolicy = TracePolicy.Inherit,
    val metricsPolicy: MetricsPolicy = MetricsPolicy.Inherit,
    val logPolicy: LogPolicy = LogPolicy.Inherit,
    val payloadPolicy: PayloadPolicy = PayloadPolicy.None,
    val sampling: SamplingPolicy = SamplingPolicy.Inherit,
    val slo: SloSpec? = null,
)

Inheritance runs Graph → Module → Node → Port/Edge, so the common case writes almost nothing:

graph.telemetry { standard() }

paymentPort.telemetry { detailed(); payloads = None }
animationFramePort.telemetry { traces = Off; metrics = AggregateOnly }

The facet holds policy. It never holds a collected value. payloadPolicy defaults to None (§17.11), and TracePolicy.Off on a hot path is a first-class answer.

17.6 TelemetryVisitor compiles a TelemetryPlan

Instrumentation is derived from the graph by a compiler pass, not scattered through node code. This is the cheapest part of the design to build, because reaktor-graph-port/visitor/Visitor.kt already ships the mechanism: Selector, Traverser, and a PortGraphVisitor whose visitGraph / visitNode / visitConsumerPort / visitProviderPort / visitEdge each return an ExitScope — enter/exit semantics, which is useful for selecting instrumentation. Traversal scope is not runtime-call scope: installed interceptors own call timing and teardown. The current implementation is TelemetryPlanVisitor; HierarchyVisitor is an example of traversal.

PortGraph ──▶ TelemetryVisitor ──▶ TelemetryPlan
                                     ├── port instrumentation (by shape)
                                     ├── transport instrumentation
                                     ├── metric instruments
                                     ├── sampling rules
                                     ├── redaction rules
                                     └── propagation rules

The plan is computed once per revision and is itself a fact the workbench can display: this is what is instrumented and why, answerable without reading code.

17.7 ExecutionContext and causality across boundaries

data class ExecutionContext(
    val otel: Context,           // OTel Context — carries trace and baggage
    val deadline: Deadline?,
    val cancellation: Cancellation?,
    val auth: AuthContext?,
    val request: RequestContext?,
)

Because OTel is fixed (§17.2), the trace and baggage slots are io.opentelemetry.context.Context rather than Reaktor types wrapping it, and propagation is W3C traceparent / tracestate via OTel's own propagators. Nothing here is reimplemented.

Application APIs never take this parameter. It rides the coroutine context locally and is attached by the transport adapter at boundaries:

Reaktor ExecutionContext → TransportAdapter → traceparent/tracestate
    → HTTP / WebSocket / Queue / RPC →
TransportAdapter → Reaktor ExecutionContext

so the intended Android → Worker → service → queue → worker → Postgres path can be represented as one causal execution. This context contract and end-to-end propagation require qualification; the current local SDKs alone do not prove that path is connected.

Asynchronous causality is not a call stack. For mailboxes, queues and pub/sub, span links are the default, which is what OTel's messaging conventions recommend:

Actor A.send() ──SpanLink──▶ mailbox ──▶ Actor B.process()

Corrected 13 September 2026. An earlier version justified this by claiming a parent-child relationship asserts that the child's work happened inside the parent's duration. It does not — parentage carries no temporal containment guarantee, and OTel explicitly permits a message creation context as the parent of a process span for single-message processing. Links remain the default because they handle batches and ambient contexts correctly; document the single-message option alongside them. Render an execution as a causal graph that may cross trace ids, and test batching, fan-out, retries and delayed consumption. Never expand a producer span to encompass asynchronous processing.

ExecutionContext is bigger than telemetry — deadline and cancellation propagation are worth having on their own — and that is an argument for building it early, not for scoping it here.

17.8 Cardinality is enforced, not advised

OTel SDKs apply a default limit of 2,000 attribute combinations per metric stream because every unique combination costs aggregation state. Reaktor should catch the violation at authoring time, not at the collector.

TelemetryValidator runs in the same pass as the visitor:

⚠ High-cardinality telemetry attribute
  userId is unbounded.
  Suggested: traceAttribute(userId)
  Instead of: metricAttribute(userId)

Corrected 13 September 2026: stability is necessary and not sufficient. Stable ids prevent per-request churn; they do not bound how many ids exist or how their combinations multiply. Node × port × contract × snapshot × status can exceed a stream's limit on its own, and snapshots accumulate with every release. Rejecting a handful of attribute names cannot prove a runtime cardinality bound.

So the contract is quantitative. Specify the allowed dimensions and the projected cardinality per instrument; configure OTel views and aggregation to enforce it; keep runtime overflow counters so a breach is visible; and hold high-detail identity on spans or in separately bounded diagnostic series. Resource and release correlation is a distinct concern from unrestricted metric labels. On JVM and Android opentelemetry-java already implements views, aggregation and cardinality limits, so most of this is configuration. Test with many architectural subjects across multiple releases, rather than only asserting that forbidden names are rejected.

17.9 Two telemetry paths

                    port invocation
              ┌────────────┴────────────┐
       DEVELOPMENT PATH            PRODUCTION PATH
       full fidelity               sampled
       ring buffer                 batched
       local WebSocket             OTLP
       payload inspection          redacted
              ▼                         ▼
       Reaktor Desktop            OTel Collector

The development path can show captured invocations independently of the remote backend. Bounded buffers, raw escapes and drop accounting limit coverage; it must not claim every invocation without evidence. The development path is the same carrier the tooling layer already uses for L4 facts, so this is one socket, not two.

17.10 Capture is a graph command

The graph already knows exactly where instrumentation should be activated, which makes capture a targeted, reversible operation rather than a config change:

CaptureTelemetry(target = node("ChatService"), mode = Detailed, duration = 30.seconds)
CaptureTrace(path = graph.path(from = screen("ChatScreen"), to = repository("MessageRepository")), sampleNext = 10)

Policy override, bounded duration, automatic revert. In the workbench this is a right-click on a slow node: capture next invocation · next 10 · errors only · slower than 500ms · this session. It goes through the same prepare/approve/execute path as every other write (tooling layer §14), so a capture leaves a receipt.

17.11 Privacy is graph policy

Payload values are off by default. Telemetry normally sees the shape, never the content:

schema = SendMessageRequest · size = 481 bytes · result = Success

Schema fields carry classifications the visitor compiles into redaction rules:

data class User(
    @Telemetry.PII  val email: String,
    @Telemetry.Safe val accountType: AccountType,
)

Baggage gets the strictest handling of anything here. It propagates across service boundaries and can reach third parties; W3C is explicit that personally identifiable information must not go into tracestate. Treat baggage as an allowlist, validated in the same pass as cardinality, and default it empty.

17.12 Telemetry must not instrument itself

emit telemetry → TelemetryPort invocation → emit telemetry about TelemetryPort → …

Either mark the entire telemetry plane NoTelemetry, or give it a kernel-level internal channel that bypasses ordinary graph instrumentation. Preference: the second, because a marker can be forgotten when someone adds a node, and a separate channel cannot. Note that this is distinct from telemetry self-monitoring (dropped spans, queue depth, exporter health), which is required — it simply must not travel through the instrumented path.

17.13 Implemented substrate and remaining prerequisites

The 13 September missing-kernel table is obsolete. Reviewed source on 1 October contains the following substrate:

The kernel remains the owner of graph identity and invocation mechanisms; telemetry owns signal policy and SDK integration. Shadow reuses that substrate and the existing authority/receipt path. Instrumentation Plan tracks remaining delivery gates.

17.14 Module layout

reaktor-graph / reaktor-graph-port
    stable entity ids · port shapes · invocation hook · visitor (exists)
reaktor-telemetry
    TelemetryFacet · TelemetryVisitor · TelemetryPlan · TelemetryValidator
    TelemetryRuntime · sampling · redaction
    OTel SDK install · W3C propagators · OTLP exporter   ← not a separate module (§17.2)
runtime adapters
    JVM · Android · Darwin · Browser · Workers
Reaktor Desktop
    traces · waterfall · live edges · heatmaps · capture commands

The rule that keeps this honest as it grows: facets declare what should be observed, visitors compile the instrumentation, execution contexts carry causality, OpenTelemetry transports and models the data, and every signal projects back onto a stable graph entity. Telemetry is not an APM bolted onto Reaktor — it is runtime evidence attached to the same typed graph that describes the system.

Related docs

• Operations & Reliability — current reading map, source inventory and Shadow textbook.

• Perf and Observability - Use before optimizing or adding instrumentation.

• Graph Visitors - Use before designing a new graph pass or selector.

• Unified Graph - Use before building Reaktor Engine graph importers, cloud/resource nodes, operation providers, or MCP graph tools.

• Data Layer - Use before choosing a store, designing offline-first sync, or deciding authority vs cache vs projection for a datum; the centerpiece is the tier 1↔2/3 server-authoritative sync engine.

• Auth Architecture - Use before changing reaktor-auth or request authorization boundaries.

• Server - Use before adding any production backend primitive (resilience, messaging, data patterns) or designing the Spring/Workers server runtime.