Reaktor Cloud Control Plane

Operations integration, 1 October 2026. Fleet provisioning is a dependency of Shadow workspace preparation, with verified resource destinations, authority, expiry and cleanup receipts. Workspaces share observability infrastructure while isolating application state/effects. Reconcile existing installed collectors with provisioning definitions before reuse; this design does not authorize a duplicate stack.

Desktop implementation status · 30 September 2026: the Cloud pane now reads Cloudflare, Google Cloud and the cluster live, joins them with the repository's declarations, and reports health, drift and findings. What it does and how is in Reaktor Cloud; this page stays the design direction for cells and fleets.

The Cloud pane is the operator view over Reaktor Cloud. The cloud model underneath it is deliberately narrower and stronger than a generic multi-cloud abstraction:

  • GCP is the infrastructure provider. Reaktor can still deploy Workers, talk to Supabase, publish to Pub/Sub, and link to Grafana, but the fleet substrate for VMs, k3s, regional capacity, and customer-owned clusters is GCP.
  • A cell is the unit of placement. A cell is one regional k3s cluster plus the GCP project, node pools, namespaces, secrets, observability, and routing needed to run a slice of the graph.
  • Clusters do not form one stretched cluster. Each region or customer boundary is its own cell. Cells interact through the Reaktor control plane, service calls, the bus, mesh/realtime edges, and observability streams.
  • Consumers get both commercial options. They can choose location and capacity and let Reaktor create/manage the GCP VMs with billing flowing through Reaktor, or they can bring their own GCP account/cluster and let Reaktor manage it under scoped authority.
  • Customer-owned clusters are first-class. A tenant can attach its own GCP project or k3s cluster to Reaktor Cloud. The customer pays GCP directly for the VMs; Reaktor charges for the control plane, convenience, operations, and automation.
  • The attachment point is reaktor-agent. The agent runs in each cell, dials outbound to the control plane, reports inventory and health, and applies signed operation plans under scoped Kubernetes and GCP permissions.

1. The Core Decision

Reaktor Cloud is not "support every cloud." It is:

One Reaktor control plane
  -> many GCP cells
      -> some Reaktor-owned
      -> some customer-owned
      -> each running k3s and a scoped reaktor-agent

That keeps the platform understandable. The control plane owns identity, policy, graph operations, deployment intent, run history, audit, and reconciliation. The cell owns execution: pods, VMs, local Kubernetes state, service endpoints, metrics, and regional capacity.

There are two consumer-facing deployment models:

OptionWho owns/pays GCPWhat the consumer choosesWhat Reaktor manages
Reaktor-managed GCP cellReaktor owns the GCP account/project and bills the customer through ReaktorRegion, capacity class, isolation tier, budget/SLAVM creation, k3s, node pools, upgrades, workloads, observability, incident operations
Customer-owned GCP cellCustomer owns/pays their GCP account directlyGCP project/account, region, allowed scopes, management modeAgent install/registration, workload deploys, drift detection, operations within delegated policy

Both options produce the same CloudCell shape and the same graph operations. The difference is billing owner, IAM root, and how much authority the customer delegates.

The current single-VM setup is just the seed cell:

bestbuds-prod-asia-south1
  GCP Compute VM
  k3s server
  Spring app server
  Memgraph
  ClickHouse / observability pieces as they land
  Cloudflare Tunnel

Scaling this should not mean stretching that k3s cluster across the world. Scaling means creating or attaching more cells and placing tenants, workloads, actors, and data in the right cells.

2. Control Plane And Cells

The model has two ownership planes.

PlaneOwnerWhat lives thereReaktor responsibility
Control planeReaktorTenant registry, cell registry, policy, auth, service catalog, graph operations, operation ledger, signed plans, audit, billing for Reaktor convenienceDecide what should run where, authorize operations, record facts, reconcile drift
Cell planeReaktor or customerGCP project, VM/node pools, k3s cluster, namespaces, pods, local secrets, telemetry exporters, regional ingress/tunnelExecute signed plans, report observed state, enforce scoped permissions

The control plane never needs a broad credential that can mutate every customer's cloud directly from a developer laptop. It talks to agents and provider adapters through centralized interceptors, service accounts, and scoped grants.

2.1 Cell Shape

@Serializable
data class CloudCell(
    val id: String,
    val owner: CellOwner,              // ReaktorOwned | CustomerOwned(tenantId)
    val billing: CellBilling,
    val provider: CloudProvider = CloudProvider.GCP,
    val gcpProjectId: String,
    val region: String,
    val cluster: K3sClusterRef,
    val agent: AgentRef,
    val mode: CellMode,
    val status: CellStatus,
    val capacity: CellCapacity,
    val labels: Map<String, String> = emptyMap(),
)

@Serializable
sealed interface CellOwner {
    data object ReaktorOwned : CellOwner
    data class CustomerOwned(val tenantId: String) : CellOwner
}

@Serializable
sealed interface CellBilling {
    data class ReaktorManaged(val planId: String) : CellBilling
    data class CustomerDirectGcp(val billingAccountHint: String? = null) : CellBilling
}

@Serializable
enum class CellMode {
    Observe,   // inventory, health, drift only
    Deploy,    // apply app workloads
    Capacity,  // add/drain nodes inside approved bounds
    Managed,   // Reaktor can operate the full cell within policy
}

The same type covers:

  • shared Reaktor cells for small/default tenants;
  • dedicated Reaktor-owned cells for higher isolation;
  • Reaktor-managed paid cells where the customer chooses region/capacity and pays Reaktor;
  • customer-owned GCP cells for clients who want the GCP bill and legal control in their account;
  • regional cells used for latency or residency.

2.2 How Clusters Interact

Clusters interact through explicit planes, not by pretending they are one cluster.

Interaction planeWhat crosses cell boundariesMechanism
Controlinventory, desired state, signed operation plans, run events, drift reportsreaktor-agent outbound channel to Reaktor Cloud
Servicetyped request/response calls between servicesreaktor-service clients with staged interceptors, service auth, tracing, and routing
Busdurable facts, jobs, fan-out, retriesGCP Pub/Sub / future Reaktor Bus envelopes with tenant and cell context
Mesh/realtimelive session data, peer state, WebSocket setupReaktor Mesh / realtime token exchange, with connection setup interceptors
Trafficuser ingress and service ingressCloudflare Tunnel, GCP load balancing, gateway routing, cell-aware selectors
Observabilitymetrics, traces, logs, health, cost factsOTel and Grafana/ClickHouse with bounded tenant/cell labels

This is the important rule: a pod in one cell does not reach into another cell's Kubernetes API. It calls a service, publishes a fact, or asks the control plane for a gated operation.

2.3 Same Region vs New Location

There are two different scaling verbs:

NeedOperationResult
More capacity in an existing locationCloud.AddNode / Cloud.ResizeNodePoolAdd a GCP VM as a k3s agent node, rebalance workloads, update capacity facts
Lower latency, residency, or tenant isolation in a new locationCloud.CreateCell / Cloud.AttachCellNew regional cell with its own cluster, agent, policies, and placement rules

Adding VMs is a capacity operation inside a cell. Adding locations or ownership boundaries is a cell operation.

3. Reaktor-Managed GCP Cells

The default commercial model can stay simple: the customer chooses where and how much capacity they need, and Reaktor provisions the GCP resources.

Customer chooses:
  region = asia-south1
  capacity = 3 x e2-standard-4
  isolation = dedicated cell
  budget/SLA = plan limit

Reaktor does:
  create/resize GCP VMs
  install/upgrade k3s
  attach reaktor-agent
  deploy workloads
  operate incidents and rollbacks
  bill the customer through Reaktor

This is the fastest path for consumers who want convenience over cloud-account ownership. It also gives Reaktor the most complete operating envelope: one IAM model, one billing relationship, one upgrade path, one fleet baseline.

The customer still gets control-plane visibility:

  • region and capacity selection;
  • cost estimate and monthly budget guardrails;
  • workload placement and isolation tier;
  • operation history and audit;
  • export path to customer-owned GCP later if the contract requires it.

4. Customer-Owned GCP Cells

Yes: customers should be able to connect their own GCP clusters to the Reaktor Cloud control plane.

The business model is clean:

  • customer pays GCP for the VMs, disks, network, and regional resources;
  • Reaktor bills for the control plane, automation, deployment experience, observability, policy, fleet operations, and support;
  • the customer can revoke the connection without asking Reaktor to export their infrastructure.

This is the strongest multitenancy tier. It is not just "tenant_id in a shared database"; it is an infrastructure ownership boundary.

3.1 Attachment Flow

1. Customer creates or selects a GCP project.
2. Reaktor creates a CloudConnection request with required scopes and region.
3. Customer grants GCP access through a least-privilege service account or Workload Identity Federation.
4. Customer installs reaktor-agent into the k3s cluster, or lets Reaktor bootstrap k3s if the mode permits it.
5. Agent dials outbound to Reaktor Cloud and presents a registration token.
6. Control plane verifies project, region, cluster identity, tenant policy, and agent attestation.
7. Cell becomes Observe mode first.
8. Customer explicitly upgrades to Deploy, Capacity, or Managed mode.

The initial mode is always Observe. That gives the customer and operator a read-only inventory and drift view before any mutating operation is possible.

3.2 Connection Model

@Serializable
data class GcpCellConnection(
    val id: String,
    val tenantId: String,
    val gcpProjectId: String,
    val region: String,
    val clusterName: String,
    val billingOwner: BillingOwner = BillingOwner.CustomerDirectGcp,
    val auth: GcpConnectionAuth,
    val allowedModes: Set<CellMode>,
    val status: ConnectionStatus,
)

@Serializable
sealed interface GcpConnectionAuth {
    data class WorkloadIdentityFederation(val providerResource: String) : GcpConnectionAuth
    data class ServiceAccountImpersonation(val serviceAccountEmail: String) : GcpConnectionAuth
    data class BootstrapToken(val secretRef: String) : GcpConnectionAuth // short-lived install only
}

@Serializable
enum class BillingOwner {
    ReaktorManaged,
    CustomerDirectGcp,
}

Prefer keyless identity. Long-lived service-account JSON should be treated as a temporary bootstrap fallback, not the normal state.

3.3 Operating Modes

ModeReaktor can doCustomer retains
ObserveRead inventory, health, versions, drift, cost hints, logs the customer permitsAll mutation rights
DeployApply Reaktor workloads into approved namespaces, roll out/rollback apps, run migrations under policyCluster lifecycle and node capacity
CapacityAdd/drain/resize approved node pools inside budget/location limitsProject ownership and hard budget controls
ManagedFull day-2 operation inside the agreed policy: upgrades, patching, scaling, incident actionsBilling ownership, IAM root, revocation right

Mode is part of TenantPolicy and CloudCell, not an ad hoc flag in deployment scripts.

4. reaktor-agent

reaktor-agent is the only component that must run inside every cell.

It should be boring and narrow:

  • dial outbound to the control plane; no public inbound Kubernetes API required;
  • register the cell and renew agent identity;
  • report observed Kubernetes/GCP/resource state;
  • stream bounded health, metrics, events, and drift facts;
  • pull signed operation plans;
  • apply plans only when the plan signature, tenant, cell, mode, and policy match;
  • run with scoped Kubernetes RBAC and minimal GCP permissions;
  • write every mutation to the operation ledger.
@Serializable
data class AgentRegistration(
    val agentId: String,
    val cellId: String,
    val tenantId: String?,
    val clusterUid: String,
    val gcpProjectId: String,
    val region: String,
    val capabilities: Set<AgentCapability>,
    val publicKey: String,
)

@Serializable
data class SignedCellPlan(
    val id: String,
    val cellId: String,
    val tenantId: String?,
    val operation: String,
    val safetyClass: SafetyClass,
    val expiresAtEpochSeconds: Long,
    val payload: ByteArray,
    val signature: ByteArray,
)

The agent is an executor, not the source of truth. The source of truth is the signed desired state plus the observed facts the agent reports back.

5. Operations

Every cloud action is a typed graph operation. The pane, CLI, agents, and service APIs all call the same operations through the same interceptor chain.

OperationPurposeSafety
Cloud.InventoryList cells, projects, clusters, node pools, workloads, costs, and driftRead
Cloud.AttachCellRegister a customer-owned GCP cellGated
Cloud.CreateCellBootstrap a Reaktor-owned GCP cellGated
Cloud.AddNodeAdd a VM/k3s node to a cellGated
Cloud.DrainNodeEvict workloads and remove a nodeGated/destructive
Cloud.DeployGraphPatchApply a partitioned app change to selected cellsGated
Cloud.RollbackGraphPatchRoll back a graph version in selected cellsGated/destructive
Cloud.RotateCellSecretRotate a cell-scoped secret or agent credentialGated
Cloud.ReconcileCompare desired vs observed state and produce a planRead or gated, depending on apply
@Serializable
data class CloudOperationRequest(
    val operationId: String,
    val tenantId: String?,
    val cellId: String?,
    val reason: String,
    val dryRun: Boolean = true,
    val attributes: Map<String, String> = emptyMap(),
)

The low-level details live in providers and agents. Product code should never hand-format Authorization, GCP tokens, kubeconfig paths, or WebSocket auth query strings to run these operations.

6. Tool Execution Boundary

The JVM remains the control surface:

  • provider-neutral models live in common code;
  • GCP, Kubernetes, Memgraph, Spring Actuator, Grafana, and Cloudflare/Supabase inventory adapters are JVM clients;
  • Pulumi is driven through the Java Automation API and CLI where required;
  • Dagger is driven by the dagger CLI as a supervised subprocess;
  • wrangler, kubectl, and gcloud are fallback tool runners only where a typed provider adapter does not cover the operation yet.

There is no embedded Node sidecar. A tool can run Node internally, as Dagger or Pulumi may for a specific module, but Reaktor supervises the tool process and maps output into CloudEvents.

7. Interceptors And Auth

Cloud operations must use the same staged interceptor chain as services:

StageCloud use
CLIENT_APPLICATIONattach user/service auth, tenant/cell intent, trace id, idempotency key
CLIENT_TRANSPORTshape final HTTP/gRPC/WebSocket/Kubernetes request
SERVER_TRANSPORTparse bearer token, request id, origin, rate-limit key
SERVER_APPLICATIONenforce AuthRequirement, tenant/cell policy, safety class, budget
CONNECTION_SETUPauthenticate realtime, PartyServer, or agent connection setup before binding connection state

This is where the earlier auth cleanup belongs. Middleware, Spring filters, Hono handlers, Ktor clients, WebSocket setup, and PartyServer hooks are adapters into the same interceptor mechanism. They should not parse bearer strings, dev headers, service tokens, or socket query auth by hand.

For realtime and agent setup:

  • normal API calls use centralized bearer auth;
  • /realtime/token or an equivalent connection-token operation mints short-lived delegated connection tokens;
  • CONNECTION_SETUP validates audience, tenant, cell/context, expiry, and actor before any durable connection is accepted;
  • the verified AuthContext is attached to connection state.

8. Control Plane UI

The pane should show the fleet as a graph of cells, not as a pile of provider dashboards.

Core views:

  • Fleet map: cells by owner, region, mode, health, capacity, and current graph version.
  • Cell detail: cluster nodes, workloads, namespaces, agent status, drift, secrets, ingress, and recent operations.
  • Tenant view: which cells a tenant can use, current isolation tier, residency, budget, quotas, and customer-owned connections.
  • Operations console: Dagger/Pulumi/k8s/GCP runs with logs, dry-run plans, approval state, and audit history.
  • Connections: customer-owned GCP project/cluster attachments, mode upgrades, credential status, and revoke flow.
  • Deep links: GCP console, Grafana dashboards, Pulumi stacks, Cloudflare dashboards, Supabase dashboards where those providers remain part of the product runtime.

The pane can still list Workers, D1/R2/KV, Supabase, Pub/Sub, k3s pods, Memgraph, and Grafana. The difference is that every resource is grouped under owner, tenant, cell, and graph node.

9. Multitenancy Link

Tenant isolation has three physical tiers:

TierInfrastructure shape
PoolShared cell, shared stores, enforced tenant scope
BridgeShared cell, tenant-specific namespace/schema/database
SiloDedicated cell or customer-owned GCP cell

Customer-owned GCP cells are the strongest silo tier. They let enterprise tenants keep billing, IAM root, and infrastructure jurisdiction in their own GCP account while still consuming Reaktor's graph operations and convenience layer.

Placement should be declared in TenantPolicy:

@Serializable
data class TenantPlacementPolicy(
    val isolation: IsolationTier,
    val allowedRegions: Set<String>,
    val preferredCellIds: Set<String> = emptySet(),
    val requireCustomerOwnedGcp: Boolean = false,
    val maxMonthlyGcpBudgetUsd: Long? = null,
)

The app should not choose clusters manually. The graph placement pass selects cells from tenant policy, capacity, residency, and safety constraints.

10. Build Plan

PhaseDeliverable
C0 - Current cell inventoryImport the existing BestBuds GCP VM/k3s state as a CloudCell; show it in the pane with read-only inventory and deep links.
C1 - Staged interceptorsRoute cloud/service/auth/realtime setup through the common ServiceInterceptor stages; remove ad hoc auth header/query formation from cloud-facing code.
C2 - Cell registryAdd CloudCell, GcpCellConnection, AgentRegistration, operation ledger, and desired-vs-observed facts to the graph.
C3 - Agent observe modeInstall reaktor-agent in the current k3s cell; report inventory, health, drift, and logs through an outbound channel.
C4 - Gated operationsDry-run and apply Cloud.DeployGraphPatch, Cloud.DrainNode, Cloud.RotateCellSecret, and Cloud.Reconcile through signed plans.
C5 - Capacity operationsAdd/resize/drain k3s nodes inside one GCP region with budget and safety gates.
C6 - Reaktor-managed paid cellsLet a consumer choose region/capacity/isolation, then create/resize the GCP VM/k3s cell under Reaktor-owned billing with budget guardrails.
C7 - Customer-owned GCP cell attachLet a tenant attach a GCP project/cluster, start in Observe mode, then explicitly upgrade to Deploy/Capacity/Managed.
C8 - Placement and autoscaling policyRoute tenants/workloads to cells by region, capacity, isolation tier, billing owner, and customer ownership; expose budget-aware recommendations.

11. Risks And Guardrails

  • Credential sprawl: use centralized auth providers, Workload Identity Federation where possible, scoped service accounts, short-lived bootstrap tokens, and interceptor-managed bearer propagation.
  • Customer trust: start customer-owned cells in Observe mode and make every mode upgrade explicit, audited, and revocable.
  • Agent blast radius: agent RBAC must match the selected mode and namespace scope; no cluster-admin by default.
  • Budget surprises: capacity operations need budget policy, dry-run cost estimates, and hard customer-owned GCP limits.
  • Cross-cell data leaks: tenant and cell context must travel in AuthContext, service envelopes, operation plans, telemetry, and audit logs.
  • One cluster stretched too far: do not make a multi-region k3s cluster. Create regional cells and route through graph placement.
  • Tool drift: Dagger, Pulumi, gcloud, and kubectl remain executors. Reaktor records desired state and observed facts so CLI-only drift is visible.