Reaktor Cloud Control Plane
Operations integration, 1 October 2026. Fleet provisioning is a dependency of Shadow workspace preparation, with verified resource destinations, authority, expiry and cleanup receipts. Workspaces share observability infrastructure while isolating application state/effects. Reconcile existing installed collectors with provisioning definitions before reuse; this design does not authorize a duplicate stack.
Desktop implementation status · 30 September 2026: the Cloud pane now reads Cloudflare, Google Cloud and the cluster live, joins them with the repository's declarations, and reports health, drift and findings. What it does and how is in Reaktor Cloud; this page stays the design direction for cells and fleets.
The Cloud pane is the operator view over Reaktor Cloud. The cloud model underneath it is deliberately narrower and stronger than a generic multi-cloud abstraction:
- GCP is the infrastructure provider. Reaktor can still deploy Workers, talk to Supabase, publish to Pub/Sub, and link to Grafana, but the fleet substrate for VMs, k3s, regional capacity, and customer-owned clusters is GCP.
- A cell is the unit of placement. A cell is one regional k3s cluster plus the GCP project, node pools, namespaces, secrets, observability, and routing needed to run a slice of the graph.
- Clusters do not form one stretched cluster. Each region or customer boundary is its own cell. Cells interact through the Reaktor control plane, service calls, the bus, mesh/realtime edges, and observability streams.
- Consumers get both commercial options. They can choose location and capacity and let Reaktor create/manage the GCP VMs with billing flowing through Reaktor, or they can bring their own GCP account/cluster and let Reaktor manage it under scoped authority.
- Customer-owned clusters are first-class. A tenant can attach its own GCP project or k3s cluster to Reaktor Cloud. The customer pays GCP directly for the VMs; Reaktor charges for the control plane, convenience, operations, and automation.
- The attachment point is
reaktor-agent. The agent runs in each cell, dials outbound to the control plane, reports inventory and health, and applies signed operation plans under scoped Kubernetes and GCP permissions.
1. The Core Decision
Reaktor Cloud is not "support every cloud." It is:
One Reaktor control plane
-> many GCP cells
-> some Reaktor-owned
-> some customer-owned
-> each running k3s and a scoped reaktor-agent
That keeps the platform understandable. The control plane owns identity, policy, graph operations, deployment intent, run history, audit, and reconciliation. The cell owns execution: pods, VMs, local Kubernetes state, service endpoints, metrics, and regional capacity.
There are two consumer-facing deployment models:
| Option | Who owns/pays GCP | What the consumer chooses | What Reaktor manages |
|---|---|---|---|
| Reaktor-managed GCP cell | Reaktor owns the GCP account/project and bills the customer through Reaktor | Region, capacity class, isolation tier, budget/SLA | VM creation, k3s, node pools, upgrades, workloads, observability, incident operations |
| Customer-owned GCP cell | Customer owns/pays their GCP account directly | GCP project/account, region, allowed scopes, management mode | Agent install/registration, workload deploys, drift detection, operations within delegated policy |
Both options produce the same CloudCell shape and the same graph operations. The difference is billing owner, IAM root, and how much authority the customer delegates.
The current single-VM setup is just the seed cell:
bestbuds-prod-asia-south1
GCP Compute VM
k3s server
Spring app server
Memgraph
ClickHouse / observability pieces as they land
Cloudflare Tunnel
Scaling this should not mean stretching that k3s cluster across the world. Scaling means creating or attaching more cells and placing tenants, workloads, actors, and data in the right cells.
2. Control Plane And Cells
The model has two ownership planes.
| Plane | Owner | What lives there | Reaktor responsibility |
|---|---|---|---|
| Control plane | Reaktor | Tenant registry, cell registry, policy, auth, service catalog, graph operations, operation ledger, signed plans, audit, billing for Reaktor convenience | Decide what should run where, authorize operations, record facts, reconcile drift |
| Cell plane | Reaktor or customer | GCP project, VM/node pools, k3s cluster, namespaces, pods, local secrets, telemetry exporters, regional ingress/tunnel | Execute signed plans, report observed state, enforce scoped permissions |
The control plane never needs a broad credential that can mutate every customer's cloud directly from a developer laptop. It talks to agents and provider adapters through centralized interceptors, service accounts, and scoped grants.
2.1 Cell Shape
@Serializable
data class CloudCell(
val id: String,
val owner: CellOwner, // ReaktorOwned | CustomerOwned(tenantId)
val billing: CellBilling,
val provider: CloudProvider = CloudProvider.GCP,
val gcpProjectId: String,
val region: String,
val cluster: K3sClusterRef,
val agent: AgentRef,
val mode: CellMode,
val status: CellStatus,
val capacity: CellCapacity,
val labels: Map<String, String> = emptyMap(),
)
@Serializable
sealed interface CellOwner {
data object ReaktorOwned : CellOwner
data class CustomerOwned(val tenantId: String) : CellOwner
}
@Serializable
sealed interface CellBilling {
data class ReaktorManaged(val planId: String) : CellBilling
data class CustomerDirectGcp(val billingAccountHint: String? = null) : CellBilling
}
@Serializable
enum class CellMode {
Observe, // inventory, health, drift only
Deploy, // apply app workloads
Capacity, // add/drain nodes inside approved bounds
Managed, // Reaktor can operate the full cell within policy
}
The same type covers:
- shared Reaktor cells for small/default tenants;
- dedicated Reaktor-owned cells for higher isolation;
- Reaktor-managed paid cells where the customer chooses region/capacity and pays Reaktor;
- customer-owned GCP cells for clients who want the GCP bill and legal control in their account;
- regional cells used for latency or residency.
2.2 How Clusters Interact
Clusters interact through explicit planes, not by pretending they are one cluster.
| Interaction plane | What crosses cell boundaries | Mechanism |
|---|---|---|
| Control | inventory, desired state, signed operation plans, run events, drift reports | reaktor-agent outbound channel to Reaktor Cloud |
| Service | typed request/response calls between services | reaktor-service clients with staged interceptors, service auth, tracing, and routing |
| Bus | durable facts, jobs, fan-out, retries | GCP Pub/Sub / future Reaktor Bus envelopes with tenant and cell context |
| Mesh/realtime | live session data, peer state, WebSocket setup | Reaktor Mesh / realtime token exchange, with connection setup interceptors |
| Traffic | user ingress and service ingress | Cloudflare Tunnel, GCP load balancing, gateway routing, cell-aware selectors |
| Observability | metrics, traces, logs, health, cost facts | OTel and Grafana/ClickHouse with bounded tenant/cell labels |
This is the important rule: a pod in one cell does not reach into another cell's Kubernetes API. It calls a service, publishes a fact, or asks the control plane for a gated operation.
2.3 Same Region vs New Location
There are two different scaling verbs:
| Need | Operation | Result |
|---|---|---|
| More capacity in an existing location | Cloud.AddNode / Cloud.ResizeNodePool | Add a GCP VM as a k3s agent node, rebalance workloads, update capacity facts |
| Lower latency, residency, or tenant isolation in a new location | Cloud.CreateCell / Cloud.AttachCell | New regional cell with its own cluster, agent, policies, and placement rules |
Adding VMs is a capacity operation inside a cell. Adding locations or ownership boundaries is a cell operation.
3. Reaktor-Managed GCP Cells
The default commercial model can stay simple: the customer chooses where and how much capacity they need, and Reaktor provisions the GCP resources.
Customer chooses:
region = asia-south1
capacity = 3 x e2-standard-4
isolation = dedicated cell
budget/SLA = plan limit
Reaktor does:
create/resize GCP VMs
install/upgrade k3s
attach reaktor-agent
deploy workloads
operate incidents and rollbacks
bill the customer through Reaktor
This is the fastest path for consumers who want convenience over cloud-account ownership. It also gives Reaktor the most complete operating envelope: one IAM model, one billing relationship, one upgrade path, one fleet baseline.
The customer still gets control-plane visibility:
- region and capacity selection;
- cost estimate and monthly budget guardrails;
- workload placement and isolation tier;
- operation history and audit;
- export path to customer-owned GCP later if the contract requires it.
4. Customer-Owned GCP Cells
Yes: customers should be able to connect their own GCP clusters to the Reaktor Cloud control plane.
The business model is clean:
- customer pays GCP for the VMs, disks, network, and regional resources;
- Reaktor bills for the control plane, automation, deployment experience, observability, policy, fleet operations, and support;
- the customer can revoke the connection without asking Reaktor to export their infrastructure.
This is the strongest multitenancy tier. It is not just "tenant_id in a shared database"; it is an infrastructure ownership boundary.
3.1 Attachment Flow
1. Customer creates or selects a GCP project.
2. Reaktor creates a CloudConnection request with required scopes and region.
3. Customer grants GCP access through a least-privilege service account or Workload Identity Federation.
4. Customer installs reaktor-agent into the k3s cluster, or lets Reaktor bootstrap k3s if the mode permits it.
5. Agent dials outbound to Reaktor Cloud and presents a registration token.
6. Control plane verifies project, region, cluster identity, tenant policy, and agent attestation.
7. Cell becomes Observe mode first.
8. Customer explicitly upgrades to Deploy, Capacity, or Managed mode.
The initial mode is always Observe. That gives the customer and operator a read-only inventory and drift view before any mutating operation is possible.
3.2 Connection Model
@Serializable
data class GcpCellConnection(
val id: String,
val tenantId: String,
val gcpProjectId: String,
val region: String,
val clusterName: String,
val billingOwner: BillingOwner = BillingOwner.CustomerDirectGcp,
val auth: GcpConnectionAuth,
val allowedModes: Set<CellMode>,
val status: ConnectionStatus,
)
@Serializable
sealed interface GcpConnectionAuth {
data class WorkloadIdentityFederation(val providerResource: String) : GcpConnectionAuth
data class ServiceAccountImpersonation(val serviceAccountEmail: String) : GcpConnectionAuth
data class BootstrapToken(val secretRef: String) : GcpConnectionAuth // short-lived install only
}
@Serializable
enum class BillingOwner {
ReaktorManaged,
CustomerDirectGcp,
}
Prefer keyless identity. Long-lived service-account JSON should be treated as a temporary bootstrap fallback, not the normal state.
3.3 Operating Modes
| Mode | Reaktor can do | Customer retains |
|---|---|---|
| Observe | Read inventory, health, versions, drift, cost hints, logs the customer permits | All mutation rights |
| Deploy | Apply Reaktor workloads into approved namespaces, roll out/rollback apps, run migrations under policy | Cluster lifecycle and node capacity |
| Capacity | Add/drain/resize approved node pools inside budget/location limits | Project ownership and hard budget controls |
| Managed | Full day-2 operation inside the agreed policy: upgrades, patching, scaling, incident actions | Billing ownership, IAM root, revocation right |
Mode is part of TenantPolicy and CloudCell, not an ad hoc flag in deployment scripts.
4. reaktor-agent
reaktor-agent is the only component that must run inside every cell.
It should be boring and narrow:
- dial outbound to the control plane; no public inbound Kubernetes API required;
- register the cell and renew agent identity;
- report observed Kubernetes/GCP/resource state;
- stream bounded health, metrics, events, and drift facts;
- pull signed operation plans;
- apply plans only when the plan signature, tenant, cell, mode, and policy match;
- run with scoped Kubernetes RBAC and minimal GCP permissions;
- write every mutation to the operation ledger.
@Serializable
data class AgentRegistration(
val agentId: String,
val cellId: String,
val tenantId: String?,
val clusterUid: String,
val gcpProjectId: String,
val region: String,
val capabilities: Set<AgentCapability>,
val publicKey: String,
)
@Serializable
data class SignedCellPlan(
val id: String,
val cellId: String,
val tenantId: String?,
val operation: String,
val safetyClass: SafetyClass,
val expiresAtEpochSeconds: Long,
val payload: ByteArray,
val signature: ByteArray,
)
The agent is an executor, not the source of truth. The source of truth is the signed desired state plus the observed facts the agent reports back.
5. Operations
Every cloud action is a typed graph operation. The pane, CLI, agents, and service APIs all call the same operations through the same interceptor chain.
| Operation | Purpose | Safety |
|---|---|---|
Cloud.Inventory | List cells, projects, clusters, node pools, workloads, costs, and drift | Read |
Cloud.AttachCell | Register a customer-owned GCP cell | Gated |
Cloud.CreateCell | Bootstrap a Reaktor-owned GCP cell | Gated |
Cloud.AddNode | Add a VM/k3s node to a cell | Gated |
Cloud.DrainNode | Evict workloads and remove a node | Gated/destructive |
Cloud.DeployGraphPatch | Apply a partitioned app change to selected cells | Gated |
Cloud.RollbackGraphPatch | Roll back a graph version in selected cells | Gated/destructive |
Cloud.RotateCellSecret | Rotate a cell-scoped secret or agent credential | Gated |
Cloud.Reconcile | Compare desired vs observed state and produce a plan | Read or gated, depending on apply |
@Serializable
data class CloudOperationRequest(
val operationId: String,
val tenantId: String?,
val cellId: String?,
val reason: String,
val dryRun: Boolean = true,
val attributes: Map<String, String> = emptyMap(),
)
The low-level details live in providers and agents. Product code should never hand-format Authorization, GCP tokens, kubeconfig paths, or WebSocket auth query strings to run these operations.
6. Tool Execution Boundary
The JVM remains the control surface:
- provider-neutral models live in common code;
- GCP, Kubernetes, Memgraph, Spring Actuator, Grafana, and Cloudflare/Supabase inventory adapters are JVM clients;
- Pulumi is driven through the Java Automation API and CLI where required;
- Dagger is driven by the
daggerCLI as a supervised subprocess; wrangler,kubectl, andgcloudare fallback tool runners only where a typed provider adapter does not cover the operation yet.
There is no embedded Node sidecar. A tool can run Node internally, as Dagger or Pulumi may for a specific module, but Reaktor supervises the tool process and maps output into CloudEvents.
7. Interceptors And Auth
Cloud operations must use the same staged interceptor chain as services:
| Stage | Cloud use |
|---|---|
CLIENT_APPLICATION | attach user/service auth, tenant/cell intent, trace id, idempotency key |
CLIENT_TRANSPORT | shape final HTTP/gRPC/WebSocket/Kubernetes request |
SERVER_TRANSPORT | parse bearer token, request id, origin, rate-limit key |
SERVER_APPLICATION | enforce AuthRequirement, tenant/cell policy, safety class, budget |
CONNECTION_SETUP | authenticate realtime, PartyServer, or agent connection setup before binding connection state |
This is where the earlier auth cleanup belongs. Middleware, Spring filters, Hono handlers, Ktor clients, WebSocket setup, and PartyServer hooks are adapters into the same interceptor mechanism. They should not parse bearer strings, dev headers, service tokens, or socket query auth by hand.
For realtime and agent setup:
- normal API calls use centralized bearer auth;
/realtime/tokenor an equivalent connection-token operation mints short-lived delegated connection tokens;CONNECTION_SETUPvalidates audience, tenant, cell/context, expiry, and actor before any durable connection is accepted;- the verified
AuthContextis attached to connection state.
8. Control Plane UI
The pane should show the fleet as a graph of cells, not as a pile of provider dashboards.
Core views:
- Fleet map: cells by owner, region, mode, health, capacity, and current graph version.
- Cell detail: cluster nodes, workloads, namespaces, agent status, drift, secrets, ingress, and recent operations.
- Tenant view: which cells a tenant can use, current isolation tier, residency, budget, quotas, and customer-owned connections.
- Operations console: Dagger/Pulumi/k8s/GCP runs with logs, dry-run plans, approval state, and audit history.
- Connections: customer-owned GCP project/cluster attachments, mode upgrades, credential status, and revoke flow.
- Deep links: GCP console, Grafana dashboards, Pulumi stacks, Cloudflare dashboards, Supabase dashboards where those providers remain part of the product runtime.
The pane can still list Workers, D1/R2/KV, Supabase, Pub/Sub, k3s pods, Memgraph, and Grafana. The difference is that every resource is grouped under owner, tenant, cell, and graph node.
9. Multitenancy Link
Tenant isolation has three physical tiers:
| Tier | Infrastructure shape |
|---|---|
| Pool | Shared cell, shared stores, enforced tenant scope |
| Bridge | Shared cell, tenant-specific namespace/schema/database |
| Silo | Dedicated cell or customer-owned GCP cell |
Customer-owned GCP cells are the strongest silo tier. They let enterprise tenants keep billing, IAM root, and infrastructure jurisdiction in their own GCP account while still consuming Reaktor's graph operations and convenience layer.
Placement should be declared in TenantPolicy:
@Serializable
data class TenantPlacementPolicy(
val isolation: IsolationTier,
val allowedRegions: Set<String>,
val preferredCellIds: Set<String> = emptySet(),
val requireCustomerOwnedGcp: Boolean = false,
val maxMonthlyGcpBudgetUsd: Long? = null,
)
The app should not choose clusters manually. The graph placement pass selects cells from tenant policy, capacity, residency, and safety constraints.
10. Build Plan
| Phase | Deliverable |
|---|---|
| C0 - Current cell inventory | Import the existing BestBuds GCP VM/k3s state as a CloudCell; show it in the pane with read-only inventory and deep links. |
| C1 - Staged interceptors | Route cloud/service/auth/realtime setup through the common ServiceInterceptor stages; remove ad hoc auth header/query formation from cloud-facing code. |
| C2 - Cell registry | Add CloudCell, GcpCellConnection, AgentRegistration, operation ledger, and desired-vs-observed facts to the graph. |
| C3 - Agent observe mode | Install reaktor-agent in the current k3s cell; report inventory, health, drift, and logs through an outbound channel. |
| C4 - Gated operations | Dry-run and apply Cloud.DeployGraphPatch, Cloud.DrainNode, Cloud.RotateCellSecret, and Cloud.Reconcile through signed plans. |
| C5 - Capacity operations | Add/resize/drain k3s nodes inside one GCP region with budget and safety gates. |
| C6 - Reaktor-managed paid cells | Let a consumer choose region/capacity/isolation, then create/resize the GCP VM/k3s cell under Reaktor-owned billing with budget guardrails. |
| C7 - Customer-owned GCP cell attach | Let a tenant attach a GCP project/cluster, start in Observe mode, then explicitly upgrade to Deploy/Capacity/Managed. |
| C8 - Placement and autoscaling policy | Route tenants/workloads to cells by region, capacity, isolation tier, billing owner, and customer ownership; expose budget-aware recommendations. |
11. Risks And Guardrails
- Credential sprawl: use centralized auth providers, Workload Identity Federation where possible, scoped service accounts, short-lived bootstrap tokens, and interceptor-managed bearer propagation.
- Customer trust: start customer-owned cells in Observe mode and make every mode upgrade explicit, audited, and revocable.
- Agent blast radius: agent RBAC must match the selected mode and namespace scope; no cluster-admin by default.
- Budget surprises: capacity operations need budget policy, dry-run cost estimates, and hard customer-owned GCP limits.
- Cross-cell data leaks: tenant and cell context must travel in
AuthContext, service envelopes, operation plans, telemetry, and audit logs. - One cluster stretched too far: do not make a multi-region k3s cluster. Create regional cells and route through graph placement.
- Tool drift: Dagger, Pulumi,
gcloud, andkubectlremain executors. Reaktor records desired state and observed facts so CLI-only drift is visible.
Related Docs
- Reaktor Multitenancy - tenant-owned GCP cells as the silo tier.
- Reaktor DevOps - graph deploys, capacity operations, Dagger/Pulumi, and fleet reconciliation.
- Reaktor Service - staged interceptors, operation identity, and transport adapters.
- Auth Architecture -
AuthContext, service accounts, token exchange, and delegated realtime tokens. - Reaktor Bus - durable cross-cell facts, jobs, retries, and tenant-scoped envelopes.
- Performance and Observability - fleet health, bounded tenant/cell labels, SLOs, and Grafana.
- Secrets Management - GCP Secret Manager, k3s secrets, and materialization boundaries.