Koog in reaktorDesktop

Investigation: should reaktorDesktop use JetBrains Koog to power its in-workbench AI agents? It already ships a fully-designed multi-agent system — Codex / Claude / Gemini with roles, models, permission maps, a command queue, and a "Critic reviews before apply" loop — but it's entirely fixture-mocked. reaktorDesktop is pure JVM Compose Desktop, which is Koog's strongest target (no Cloudflare-Worker isolate caveat the bot had). Koog is the missing engine.

• Subject reaktorDesktop workbench (Compose Desktop, JVM)

• Candidate Koog 1.0 (JetBrains, Apache-2, KMP/JVM)

• Verdict Adopt — ideal host

• Status Investigation / design

• Host runtime — JVM Compose Desktop — Koog native

• Agent UX — designed + fixture-mocked

• AgentInteractor — view-state only (no LLM)

• Command queue — real (apply/revert/commit/test/deploy)

• Mock agents — 3 (Codex · Claude · Gemini) + permissions

• Worker-isolate caveat — none — full JVM

• Graph context — live Graph + visitors + traces in-process

• Open-in-IDE tool — already built

1. Verdict

Three reasons it's the strongest fit in the codebase:

• The runtime is perfect. The bot eval (reaktor-koog-agents) had to keep agent loops off the Cloudflare Worker. reaktorDesktop has no such constraint — it's a desktop JVM process, exactly where Koog's agent loops, memory, and tool-calling belong.
• The scaffolding is already there. The workbench isn't a blank canvas — AgentInteractor, the 3 fixture agents with instructions + permission maps, CommandQueueActor, and WorkbenchCommandApi (hot-reload / commit / rollback / test / deploy) already exist. We're wiring an engine into a built dashboard, not designing the dashboard.
• The graph is the context. Koog tools read the live Graph — nodes, ports, edges, traces, the selected node — all in-process via the visitors and inspectors the workbench already uses. The agent reasons over real topology, not a serialized snapshot.

2. What reaktorDesktop is

reaktorDesktop (targets/reaktorDesktop/src/main/kotlin/Main.kt) is a Compose Desktop application that boots ReaktorWorkbench — the visual graph workbench: a Blueprint-style graph canvas, a node inspector, DevTools (live Compose preview, traces), a command queue, and an agent surface. It runs on the JVM with the source tree on disk (it already resolves graph nodes to source and opens them in the IDE — see the open-in-IDE work). Modes include Graph, DevTools, and an Agent view.

3. The agent system already exists (mocked)

This is the crux: the workbench already designs a multi-agent system in code — it just has no brain. From modules/engine/.../fixtures/WorkbenchAgentFixtures.kt and the data layer:

3.1 Three role-based agents (fixtures)

Each fixture carries: instructions (system prompt), model, a permissions map (readGraph, readTraces, pushCommands, applyCommands, generateCode, runTests, deploy, modifyRbac), triggers, scope, collaboration, and an activity log. It is, in effect, a complete agent spec — minus the LLM.

3.2 The real apply substrate

Unlike the agents, the command pipeline is real code (just backed by a service mock):

• CommandQueueActor (an actual ActorNode) — serializes Select / ApplyAll / RevertAll / SendSelectedToAgent / HotReload / RunTests / Deploy / Repair / Revert messages.
• WorkbenchCommandApi — hotReload(), commitBundle(), deploy(env), rollback(), runTests(harness).
• WorkbenchDataApi — schema(), query(sql, params) for reading data.
• CommandQueueRepository + CommandQueueState — the queue of proposed mutations with per-command repair/revert.

4. The gap

Everything that touches an LLM is mocked; everything mechanical (queue, apply, state) is real. Koog fills exactly the mocked half.

5. Why Koog fits here

The bot eval already verified Koog 1.0 (KotlinConf, May 2026): KMP, agent graph-strategies, tools (annotation/class/MCP), structured output, memory + persistence, OpenTelemetry, multi-provider, Spring/Ktor. For reaktorDesktop specifically, these map one-to-one onto what the workbench already designed:

Agents ↔ Koog AIAgent

Each fixture agent's instructions + model → a Koog AIAgent system prompt + LLM. The fixture is the config.

Commands ↔ structured output

"Plan in commands, never raw diffs" → Koog structured output producing typed WorkbenchCommand objects. The model can't hand you a messy patch; it hands you AddNode(...).

Critic loop ↔ subgraphs / multi-agent

"Codex pushes, Claude reviews before apply" → Koog multi-agent: a Critic subgraph that gates the command queue. Koog supports agent-to-agent + planner patterns natively.

Permissions ↔ tool sets

The fixture permission map → which Koog tools each agent is given. deploy=false means the deploy tool isn't in that agent's toolset. Capability security by construction.

Tools ↔ WorkbenchCommandApi

hotReload / commitBundle / deploy / rollback / runTests / query become Koog tools. The agent does things through the same API the human buttons call.

MCP ↔ the IntelliJ fixture

The "IntelliJ Plugin (MCP)" fixture client → a real Koog MCP connection. And the workbench can expose itself as an MCP server to external agents.

6. Architecture

Koog lives in jvmMain of the engine module, behind the existing contracts. The Compose UI (Agent pane) is unchanged; AgentInteractor gains a real backend.

reaktorDesktop (JVM Compose Desktop)
┌──────────────────────────────────────────────────────────────────────┐
│ Agent pane (Compose) ── unchanged UI, reads AgentState ──┐          │
│                                                          │          │
│ AgentInteractor ── today: view-state ──▶ now also drives ─┐         │
│                                                          │ │        │
│ ┌───────────────────────────────────────────────▼─▼──────┐ │        │
│ │ WorkbenchAgentRuntime (new, jvmMain, Koog-backed)       │ │        │
│ │ ┌───────────┐ ┌───────────┐ ┌───────────┐              │ │        │
│ │ │ Codex     │ │ Claude    │ │ Gemini    │ Koog AIAgents │ │        │
│ │ │ (build)   │ │ (critic)  │ │ (refactor)│              │ │        │
│ │ └─────┬─────┘ └─────┬─────┘ └─────┬─────┘              │ │        │
│ │       │ structured  │ output      │                    │ │        │
│ │       ▼             WorkbenchCommand[]                  │ │        │
│ │ ┌──────────────────────────────────────────────┐        │ │        │
│ │ │ Koog tools (permission-gated per agent):     │        │ │        │
│ │ │ readGraph · readTraces · query · runTests ·  │        │ │        │
│ │ │ openInIde · pushCommand · applyBundle ·      │        │ │        │
│ │ │ deploy · MCP(IntelliJ, …)                    │        │ │        │
│ │ └──────────────────┬───────────────────────────┘        │ │        │
│ └────────────────────┼────────────────────────────────────┘ │        │
│                      ▼                                      │        │
│ CommandQueueActor ──▶ WorkbenchCommandApi (hotReload/commit/rollback/│
│ (human reviews,       deploy/runTests) ── REAL ──▶ build             │
│  applies/reverts)                                                   │
└──────────────────────────────────────────────────────────────────────┘
        │ Koog OpenTelemetry
        ▼
Grafana LGTM (performance-observability)

Key property: the agent never mutates the project directly. It pushes typed commands to the existing queue; a human (or the Critic agent, within its permissions) applies them through WorkbenchCommandApi, which already supports commit / rollback / hot-reload / test / deploy.

7. Tools = workbench capabilities

The workbench's existing capabilities become the agent's tools. Sketch:

fun workbenchTools(
  graph: Graph,
  commandApi: WorkbenchCommandApi,
  dataApi: WorkbenchDataApi,
  queue: CommandQueueActor,
  permissions: Map<String, Boolean>,
) = tools {
  // read-only context tools (gated by readGraph / readTraces)
  if (permissions["readGraph"] == true) {
    tool("listNodes")     { GraphTopologyVisitor().walk(graph).summary() }
    tool("inspectNode")   { id -> graph.node(id)?.let(::describeNode) }
    tool("openInIde")     { id -> graph.node(id)?.let { GraphNodeSourceIndex.locate(it) }?.also { ReaktorIde.open(it.file, it.line) } }
  }
  if (permissions["readTraces"] == true)
    tool("recentTraces")  { node -> tracePane.recent(node) }
  tool("querySchema")    { schema -> dataApi.schema(schema) }
  tool("query")          { sql, params -> dataApi.query(sql, params) }   // read-only by policy

  // mutation tools — gated by the agent's permission map
  if (permissions["pushCommands"] == true)
    tool("pushCommand")   { cmd: WorkbenchCommand -> queue.push(cmd) }   // to the queue, not applied
  if (permissions["applyCommands"] == true)
    tool("applyBundle")   { queue.applyAll() }                          // commitBundle() under the hood
  if (permissions["runTests"] == true)
    tool("runTests")      { harness -> queue.runTests(harness) }        // Maestro/k6/Keploy (perf doc)
  if (permissions["deploy"] == true)
    tool("deploy")        { env -> queue.deploy(env) }
}

8. Commands, not diffs

The single most important design rule the fixtures state: agents "plan in commands: AddNode, ConnectPorts, UpdateSchema — never raw diffs." Koog's structured output enforces this — the model returns a typed object, not free text:

@Serializable
sealed interface WorkbenchCommand {
  @Serializable data class AddNode(val kind: String, val module: String, val label: String) : WorkbenchCommand
  @Serializable data class ConnectPorts(val from: String, val to: String) : WorkbenchCommand
  @Serializable data class UpdateSchema(val type: String, val change: String) : WorkbenchCommand
  @Serializable data class GenerateCode(val targetFile: String, val rationale: String) : WorkbenchCommand
}

// Koog: ask the agent for a typed plan, not prose.
val plan: List<WorkbenchCommand> = agent.runStructured(userRequest)
plan.forEach { queue.push(it) }   // lands in the existing CommandQueue for human review

9. Multi-agent & the Critic loop

The fixtures describe a real workflow: Codex builds → pushes commands → Claude (Critic) reviews → human applies. Koog models this as cooperating agents:

user request
    │
    ▼
Codex (Architect+Engineer)
├─ reads graph + traces (tools)
├─ produces WorkbenchCommand[] (structured output)
└─ pushCommand → queue (permission: pushCommands ✓, applyCommands ✓)
    │
    ▼
(trigger: "after Codex pushes >3 commands")
Claude (Critic+Guardian)
├─ reviews each queued command (challenge, find edge cases)
├─ writes test harnesses · adds guardrail commands
└─ blocks or annotates (permission: applyCommands ✗ — can only advise)
    │
    ▼
human (or auto-apply within policy)
└─ applyBundle → commitBundle → runTests → (rollback on red)

Gemini (Cross-Codebase Ops)
runs on its own triggers (weekly health pass, dep bumps)
— refactor commands → same queue → same review

Koog supports this directly: agents as nodes, a Critic subgraph that consumes the queue, and handoff between agents. The "trigger" fields in the fixtures ("After Codex pushes >3 commands", "Nightly: audit unused paths") become scheduled or event-driven agent invocations.

10. Permission-gated tools

The fixture permissions map isn't decoration — it becomes the agent's capability boundary. Building the toolset from the map (as in §7) means an agent cannot call a tool it lacks permission for, because the tool was never registered for it.

11. The graph as context

An agent is only as good as its context. In reaktorDesktop the context is the live graph — not a serialized export:

• Topology — a GraphTopologyVisitor (the workbench already has visitors) summarizes nodes, kinds, ports, edges, communities for the system prompt.
• Selected node — when the user selects a node, that becomes the agent's focus (with its source via the open-in-IDE locator, its ports, downstream, recent traces).
• Traces — DevTools trace data (and the OTel pipeline from performance-observability) lets the agent reason about runtime behavior, not just structure ("this node's p95 regressed").
• Data — WorkbenchDataApi.query over Postgres / Memgraph (read-only) for "what does the data actually look like."
• Memory — Koog long-term memory + persistence so a conversation about a subsystem carries across sessions (the fixtures already model "conversations" and "scouts").

Because it's in-process, feeding context is a function call, not an export/upload — the agent sees exactly what the developer sees in the inspector.

12. MCP both ways

The agent fixtures already list an MCP client — WorkbenchMcpClientFixture("IntelliJ Plugin", "disconnected", …). Koog's MCP support makes this real in both directions:

• Workbench as MCP client — the agent uses external MCP tool servers (IntelliJ plugin, Playwright, filesystem, git) through Koog's MCP integration.
• Workbench as MCP server — expose the workbench's own tools (read graph, push command, run tests, open-in-IDE) as an MCP server, so an external agent (Claude Code, Cursor, the IntelliJ plugin) can drive the reaktor graph. This is how the desktop copilot and an external coding agent share one command surface.

13. Models, privacy, offline

• Per-agent models — the fixtures already declare them (codex-5, claude-sonnet-4.7, gemini-2.5-pro); Koog's multi-provider executor wires each agent to its provider.
• Local / offline — for a desktop tool that may run without network, Koog + Ollama runs capable local models (and Koog added local Android LiteRT; on desktop, Ollama is the path). Useful for "explain this node" without a round-trip.
• Cost routing — cheap local model for navigation/explanation; premium provider for end-to-end feature generation. Koog response caching + Anthropic prompt caching cut repeat cost (graph context + tool schemas are stable across a session).
• Keys — provider keys live in the desktop app's local secure storage / env, never committed; same posture as the rest of the toolchain.

14. Observability

Koog emits OpenTelemetry across targets. In reaktorDesktop that means every agent run — prompt, tool calls, commands produced, tokens, latency — becomes a trace in the same Grafana LGTM pipeline as performance-observability §17. The workbench's existing trace panes can render agent runs alongside graph traces, so a developer sees "Codex called inspectNode → query → produced 3 commands → Critic flagged 1" as one timeline. Koog also integrates Langfuse/Weave exporters if dedicated LLM-trace tooling is wanted.

15. Migration plan

Incremental, behind the existing contracts — the Agent-pane UI and WorkbenchCommandApi don't change shape; their backends do.

Phase 1~3 days

One real agent, read-only

Add Koog to engine/jvmMain. Build a WorkbenchAgentRuntime with the Codex fixture's instructions/model, but only read-only tools (listNodes, inspectNode, openInIde, query). Wire AgentInteractor to run a conversation against it. DoD: "explain the chat subsystem" returns a grounded answer citing real nodes.

Phase 2~1 week

Structured commands → queue

WorkbenchCommand sealed type + Koog structured output. pushCommand tool lands typed commands in the real CommandQueue. Render them in the existing queue UI. DoD: "add a presence service to chat" produces reviewable AddNode/ConnectPorts commands (not applied).

Phase 3~1 week

Apply / test / rollback as tools

Make WorkbenchCommandApi real (it already shells to gradle elsewhere): applyBundle → commitBundle, runTests, rollback on red. Gate by permission map. DoD: a Codex command bundle applies, runs tests, auto-reverts on failure.

Phase 4~1 week

Multi-agent + Critic loop

Add Claude (Critic) + Gemini (Ops) as Koog agents; Critic subgraph reviews the queue before apply; triggers wired (post-push, nightly). DoD: Codex's bundle is gated by a real Critic review with test generation.

Phase 5~3 days

MCP + memory + OTel

Koog MCP client (IntelliJ/Playwright) + expose workbench as MCP server; long-term memory per conversation; OTel into the perf pipeline; agent runs in the trace panes. DoD: an external agent can drive the queue via MCP; agent runs are observable in Grafana.

16. Risks & open questions

Risks

• Agent applies something harmful. Mitigated structurally: agents emit queued commands, not writes; apply is permission-gated and goes through commit/rollback + tests; deploy/RBAC are human-only. The fixtures already encode this; Koog enforces it via toolset construction.
• Commands that don't map to real mutations yet. AddNode/ConnectPorts need a real applier (codegen + graph edit). The command queue + commitBundle exist, but the actual code-mutation behind a command is the deepest piece of work — start with GenerateCode/file edits the human reviews, expand to structural graph edits later.
• Cost / latency on a desktop tool. Premium models per keystroke is expensive. Mitigation: explicit invoke (not autocomplete), local Ollama for cheap ops, caching of stable context.
• Context size. A large graph won't fit a prompt. Mitigation: tools fetch on demand (inspectNode, query) + community-scoped summaries rather than dumping the whole graph.
• New dependency in the desktop build. Koog is JVM-native and Apache-2 with a 1.0 stability guarantee; it sits behind AgentInteractor so it's swappable.
• Offline expectation. If reaktorDesktop is used without network, only Ollama-backed agents work; cloud agents must degrade gracefully (show "offline" like the MCP fixture already does).

Open questions

• Does an agent command ever auto-apply (within permissions + green tests), or always human-gated for v1? (Recommend: human-gated v1, opt-in auto-apply for low-risk command kinds later.)
• Real models behind the fixture names: codex-5 → which provider? Map each fixture agent to a concrete Koog provider/model.
• How deep does "AddNode/ConnectPorts" go — real graph mutation + codegen, or start as scaffolding suggestions the human wires?
• Is the desktop copilot the same agent surface as the BestBuds in-app bot (reaktor-koog-agents), or separate? (They share Koog + tool patterns but differ in tools and trust.)
• Workbench-as-MCP-server: expose to any local agent (Claude Code/Cursor) or gated to the IntelliJ plugin only?
• Where does agent memory persist on desktop — local SQLite (the actor ObjectStore) or a user-scoped store?

17. References

Workbench code investigated

• targets/reaktorDesktop/src/main/kotlin/Main.kt — Compose Desktop entry → ReaktorWorkbench.
• modules/engine/.../data/interactors/AgentInteractor.kt — agent view-state (no LLM today).
• modules/engine/.../fixtures/WorkbenchAgentFixtures.kt — the 3 mocked agents + instructions + permission maps + triggers.
• modules/engine/.../data/actors/CommandQueueActor.kt — real ActorNode: apply/revert/commit/hot-reload/test/deploy.
• modules/engine/.../data/services/WorkbenchApis.kt — WorkbenchCommandApi, WorkbenchDataApi (apply substrate + DB access).
• modules/engine/.../model/WorkbenchStates.kt — CommandQueueState, AgentState.
• modules/engine/.../ide/GraphNodeSource.kt — open-in-IDE locator + opener (a ready-made agent tool).

Peer designs

• reaktor-koog-agents — Koog for the BestBuds in-app bot (the sibling eval; shares Koog facts).
• reaktor-distributed-actors — actor model the CommandQueueActor builds on.
• performance-observability — OTel pipeline Koog agent runs flow into; test drivers the runTests tool invokes.
• shadow-dev-environment — safe data + replay for agent-driven changes.
• graph-kernel-analysis — graph/visitor foundation the agent's read tools use.

Koog (external, verified May 2026)

• JetBrains/koog — GitHub (Apache-2, KMP incl. JVM)
• Koog 1.0 release — JetBrains AI blog
• Koog docs · agent patterns, tools, structured output, memory, MCP, OTel

Doc lineage. Authored 2026-05-28. Grounded in the actual reaktorDesktop workbench code — the fixture-mocked multi-agent system (AgentInteractor, WorkbenchAgentFixtures with 3 agents + permission maps), the real CommandQueueActor + WorkbenchCommandApi apply substrate, and the just-built open-in-IDE locator — plus verified Koog 1.0 facts. Verdict: reaktorDesktop is the ideal Koog host; wire Koog behind the existing AgentInteractor / WorkbenchCommandApi contracts. Sibling to the bot-focused reaktor-koog-agents.

Related docs

• Control Plane - Use when exposing Reaktor safely to humans and agents.
• Gemini DevOps Agent - Use before handing DevOps execution from Codex or Claude to Antigravity CLI, adding agent-run recipes, or wiring build/test/deploy operations into reaktorDesktop.
• Koog Bot - Use before changing bot orchestration, moderation, or RAG over Memgraph.
• Workbench Gaps - Use when prioritizing visible workbench functionality.
• Desktop v3 - Use to inspect the current visual target and interaction model.
• Visual Language - Use when designing the node-based authoring/orchestration language and its editor, compiler, and agentic surface.