Koog for the BestBuds Bot

Evaluation: should BestBuds adopt JetBrains Koog (the Kotlin agent framework, 1.0 shipped May 2026) for the bot — moderation today, tool-using / memory-backed agents tomorrow? Short answer: yes, as the JVM-side "brain", keeping the Cloudflare edge as a fast pre-filter. Koog is Kotlin-Multiplatform and its agents are graphs — so it composes with the Reaktor graph kernel rather than competing with it.

• Subject — botServer BotService

• Candidate Koog 1.0 (JetBrains, Apache-2, KMP)

• Verdict Adopt (JVM brain + edge prefilter)

• Status Evaluation / design

• Bot today — Workers AI single-turn + isSafe() boolean

• Koog targets — JVM · JS · WasmJS · Android · iOS

• Koog moderation — moderate() → categorized result

• Koog agents — graph strategies + tools + memory + MCP

• Spring Boot — first-class integration

• Observability — OpenTelemetry (all targets)

• Stability — 1.0 — no breaking changes ≥ 1 yr

• Run-on-Worker caveat — prefer JVM for agent loops

1. Verdict

The three reasons it's a clear yes:

• Moderation is a direct, strict upgrade. Today's ai.isSafe() returns a boolean. Koog's moderate() returns a categorized ModerationResult (harassment, hate, self-harm, sexual, violence, plus Llama-Guard S1–S4 crime classes) across multiple backends. Categorized + auditable + provider-agnostic beats a boolean.
• It's Kotlin-Multiplatform and graph-shaped. Reaktor is a KMP graph kernel; Koog agents are graph strategies with subgraphs and tools. A Koog agent wraps cleanly as a Reaktor node, and Koog tools can call Reaktor services (Memgraph, social, events). Same language, same shape, no FFI, no second runtime.
• It unlocks the "future features" without a rewrite. Tool use, structured output, persistent memory + RAG over the knowledge graph, multi-step planners, MCP — all first-class in Koog 1.0. The current hand-rolled BotService would need to reinvent each of these.

2. Current bot

From bestbuds/modules/app/src/jsMain/.../cloudflare/bot/{BotWorker,AgentRegistry}.kt:

• Runtime: a Cloudflare Worker (CloudflareBotService : BotService), Kotlin/JS, using the Workers AI binding.
• Generation: single-turn ai.textGeneration(model, systemPrompt, userMessage, temperature) on @cf/meta/llama-3.3-70b-instruct-fp8-fast.
• Moderation: ai.isSafe(text) on input and output → boolean → BotResponse.Failure.ContentViolation.
• Personas: AgentRegistry with BestBud / Writer / Support = (id, model, systemPrompt, temperature).
• Operations: /generate and /chat-respond (the latter flattens recent messages into the prompt).

The bot is fine as a v0 — but every "future feature" the question hints at (smarter moderation, tools, memory) means reinventing an agent framework inside BotService. That's exactly what Koog is.

3. What Koog 1.0 is

Koog is JetBrains' open-source (Apache-2) Kotlin framework for building AI agents. 1.0 shipped at KotlinConf, May 2026, with a stability guarantee (no breaking changes on stable modules for ≥ 1 year). Verified feature set:

Multiplatform

Deploys agents across JVM, JS, WasmJS, Android, iOS via Kotlin Multiplatform. Same agent code, multiple targets.

Agent patterns

Basic (predefined strategy), Graph-based (custom strategy graph), Functional (lambda), Planner (iterative plan-and-execute), and subgraphs for modular workflows.

Tools / function calling

Annotation-based, class-based, and custom-schema tools. Parallel tool calls. MCP (Model Context Protocol) — use external MCP tool servers directly.

Structured output

First-class typed/structured response generation — extract typed data from model output reliably.

Memory & persistence

Chat memory, long-term memory via vector DBs (RAG), agent persistence/checkpoints (restore agent state mid-execution), LLM response caching, Anthropic prompt caching.

Content moderation

A moderate() method → ModerationResult. OpenAI Omni moderation model; Llama Guard 3 via Ollama (S1–S4 hazard classes); cross-provider moderation via MultiLLMPromptExecutor.

Providers

OpenAI, Anthropic, Google/Gemini, OpenRouter, DeepSeek, Ollama, Bedrock; local Android via LiteRT. Unified executor abstracts them.

Observability

Configurable tracing + OpenTelemetry across all targets, with Datadog / Langfuse / Weave exporters.

Framework integration

First-class Spring Boot (and Spring AI) + Ktor plugin. Embed Koog into existing JVM services.

4. Fit assessment

Where Koog wins decisively

• Moderation: categorized, multi-backend, auditable vs a boolean.
• Tool use: bot can call Reaktor services (events, friends, knowledge graph) as typed tools.
• Memory / RAG: persistent context + retrieval over Memgraph / message history.
• Structured output: extract event details, RSVPs, intents reliably.
• Provider flexibility: cheap local model for moderation, premium for hard tasks.
• Observability: OTel into the same Grafana LGTM pipeline as everything else.
• Spring Boot: drops into reaktorServer with an official integration.

Where to be careful

• Don't run agent loops in a CF Worker: isolate CPU/time limits. Keep the brain on JVM.
• Cost & latency: external LLM providers cost money + add egress. Keep Workers AI for the instant edge pre-filter and cheap-path generation.
• New dependency: a JetBrains framework in the critical path. Mitigated by 1.0 stability guarantee + Apache-2 + it's "just Kotlin."
• Secrets: provider keys must be managed (GCP Secret Manager; never into the shadow env).
• Privacy: routing chat content to a third-party LLM is a policy decision — prefer self-hosted (Ollama/Llama Guard on k3s) for moderation of private messages.

5. Architecture — edge pre-filter + JVM brain

Two tiers, matching the split already established in the distributed-actors and performance docs:

user message
    │
    ▼
Cloudflare Worker (edge, Kotlin/JS) ← stays thin, instant, cheap
┌──────────────────────────────────────┐
│ 1. Workers AI isSafe() PRE-FILTER    │ ~edge-local, <50ms
│    └─ obvious violation → reject now │
│ 2. cheap small-talk → Workers AI     │ (optional fast path)
│ 3. otherwise → proxy to core agent   │
└───────────────────┬──────────────────┘
                    │ Pub/Sub or service binding (correlation id)
                    ▼
JVM reaktorServer (k3s, Spring Boot) ← the Koog brain
┌──────────────────────────────────────────────────────────┐
│ Koog agent (graph strategy)                              │
│ ├─ moderate() input (Llama Guard via Ollama on k3s)       │
│ ├─ tools: events · friends · knowledge graph (Memgraph)  │
│ ├─ memory: vector RAG over chat history + KG             │
│ ├─ structured output (typed extraction)                  │
│ ├─ moderate() output                                     │
│ └─ OpenTelemetry spans → Grafana LGTM                    │
└──────────────────────────────────────────────────────────┘

• Edge keeps: the instant isSafe() gate (reject blatant abuse before it costs a server hop) and optionally a cheap small-talk fast path on Workers AI.
• Core gets: the Koog agent — moderation with categories, tools, memory, structured output, multi-step reasoning — where there are no isolate limits and direct DB access.
• Wire: edge → core over a service binding (low latency) or Pub/Sub (durable), carrying a correlation id so the whole turn is one OTel trace.

6. Moderation upgrade (the direct ask)

Replace the boolean isSafe() with Koog's categorized moderate() on the JVM. Sketch:

// jvmMain — inside reaktorServer, behind the existing BotService contract
class KoogBotService(
  private val executor: PromptExecutor,        // MultiLLMPromptExecutor(openAI, ollama, …)
  private val moderationModel: LLModel,     // e.g. Ollama Llama Guard 3, or OpenAI omni-moderation
) : BotService() {

  override val generate = PostHandler<BotRequest.Generate, BotResponse>("/generate") { req ->
    // 1. moderate the INPUT — categorized, not boolean
    val inModeration = executor.moderate(prompt(req.input), moderationModel)
    if (inModeration.isHarmful) {
      return@PostHandler BotResponse.Failure.ContentViolation(
        reason = inModeration.violatedCategories(),     // Harassment, Violence, S3 …
      )
    }

    // 2. generate via a Koog agent (tools + memory available here)
    val answer = bestBudAgent.run(req.input)

    // 3. moderate the OUTPUT
    val outModeration = executor.moderate(prompt(answer), moderationModel)
    if (outModeration.isHarmful) return@PostHandler BotResponse.Failure.ContentViolation(outModeration.violatedCategories())

    BotResponse.Generated(answer)
  }
}

7. Koog × Reaktor graph synergy

Koog agents and Reaktor nodes are both graphs. They compose two ways:

Koog agent inside a Reaktor node

The existing BotService contract (commonMain) is unchanged. Its JVM implementation becomes a Koog agent invocation. To the rest of the graph it's still a ServiceNode with /generate + /chat-respond — the agent is an implementation detail.

Reaktor services as Koog tools

Koog tools wrap Reaktor services. The bot gains abilities by exposing existing services as tools — no new model training, just tool descriptors.

// Reaktor services become Koog tools the agent can call.
val bestBudAgent = AIAgent(
  executor   = executor,
  llmModel   = OpenRouterModels.…,          // or Workers AI / Ollama / Anthropic
  systemPrompt = AgentRegistry.BestBud.systemPrompt,
  tools = tools {
    tool("findEventsNearby")   { args -> eventService.nearby(args.userId, args.radiusKm) }
    tool("mutualFriends")      { args -> socialService.mutuals(args.a, args.b) }   // Memgraph
    tool("recentMessages")     { args -> messagingService.recent(args.chatId, args.limit) }
    tool("summarizeWeek")      { args -> chatService.weekDigest(args.chatId) }
  },
)

8. Bot as a distributed actor

The Koog bot brain is a textbook core actor from the distributed-actors design: stateful (conversation + memory), single-writer per conversation, supervised, and a natural fit for the JVM shard tier.

• One actor per conversation: BotConversationActor keyed by chat/user id. Its mailbox serializes turns (no interleaving), its ObjectStore holds Koog's persistent memory + checkpoints.
• Supervision: a model timeout or provider error is a Restart directive — the actor reactivates from its last checkpoint (Koog persistence) rather than losing the conversation. This addresses the "agents lose intelligence at restart" failure mode directly: state lives in the actor's ObjectStore, not process memory.
• Virtual activation: conversations are virtual actors — idle ones passivate, reactivate on the next message with full memory. Millions of potential conversations, only active ones resident.
• Observability: Koog OTel spans nest inside the actor's message-handling span → one trace from edge tap to model call to tool call to reply.

9. Future features unlocked

10. Provider & cost strategy

• Edge pre-filter: Workers AI isSafe() — already wired, edge-local, effectively free. Keep it as the first gate.
• Moderation (core): Llama Guard 3 via Ollama on k3s — self-hosted, private, zero per-call cost, categorized. Best for moderating private chat content.
• Routine generation: a small/cheap model (Workers AI Llama, or a local Ollama model) for small talk + simple replies.
• Hard tasks: a premium model (Anthropic / OpenAI / Gemini via Koog's unified executor) for planning, tool-heavy reasoning, long-context summarization — gated so it's only used when the cheap path can't.
• Caching: Koog LLM response caching + Anthropic prompt caching to cut repeated-prompt cost/latency (system prompts, tool schemas).
• Routing: a small classifier (or rules) decides cheap-vs-premium per turn — keeps the bill proportional to value.

11. Migration plan

Incremental and low-risk — the BotService contract in commonMain never changes; only its implementation evolves.

Phase 1~3 days

Koog moderation on the JVM

Stand up Ollama + Llama Guard 3 on k3s. Add a KoogModerationService on reaktorServer exposing categorized moderate(). Edge worker keeps isSafe() as pre-filter, calls the JVM moderation for the categorized verdict + audit record. DoD: a flagged message returns its category, not just "blocked".

Phase 2~1 week

Koog agent behind BotService

JVM KoogBotService implements /generate + /chat-respond via a basic Koog agent (no tools yet). Edge worker proxies non-trivial turns to it; keeps cheap small-talk on Workers AI. Wire Koog OpenTelemetry into the existing Grafana pipeline. DoD: bot replies come from Koog on JVM; one OTel trace spans edge→core→model.

Phase 3~1 week

Tools + structured output

Expose eventService, socialService, messagingService as Koog tools. Add structured output for RSVP/event extraction. DoD: "find events this weekend" returns typed event cards from a real service call.

Phase 4~1 week

Memory + RAG over the knowledge graph

Long-term memory + retrieval over Memgraph + chat history. Per-user/per-conversation memory. DoD: the bot recalls prior context across sessions.

Phase 5~1 week

Bot as a distributed actor

Wrap the agent as a BotConversationActor (per the distributed-actors design): per-conversation mailbox, ObjectStore-backed Koog memory/checkpoints, supervision-driven restart-from-checkpoint, virtual activation. DoD: a pod restart mid-conversation resumes from checkpoint with memory intact.

Phase 6ongoing

Planners, MCP, cost routing

Planner agents for multi-step features (event coordination, weekly digest), MCP for external integrations, cheap-vs-premium provider routing + caching.

12. Risks & open questions

Risks

• Latency/cost regression vs Workers AI. Edge Workers AI is fast + cheap; a JVM round-trip + premium provider is slower + costs money. Mitigation: edge pre-filter + cheap-path small talk; only escalate to the Koog brain when the turn warrants it.
• Running Koog on the CF Worker. Tempting (KMP/JS) but isolate limits make agent loops fragile. Mitigation: brain on JVM; if any Koog runs on JS, restrict it to a stateless single LLM call, not an agent loop.
• Provider lock-in / outage. Mitigation: Koog's multi-provider executor + self-hosted Ollama fallback means no single provider is critical-path.
• Privacy of chat content. Mitigation: self-hosted moderation + generation for private surfaces; managed providers only where consented.
• Memory-at-restart. A known failure mode for JVM agents (state lost on restart). Mitigation: Koog persistence/checkpoints stored in the actor's ObjectStore (externalized per the distributed-actors design) so restarts resume.
• New dependency surface. Mitigation: 1.0 stability guarantee, Apache-2, pure Kotlin; pin versions; it sits behind the stable BotService contract so it's swappable.

Open questions

• Moderation backend: self-hosted Llama Guard 3 (private, free, ops cost) vs OpenAI Omni (managed, costs, data leaves) — or both (self-host default, managed for appeals)?
• Which premium provider for hard tasks — Anthropic, OpenAI, or Gemini — and what's the cost ceiling per active user?
• Does any Koog run on the edge at all, or is the edge strictly Workers-AI + proxy?
• Vector store for RAG: Memgraph (already deployed, graph+vector) vs a dedicated vector DB?
• How does bot-generated content flow through the notifications + shadow pipelines (capture, replay, moderation audit)?
• Spring AI vs Koog-native on reaktorServer — Koog integrates with both; do we standardize on Koog's own abstractions?

13. References

Current code

• bestbuds/modules/app/src/jsMain/.../cloudflare/bot/BotWorker.kt — CloudflareBotService, Workers AI textGeneration + isSafe().
• bestbuds/modules/app/src/jsMain/.../cloudflare/bot/AgentRegistry.kt — BestBud / Writer / Support personas.
• bestbuds/modules/app/src/commonMain/.../data/services/BotService.kt — the stable contract Koog slots behind.
• reaktor-cloudflare/.../WorkersAI — the edge AI binding (keep as pre-filter).

Peer designs

• reaktor-distributed-actors — the bot brain as a core actor; checkpoint-on-restart.
• reaktor-notifications-design — bot-generated content delivery.
• shadow-dev-environment — capture/replay bot turns; moderation audit corpus.
• performance-observability — OTel pipeline Koog spans flow into.

Koog (external, verified May 2026)

• JetBrains/koog — GitHub (Apache-2, KMP: JVM/JS/WasmJS/Android/iOS)
• Koog 1.0 release — JetBrains AI blog
• Koog docs — overview
• Koog content moderation (moderate(), OpenAI Omni, Llama Guard 3)
• Koog model capabilities (providers + moderation categories)
• Koog Ktor plugin · Spring Boot integration

Doc lineage. Authored 2026-05-28. Grounded in the real BestBuds bot code (Cloudflare Worker + Workers AI) and verified Koog 1.0 facts (KotlinConf May 2026 release; KMP; moderate() with OpenAI Omni + Llama Guard 3; Spring Boot/Ktor; OpenTelemetry; tools/memory/MCP). Verdict: adopt as the JVM-side brain behind the existing BotService contract, edge stays a thin Workers-AI pre-filter. Composes with the distributed-actors, notifications, shadow, and performance designs.

Related docs

• Koog Workbench - Use before wiring real agent behavior into reaktorDesktop.
• Control Plane - Use when exposing Reaktor safely to humans and agents.
• Service Layer - Use before adding service APIs, splitting reaktor-service out of reaktor-graph, wiring handlers to Spring/Workers, or making modules graph-projectable without hard graph dependencies.
• Data Layer - Use before choosing a store, designing offline-first sync, or deciding authority vs cache vs projection for a datum; the centerpiece is the tier 1↔2/3 server-authoritative sync engine.
• Browser-Local AI - Use before adding WebLLM, Chrome built-in AI, or browser-local model providers to Reaktor Web, docs, or workbench surfaces.
• Server - Use before adding any production backend primitive (resilience, messaging, data patterns) or designing the Spring/Workers server runtime.