Building a Koog agent

In this topic, we will build a complete, practical Koog agent step by step — starting from tool definitions, then adding structured output, session memory, and error handling. The agent we build here is a customer support assistant for a fictional e-commerce platform. The following topic extends it with long-term knowledge retrieval using RAG.

Defining tools

Our agent handles customer support requests. It needs to look up order status and process refunds using tools, return responses in a structured format suitable for downstream processing, remember the conversation within a session, and handle failures gracefully.

The agent needs two tools: one to look up order status and one to process refunds. Both are defined as annotated functions inside a ToolSet class:

import ai.koog.agents.core.tools.ToolSet
import ai.koog.agents.core.tools.annotations.LLMDescription
import ai.koog.agents.core.tools.annotations.Tool

class SupportTools : ToolSet {

    @Tool
    @LLMDescription(
        "Looks up the current status of a customer order. " +
        "Returns the order status and estimated delivery date."
    )
    fun getOrderStatus(
        @LLMDescription("The unique order ID, e.g. 'ORD-12345'")
        orderId: String
    ): String {
        // Real implementation would query a database here
        return when (orderId) {
            "ORD-12345" -> "Status: In Transit. Estimated delivery: tomorrow by 5 PM."
            "ORD-99999" -> "Status: Delivered on 2025-06-10."
            else -> "Order $orderId not found."
        }
    }

    @Tool
    @LLMDescription(
        "Initiates a refund for a delivered order. " +
        "Only call this if the customer explicitly requests a refund and the order has been delivered."
    )
    fun processRefund(
        @LLMDescription("The unique order ID to refund")
        orderId: String,
        @LLMDescription("The reason for the refund, as stated by the customer")
        reason: String
    ): String {
        // Real implementation would call a payment API here
        return "Refund for order $orderId initiated successfully. Reason recorded: '$reason'."
    }
}

A few things worth noting:

• The @LLMDescription on processRefund explicitly says "only call this if the order has been delivered" — this is a guardrail written in natural language. The LLM reads it and will avoid initiating refunds for orders that are still in transit, even if the customer asks.

• The return type is String in both cases, which keeps the tool simple. For tools that need to return structured data, you can also return a @Serializable data class — Koog serializes it to JSON automatically.

Now register the tools and build the agent:

import ai.koog.agents.core.agent.AIAgent
import ai.koog.agents.core.tools.ToolRegistry
import ai.koog.agents.core.tools.reflect.asTools
import ai.koog.prompt.executor.clients.openai.OpenAIModels
import ai.koog.prompt.executor.llms.all.simpleOpenAIExecutor
import kotlinx.coroutines.runBlocking

fun main() = runBlocking {
    val toolRegistry = ToolRegistry {
        tools(SupportTools().asTools())
    }

    val agent = AIAgent(
        promptExecutor = simpleOpenAIExecutor(System.getenv("OPENAI_API_KEY")),
        systemPrompt = """
            You are a customer support assistant for ShopFast.
            Be concise and policy-aware.
            Never invent order data — always use the available tools to look it up.
            Only process refunds when the customer explicitly requests one and the order is delivered.
        """.trimIndent(),
        llmModel = OpenAIModels.Chat.GPT4oMini,
        toolRegistry = toolRegistry
    )

    val result = agent.run("What's the status of my order ORD-12345?")
    println(result)
}

Structured output

Right now the agent returns a plain String. For a real backend service, you typically want a structured response so the calling code can branch on the outcome — did the agent resolve the issue, or does it need escalation?

In Koog, structured output is defined with @Serializable, @SerialName, and @LLMDescription annotations on a Kotlin data class. Koog generates a JSON schema from the class automatically and uses it to constrain the model's response:

import ai.koog.agents.core.annotation.LLMDescription
import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable

@Serializable
@SerialName("SupportResponse")
@LLMDescription("A structured response from the customer support agent.")
data class SupportResponse(
    @property:LLMDescription(
        "Whether the customer's issue was fully resolved. " +
        "False if the agent needs to escalate or couldn't help."
    )
    val resolved: Boolean,

    @property:LLMDescription("A concise, customer-facing reply explaining what was done or what happens next.")
    val reply: String,

    @property:LLMDescription(
        "The category of the request: ORDER_STATUS, REFUND, or OTHER."
    )
    val category: RequestCategory
)

@Serializable
@SerialName("RequestCategory")
enum class RequestCategory { ORDER_STATUS, REFUND, OTHER }

To get a structured response from the agent, use nodeLLMRequestStructured inside a custom strategy graph. The StructureFixingParser adds automatic error correction — if the model's JSON is malformed, Koog retries the parse using an auxiliary model call up to the specified number of times:

import ai.koog.agents.core.dsl.builder.forwardTo
import ai.koog.agents.core.dsl.builder.strategy
import ai.koog.agents.core.dsl.extension.node
import ai.koog.agents.core.dsl.extension.nodeLLMRequest
import ai.koog.agents.core.dsl.extension.nodeLLMRequestStructured
import ai.koog.prompt.executor.model.StructureFixingParser

val supportStrategy = strategy<String, String>("support") {

    // Standard tool-calling loop — the LLM calls tools until it's ready to answer
    val think by nodeLLMRequest()
    val executeTools by nodeExecuteTools()
    val sendToolResults by nodeLLMSendToolResults()

    // Once the LLM has all the information it needs, ask for a structured response
    val structuredReply by nodeLLMRequestStructured<SupportResponse>(
        fixingParser = StructureFixingParser(
            model = OpenAIModels.Chat.GPT4oMini,
            retries = 3
        )
    )

    // Process the result — branch on success or failure
    val processResult by node<Result<StructuredResponse<SupportResponse>>, String> { result ->
        result.getOrNull()?.data?.let { response ->
            buildString {
                appendLine("Resolved: ${response.resolved}")
                appendLine("Category: ${response.category}")
                appendLine("Reply: ${response.reply}")
            }
        } ?: "Failed to produce a structured response: ${result.exceptionOrNull()?.message}"
    }

    // Wire the graph
    edge(nodeStart forwardTo think)
    edge((think forwardTo executeTools) onToolCalls { true })
    edge((think forwardTo structuredReply) onTextMessage { true })
    edge(executeTools forwardTo sendToolResults)
    edge(sendToolResults forwardTo think)
    edge(structuredReply forwardTo processResult)
    edge(processResult forwardTo nodeFinish)
}

Then pass the strategy to the agent:

val agent = AIAgent(
    promptExecutor = simpleOpenAIExecutor(System.getenv("OPENAI_API_KEY")),
    systemPrompt = """
        You are a customer support assistant for ShopFast.
        Be concise and policy-aware.
        Never invent order data — always use the available tools to look it up.
        Only process refunds when the customer explicitly requests one and the order is delivered.
    """.trimIndent(),
    llmModel = OpenAIModels.Chat.GPT4oMini,
    toolRegistry = toolRegistry,
    strategy = supportStrategy
)

val result = agent.run("I want to return my order ORD-99999, it arrived broken.")
println(result)
// Resolved: true
// Category: REFUND
// Reply: I've initiated a refund for order ORD-99999. Reason recorded: 'arrived broken'.
//         You should receive the funds within 3-5 business days.

Session memory with ChatMemory

At this point the agent handles a single turn. But a real support conversation spans multiple messages — the agent needs to remember that the customer already mentioned their order number, or that they already confirmed they want a refund.

Install ChatMemory to persist conversation history across agent.run() calls within the same session:

import ai.koog.agents.features.memory.ChatMemory

val agent = AIAgent(
    promptExecutor = simpleOpenAIExecutor(System.getenv("OPENAI_API_KEY")),
    systemPrompt = """
        You are a customer support assistant for ShopFast.
        Be concise and policy-aware.
        Never invent order data — always use the available tools to look it up.
        Only process refunds when the customer explicitly requests one and the order is delivered.
    """.trimIndent(),
    llmModel = OpenAIModels.Chat.GPT4oMini,
    toolRegistry = toolRegistry,
    strategy = supportStrategy
) {
    install(ChatMemory) {
        // In production, swap this for a database-backed ChatHistoryProvider
        windowSize(20) // keep the last 20 messages; older messages are dropped
    }
}

// The session ID ties turns together — same ID means the agent loads the
// previous conversation history before responding
val sessionId = "customer-session-alice"

agent.run("What's the status of my order ORD-12345?", sessionId)
// → Looks up order, replies "In Transit, delivery tomorrow"

agent.run("Actually I'd like to return it.", sessionId)
// → Agent remembers it's order ORD-12345 and that it's still in transit;
//   declines the refund per policy and explains why

The session ID is the key that ChatMemory uses to store and retrieve history. Different session IDs produce completely separate conversation threads — useful for handling multiple customers concurrently without their histories bleeding into each other. The default windowSize of 20 messages is a reasonable starting point; raise or lower it based on your token budget and how long your typical conversations run.

Error handling

Two things go wrong in practice: tools throw exceptions, and the LLM produces responses that can't be parsed into the expected structure even after fixing retries. Here's how to handle both.

Tool errors propagate out of agent.run() as exceptions unless you catch them inside the tool itself. The simplest approach is to return an error string from the tool rather than throwing — the agent then tells the LLM what went wrong, and the LLM can decide how to respond:

@Tool
@LLMDescription("Looks up the current status of a customer order.")
fun getOrderStatus(
    @LLMDescription("The unique order ID, e.g. 'ORD-12345'")
    orderId: String
): String {
    return try {
        orderDatabase.find(orderId)?.toStatusString()
            ?: "Order $orderId not found."
    } catch (e: DatabaseException) {
        // Return a descriptive error string rather than throwing.
        // The LLM will include this in its response to the customer.
        "Failed to retrieve order $orderId: service temporarily unavailable. Please try again."
    }
}

Structured output failures are handled in the processResult node we already wrote — the Result<StructuredResponse<SupportResponse>> wrapper carries either the parsed data or the exception from the last failed fixing attempt:

val processResult by node<Result<StructuredResponse<SupportResponse>>, String> { result ->
    result.fold(
        onSuccess = { structured ->
            val response = structured.data
            buildString {
                appendLine("Resolved: ${response.resolved}")
                appendLine("Category: ${response.category}")
                appendLine("Reply: ${response.reply}")
            }
        },
        onFailure = { error ->
            // Log the failure for observability, return a safe fallback to the caller
            println("Structured output failed: ${error.message}")
            "I'm sorry, I encountered an issue processing your request. Please try again or contact support."
        }
    )
}

Agent-level failures — if something goes wrong that isn't caught inside a tool or node (for example, the LLM provider is unreachable) — surface as exceptions from agent.run(). Wrap it in a try-catch at the call site:

val result = try {
    agent.run(userInput, sessionId)
} catch (e: Exception) {
    println("Agent run failed: ${e.message}")
    "I'm unable to process your request right now. Please try again in a moment."
}

For production workloads where a mid-session crash would be costly, Koog also provides a Persistence feature (install(Persistence) { ... }) that checkpoints the agent's execution state after each node — so a crashed agent can resume from exactly where it left off. That's beyond the scope of this topic but is worth knowing about for long-running sessions.

Event handlers for observability

As a finishing touch, add event handlers to log what the agent is doing. In production you'd send these to Langfuse, W&B Weave, or your own logging system — here we just print to stdout:

val agent = AIAgent(
    promptExecutor = simpleOpenAIExecutor(System.getenv("OPENAI_API_KEY")),
    systemPrompt = "...",
    llmModel = OpenAIModels.Chat.GPT4oMini,
    toolRegistry = toolRegistry,
    strategy = supportStrategy
) {
    install(ChatMemory) { windowSize(20) }

    handleEvents {
        onToolCallStarting { ctx ->
            println("[Tool] Calling '${ctx.toolName}' with: ${ctx.toolArgs}")
        }
        onToolCallFinished { ctx ->
            println("[Tool] '${ctx.toolName}' returned: ${ctx.toolResult}")
        }
        onAgentFinished { _, result ->
            println("[Agent] Finished. Result: $result")
        }
    }
}

The agent we've built in this topic covers the full Thought → Act → Observe loop in a realistic scenario:

User message
    → ChatMemory loads session history
    → LLM thinks, calls getOrderStatus or processRefund as needed
    → Tools execute, results fed back to LLM
    → LLM produces SupportResponse (structured, with StructureFixingParser as fallback)
    → processResult node extracts or handles failure
    → ChatMemory saves updated session history
    → Structured result returned to caller

The next topic adds LongTermMemory to give this same agent persistent, searchable knowledge about your product catalogue and policies — so it can answer questions it wasn't explicitly programmed to handle.

Conclusion

Let's recap what we've learned in this topic:

• Tools are defined with @Tool and @LLMDescription on functions inside a ToolSet. The @LLMDescription text is the primary mechanism for telling the LLM when and how to call each tool — precise descriptions act as natural-language guardrails.

• Structured output is defined with @Serializable, @SerialName, and @LLMDescription on a Kotlin data class, and produced via nodeLLMRequestStructured. The StructureFixingParser adds automatic JSON correction using an auxiliary model call when parsing fails.

• ChatMemory persists conversation history across agent.run() calls within the same session ID. A windowSize limit prevents unbounded token growth.

• Tool errors are best handled by returning descriptive error strings rather than throwing, so the LLM can incorporate the failure into its response. Structured output failures and agent-level exceptions are caught with Result.fold and try-catch respectively.