Computer scienceProgramming languagesKotlinAI engineering with KotlinBuilding agents in Kotlin

What is an LLM agent?

14 minutes read

In this topic, you'll learn what an LLM agent is, how it differs from an LLM or a chatbot, what its core building blocks are, why its behavior is less predictable, and when it is the right choice for a task.

What is an LLM agent?

An LLM agent is a system that carries out tasks given in natural language. It uses an LLM to plan, calls tools to interact with its environment, and keeps a memory of what has happened. Because the LLM decides what to do, both the process and the outcome are non-deterministic.

A common way to explain agents is to compare them to a butler.

A plain LLM is a butler locked in a windowless, dark room. He is well-informed and can talk to you, but he can't see what is happening outside. If you ask about the weather, he will try to please you with an answer that may be made up. The same happens when you ask an online shop's assistant whether a product is in stock: the LLM has never seen the shop's catalog, so it can only guess.

Give him a window and a thermometer, and he can check the weather before answering. These are tools: ways for an agent to read from or act on the outside world.

Give him access to your wardrobe and ask him to prepare an outfit, and he has to plan: check the weather, look at the clothes, then choose. He works out the steps himself.

Finally, give him a notebook, and he can remember what you told him earlier, such as your clothing preferences. That is memory.

Butler

Agent component

Talks in the dark room

LLM

Works out the steps himself

Planning by the LLM

Window and thermometer

Tools for reading

Wardrobe

Tools for acting

Notebook

Memory

How an agent differs from a chatbot

A plain LLM or a simple chatbot answers in one step: a question goes in, and a reply comes out. It can only use what it learned during training and what is in the conversation so far. An agent can go and get what it needs, and it can do things in the world.

Plain LLM or chatbot

Agent

Information

Training data and the conversation

Also fresh or private data, fetched through tools

Actions

None. It only produces text

Can change things through tools, such as placing an order or sending an email

Steps per request

One model call

Several model calls, with tool calls in between

Who chooses the steps

The developer's code

The model

The last row is the key difference. In a chatbot, your code decides what happens: it sends the message and shows the reply. In an agent, the model decides which tool to use next, looks at the result, and decides again, until it has enough to give a final answer. A chatbot can have memory too, such as a conversation history. What makes a system an agent is the combination of planning and tools.

Core building blocks

Every agent is built from the same three parts.

The LLM is the decision maker. It reads everything it has been given, decides whether to answer or to use a tool, and writes the final reply. It never acts directly: when it wants a tool, it returns the tool's name and arguments as text, and your application, or the framework you use, executes the call and passes the result back to the model.

Tools connect the agent to the outside world. Tools for reading, such as a catalog search or a database query, bring in information the model doesn't have. Tools for acting, such as managing a cart or sending a message, change something outside the conversation. The model chooses a tool based on its name and description, so tools have to be described clearly.

Memory is how the agent keeps track of what happened. Models are stateless, so the application stores the information and puts the relevant part into each request. Memory for the current conversation is short-term memory. Information that should survive between conversations, such as a user's preferences, is long-term memory.

Everything the model sees in one request is called its context: the system prompt, the conversation so far, the descriptions of the available tools, and the results of earlier tool calls. The context is limited in size, costs money, and takes time to process, so deciding what goes into it is part of building an agent.

Non-determinism in action paths

A plain LLM already produces different text for the same prompt. With an agent, the difference goes further. The sequence of actions can vary, and that can change the result.

Ask the butler to choose an outfit twice, and he may check the weather first one day and look through the wardrobe first the next. He may pick a sweater and jeans, or a jacket and trousers. Both are reasonable, and the goal is reached either way.

Agents behave the same way. Given the same request, one run may search by keyword and another may browse by date. One may call three tools, and another may call only one. The final answers can differ because they are based on different information. And since some tools act on the outside world, different paths can also leave different effects behind, not just different wording.

This is not a bug to be removed. It follows from the fact that the model chooses the steps. It does mean that you can't predict an agent's exact path in advance, and that you need safeguards for the actions that matter.

When an agent is the right fit

An agent is a good choice when:

  • the steps depend on what earlier steps return, so you can't write them down in advance;

  • the task needs information that the model doesn't have, such as live or private data;

  • the task requires actions in other systems;

  • a variable path is acceptable, as long as the goal is reached.

An agent is usually the wrong choice when:

  • the steps are always the same. A fixed sequence is simpler, cheaper, and more predictable as ordinary code, even if some steps call an LLM;

  • one model call is enough, for example to summarize or translate a text;

  • every run must follow exactly the same path, or a mistake would be costly and can't be checked or reversed.

Because an agent makes several model calls and decides on its own what to do, it is also slower and more expensive than a single call. Use it when the flexibility is worth that price.

Conclusion

An LLM agent combines an LLM that plans, tools that let it read from and act on the outside world, and memory that keeps track of what happened. In a chatbot, your code decides what happens next. In an agent, the model does. This lets agents handle tasks whose steps can't be fixed in advance, but it also makes their behavior non-deterministic: the same request can lead to different sequences of actions, and so to different results.

Use an agent when a task needs live data, actions, and steps that depend on intermediate results. When the steps are fixed or one call is enough, a simpler design is the better choice.

How did you like the theory?
Report a typo