In this topic, we'll focus on the Thought component of an agent – the part responsible for understanding context, processing information, and planning actions. You can think of it as the agent's "brain". It allows the agent to break complex problems into manageable steps, learn from previous interactions, and continually adjust its strategy as new information becomes available.
Types of thought
Different reasoning patterns help agents handle various cognitive tasks. Here are the main types:
Type of thought | Example |
|---|---|
Planning | "To build this API, I need to: 1) design the database schema, 2) implement authentication, 3) create endpoints, 4) write tests" |
Analysis | "The application crashes only when processing files larger than 10MB, which suggests a memory allocation issue" |
Decision making | "Since API response time is critical here, I'll use caching even though it adds complexity" |
Problem solving | "We can resolve the circular dependency by introducing an interface layer between these two modules" |
Memory integration | "The user works on an e-commerce project in React, so I'll suggest state management solutions that fit this context" |
Self-reflection | "The recursive approach caused stack overflow; I should switch to using iteration instead" |
Goal setting | "Before implementing the feature, I need to define success metrics and edge cases to handle" |
Prioritization | "The payment processing bug affects revenue, so it takes priority over the UI alignment issue" |
Basic implementation
The overall logic for when and how to reason resides in the system prompt. This prompt tells the LLM how to think through problems, when to use tools, and how to check if it has achieved its goals. The system then calls the LLM with this prompt together with the user input.
Take a look at the example below:
SYSTEM_PROMPT = """You are an AI assistant capable of reasoning and using tools.
For each user request:
1. Analyze what needs to be done
2. Determine which tools (if any) are needed
3. Use tools to gather information or take actions
4. Evaluate the results
5. Continue until the goal is achieved
"""The system prompt serves as a powerful way to configure your agent. You can fully control which reasoning patterns to enable or disable.
Chain-of-thought (CoT)
Research shows that models deliver more accurate results when they show their intermediate reasoning steps before giving the final answer, instead of jumping straight to conclusions. This is especially true for mathematical problems, although it was more noticeable in older models (pre-2025).
Example:
Odd number sum problem without CoT
User: The odd numbers in this group add up to an even number: 4, 8, 9, 15, 12, 2, 1.
Model: True❌ Incorrect. Here, the model guessed without checking.
Odd number sum problem with CoT
User: The odd numbers in this group add up to an even number: 4, 8, 9, 15, 12, 2, 1.
Think step by step.
Model: Odd numbers: 9, 15, 1
Sum: 9 + 15 + 1 = 25
25 is odd.
Answer: False✓ Correct. By breaking down the calculation, the model arrives at the accurate answer.
Developers have created techniques to help models generate output gradually rather than immediately. This approach significantly improves quality. This technique, known as chain-of-thought (CoT), has become a standard practice and is now considered fundamental.
CoT in agent systems
Chain-of-thought has practical applications in agent architectures. The simplest way to implement it is by adding the phrase "think step by step" to your agent's system prompt.
For more reliable implementations, you can clearly define the reasoning structure using structured outputs. For example, OpenAI's API supports JSON schema definitions that enforce a specific thought structure, ensuring the agent follows a consistent reasoning pattern. This method offers better control and predictability, particularly in production environments where consistent behavior is essential. The main benefit of CoT in agents is transparency: by making the reasoning process visible, you can fix issues, understand decision-making paths, and build user trust by showing how the agent reaches its conclusions.
ReAct framework
Another well-known technique is ReAct (Reasoning and Acting), where the model breaks down its actions into multiple steps and moves forward gradually. The approach represents Reasoning + Acting.
Consider the following example:
Thought: The user wants to know the population of Tokyo. I need to search
for current demographic data.
Action: Search["Tokyo population 2024"]
Observation: Tokyo has approximately 14 million residents in the city proper,
and 37 million in the greater metropolitan area.
Thought: I have the information, but should clarify which figure the user
likely wants. The metropolitan area is more commonly referenced.
Action: Finish["Tokyo has approximately 37 million people in its metropolitan
area, making it the world's most populous metropolitan area. The city proper
has about 14 million residents."]We already implemented this approach when we discussed the agent loop earlier. ReAct has become a standard, foundational pattern in agent design.
Schema-guided reasoning (SGR)
An interesting extension of chain-of-thought is SGR – schema guided reasoning. This approach specifies which intermediate conclusions we need to prepare to produce better final answers, rather than just performing a generic "reasoning" step.
This approach is a logical evolution of chain-of-thought and works especially well when:
The agent operates in a complex domain
We understand what intermediate reasoning steps will lead to a quality answer
By providing a structured schema for reasoning, we help guide the agent through a more deliberate and domain-specific thought process.
Conclusion
The thought component turns agents from simple responders into purposeful problem-solvers. By choosing appropriate reasoning patterns – from basic chain-of-thought to structured schema-guided reasoning – agents can handle complex tasks more accurately and transparently.
Key takeaways:
Thought patterns are customizable through system prompts, letting you control agent reasoning
Chain-of-thought improves accuracy by making reasoning explicit
ReAct combines reasoning with action in an iterative loop
Schema-guided reasoning provides structured guidance for complex domains
Select reasoning patterns that fit your use case. The right mix of thought processes can significantly improve your agent's performance and reliability.