Reason and act (ReAct) prompting

When we think about how humans process information, we observe our environment, think about what actions to take, and then act. This cycle of observing, thinking, and doing helps us handle new challenges, learn from outcomes, and adjust our approach as needed.

In this topic, we will explore the ReAct framework, which applies this same observation-reflection-action loop to artificial intelligence systems. It helps LLMs break down complex tasks into separate reasoning and action phases, using observations from each action to guide the next step.

What is ReAct prompting?

ReAct prompting is an approach designed for language models proposed by Yao and others in 2022. It guides LLMs to produce reasoning traces and specific actions in a combined sequence. Drawing inspiration from the interaction between "reasoning" and "acting" (REasoning + ACTing = ReAct) in human thought processes, ReAct prompting advances the functionality of LLMs.

In the ReAct process, the LLM does this:

1. Reason — it analyzes the problem and plans the next step.

2. Act — it executes a chosen operation (e.g., generate text or make an API call).

3. Observe — it examines the outcome of that action.

4. React — it updates its understanding and adjusts the plan based on the observations.

5. Repeat — it cycles through these steps until the task is complete.

For example, imagine you want the LLM to check a given city's time and temperature. It would follow these steps:

1. Reason — first, it determines that it needs to respond with both the current time and temperature for the specified city.

2. Act (Weather) — it queries a weather API for the temperature.

3. Observe (Weather) — it receives a response: "68°F, cloudy."

4. React (Weather) — the LLM confirms that the weather requirement is satisfied.

5. Act (Time) — since we also need time, the LLM uses a tool to check the city's local time.

6. Observe (Time) — a response is received: "3:45 PM."

7. React (Time) — the LLM confirms that the time requirement is satisfied.

8. Finish — Since both pieces are obtained, it confirms that the task is complete and responds like: "In <city> it's 3:45 PM and 68°F with clouds."

The ReAct process helps language models solve more complex tasks by interacting with the environment to get additional data when needed. This approach has led to the development of ReAct agents — autonomous systems that use LLMs to reason and perform automated actions to achieve a given objective. We'll take a closer look at agents in upcoming topics.

Creating ReAct prompts

ReAct prompts are designed to guide the LLM to follow the ReAct process: reason, act, observe. Therefore, a good ReAct prompt:

• Instructs the LLM to reason about the task at hand (chain of thought reasoning);

• Specifies the tools available for the model to use.

• Guides the model to make observations after taking actions or reasoning.

• Instructs the model to loop over and over until the task is complete.

• Tells the model to respond appropriately once the task is complete.

Here’s an example:

You are built to assist with diverse tasks, including responding to queries, generating summaries, and performing various analyses.

Tools
You have at your disposal an extensive set of tools. You should invoke these tools in whatever order suits the task best.

This may involve decomposing the overall task into smaller steps and employing different tools for each step.
The tools available to you are:
{tool_desc}

Output Format
Please adhere to the structure below:
Thought: I need to call a tool to proceed.
Action: tool name (choose one from {tool_names})
Action Input: a JSON object with the arguments for the tool (for example, {"name": "Mike", "age": 15})

Always begin with a Thought.
Do NOT wrap your entire response in markdown code fences. You may include code fences inside your response if needed.
Ensure that Action Input is valid JSON. Do NOT use Python-style quotes like {{'name': 'Mike', 'age': 15}}.

When you invoke a tool, you will receive:
Observation: tool response
Continue the Thought/Action/Action Input/Observation cycle until you have gathered enough information. Then conclude with one of these formats:

Thought: I have sufficient information and can answer without using additional tools.
Answer: [your answer here]

Thought: Despite the tools, I cannot answer the question.
Answer: [your answer here]

Here, we are instructing the LLM to decompose the given task into smaller steps and use tools whenever needed to complete the task. After using a specified tool, the LLM should analyze the information and decide what to do next. This may involve using another tool, or, if there are no tools to solve that specific problem, respond with the appropriate answer.

Tools, in this case, are special functions that an LLM can run to perform a specific action. For example, the function could allow the LLM to perform a calculation, interact with a database, or an external API. Therefore, you will need an LLM that supports calling functions (also known as tool use) to use this functionality.

Note that the ReAct process is a loop. After every iteration, the LLM will decide what to do next. It could repeat (with different tools, or the same tool with different inputs), or present the final answer if the task is accomplished (or if there are no more tools to call). At this point, it will respond with the appropriate final answer or note that it was unable to perform the requested operation.

Example: reversing and concatenating strings

Now, let's look at an example. Imagine you want your LLM app to reverse a string and concatenate two strings. You want your users to send queries like:

Reverse the string 'hello' and concatenate it with 'world'.

Although modern LLMs can handle this task without additional tools, let's assume we have two tools:

• One to reverse a given string.

• Another one to concatenate two strings.

DEFINE function concatenate_strings(str1, str2)
    RETURN str1 + str2

DEFINE function reverse_string(str)
    RETURN reversed version of str

When the LLM receives the request "Reverse the string 'hello' and concatenate it with 'world'," it starts by framing the user's instruction in its conversation history. The model then creates a reasoning step, noting that it needs to reverse the input first. It calls the reverse function with "hello," gets the output "olleh," and adds that result to its context.

With the reversed string ready, the agent reviews the overall goal and realizes that we also need to perform the concatenation. It calls the concatenation function using "olleh" and "world," gets "ollehworld," and adds this outcome to its dialogue history. Now, with both intermediate results, the model knows it has met the user's request.

Finally, the LLM creates a clear, natural-language answer based on the collected information:

"The concatenated result of the reversed string 'hello' and 'world' is 'ollehworld'." 

By switching between reasoning steps ("I need to do X next") and exact function executions, the system ensures both clarity in its decision process and reliability in its operations.

If you want to see this in action, try out the code yourself. After you install the packages and set up your API keys, you'll see the following output for a given query on reversing or concatenating strings:

Thought: I need to use a tool to reverse the string 'hello' first.
Action: reverse_string
Action Input: {"string":"hello"}
Thought: Now that I have reversed the string 'hello' to 'olleh', I will concatenate it with 'world'.
Action: concatenate_strings
Action Input: {'first_string': 'olleh', 'second_string': 'world'}
Thought: I can answer without using any more tools. 
Answer: The concatenated result of the reversed string 'hello' and 'world' is 'ollehworld'.

Advantages and use cases of ReAct prompting

Why should we use the ReAct prompting approach?

• Models can reason and interact with the environment to perform complex actions.

• It enables models to adjust their thinking and plans as conversations progress, making them respond more intelligently to changing situations.

• It connects LLMs with external resources, such as online databases, allowing them to provide more accurate answers using current information.

• By combining thinking and action, ReAct helps prevent the model from fabricating facts or making errors that get worse over time.

• When LLMs pair their reasoning with clear action steps, their responses become more understandable and trustworthy to users.

Here are some real-world applications:

1. ReAct tackles complex questions and shows its reasoning process, which is especially valuable for advanced topics where basic zero-shot prompting falls short.

2. For time-sensitive information, ReAct enables models to verify current databases, ensuring they base their reasoning on the latest data.

3. ReAct has pioneered the development of agents, expanding LLM capabilities beyond text generation.

Conclusion

In this topic, you learned about ReAct prompting. It replicates how people think by using the 'reasoning' and 'acting' capabilities of our brains in LLMs. Combined with functions that perform special operations, this method enables LLMs to complete complex tasks they normally couldn't handle on their own.