Glossary
What is agent reflection
In short
Agent reflection is a technique in which an AI agent evaluates its result or a failed attempt, writes down what it learned and makes the next attempt with those notes.
AI agents · reflection
MMOOZZGG.com editorsChecked: 1 min read
For a child
When you write a dictation at school, the teacher tells you to read it again before you hand it in. You find a missing letter and fix it. And if you made a mistake last time, you remember it and write more carefully. A smart program can do the same: check its work, see where it went wrong and try again. But it still needs the teacher to check it.
In plain words
A standard model gives an answer on the first try and does not go back to it. With reflection, the work goes in several rounds. The agent makes a draft, then looks at it critically: are all points of the task done, are there contradictions, did the tests pass? It writes down the shortcomings in words and redoes the work. If an attempt failed, the lesson stays in memory and is used next time. For example, an agent wrote an email to a customer, read it again, noticed it forgot the delivery date and added it.
For business
The technique raises quality where you can check the result: the code passes tests, the table matches the totals, the answer matches the policy. You pay for it in time and tokens: each round of checking costs as much as one more request to the model. The main caveat is that a model is poor at catching its own slips without an outside signal. A study presented at ICLR 2024 showed that without outside feedback, language models struggle to fix their reasoning and sometimes answer worse after self-checking. So it is safer to give the agent an outside check: tests, a check against a database, rules. How to set this up and who is responsible for a failure is covered in the article on AI errors.
For a specialist
Reflection in agent systems is a loop in which the model generates a result, receives an evaluation and turns it into text feedback for the next attempt, without changing its weights. In the Reflexion method (Shinn and co-authors, 2023), the agent keeps such self-reports in episodic memory. The signal can be a number or free text, from outside or simulated by the system itself. The authors reported 91% pass@1 on HumanEval, against 80% for GPT-4. Self-Refine (Madaan and co-authors, 2023) makes one model the generator, the critic and the editor: across seven tasks, the average gain was about 20 percentage points. Huang and co-authors (ICLR 2024) showed the limit: without outside feedback, self-correction of reasoning often does not help. Anthropic calls a similar scheme evaluator-optimizer: one model call generates, another evaluates and returns remarks.
Example
An agent prepares a monthly sales report. A check against the CRM shows a mismatch in the total for one region. The agent writes down the reason, recalculates the table and next month takes the numbers from the right field at once.
Related terms
Questions and answers
Can a model find all its own mistakes?
Sources
- Shinn et al., Reflexion: Language Agents with Verbal Reinforcement Learning, 2023arxiv.org
- Madaan et al., Self-Refine: Iterative Refinement with Self-Feedback, 2023arxiv.org
- Huang et al., Large Language Models Cannot Self-Correct Reasoning Yet, ICLR 2024arxiv.org
- Anthropic, Building effective agents, 19.12.2024anthropic.com
Not sure whether your company needs this?
AI solutions map