When RAG is not needed
RAG, a search in a document base before the answer, is needed when there is a lot of knowledge and it lives in texts. In many tasks this is not the case.
The first case: there is little knowledge. A price list, addresses, opening hours and a dozen rules fit into the agent’s instructions in full. In September 2024 Anthropic wrote that a knowledge base under 200,000 tokens, about 500 pages, can go into the prompt to the model in full. A token is a piece of a word, the unit in which the model counts text.
Long instructions have a price: the model reads them with every question, and you pay for every token it reads. While the instructions take a few pages, the cost is tiny; when they grow to hundreds of pages, a search in a base becomes cheaper.
The second case: the answer lives in a system, not in a document. A free slot, an order status and warehouse stock change every minute, and a document base will lag behind them.
The third case: the task is about action, not knowledge. Create a deal, send a reminder, build a report: the agent needs tools, not a text search.
This matches the advice in Anthropic’s article “Building effective agents” (December 2024): start simple and add complexity only if it brings results. Each extra part of an agent is one more place to test and update.
Instructions and examples instead of a base
Instructions, or a system prompt, are the text the agent reads before every conversation: who it is, what it does, what it does not do and when it calls a person.
Examples often work better than rules. In the paper on the GPT-3 model (May 2020), the authors showed that the model solves new tasks without fine-tuning if the prompt gives it the task and a few sample solutions. For an agent, this means three to five samples of a good answer right in the instructions.
What goes into the instructions: the price list and terms, the tone of answers, a list of forbidden promises, sample answers to frequent questions and the procedure for handing a conversation to a person.
An example for a booking agent: the role (“you are a clinic receptionist”), a list of services with their length, the rule “book only into free slots from the calendar”, three sample dialogs and the condition under which a conversation goes to the receptionist. This is two or three pages of text, and there is no knowledge base here.
The line is simple: if the instructions have grown to dozens of pages and are hard to update, it is time to move the knowledge into a base. How an agent with a knowledge base works is covered separately.
Data from systems through tools
An agent tool is an action the agent can call: read a calendar, find a customer in the CRM, create a task. The model itself decides which tool a question needs and gets fresh data at the moment of the answer.
In n8n this rule is written right in the documentation: the AI Agent node needs a model and at least one tool, and the agent itself chooses which one to call.
An example of a report agent: every morning it reads yesterday’s deals from the CRM and payments from 1C accounting software and sends the manager a short summary. It needs two read-only tools and instructions on which figures to count and how to show them. A document base is of no use to it: all the figures are fresh and live in the systems.
There is a common standard for connecting to systems. On 25 November 2024 Anthropic opened the MCP protocol (Model Context Protocol): one way to connect AI to data and services. An MCP server for your CRM is written once, and different agents use it.
Rights are given at a minimum: a report agent needs only read access, and a booking agent needs write access only to the calendar. How an agent works with a CRM, email and 1C is covered on the page about agent tools.
Comparing the two approaches
An agent with a knowledge base looks for the answer in documents. The RAG method was described in 2020: before answering, the model finds pieces in the base and answers from them without training. It suits support based on internal rules, a product help desk and search across contracts.
An agent without a base relies on instructions and tools. It suits customer bookings, taking requests, reports from the CRM and 1C, and reminders.
The difference in the build is visible. A knowledge base needs you to collect and clean documents, cut them into pieces, choose an embedding model and test the search quality. An agent without a base needs instructions, examples and access to the systems.
There is also a difference in support after launch. A knowledge base is updated every time a document changes, otherwise the agent answers from the old version. A tool always takes current data from the system, but it breaks when the system itself changes: a new field in the CRM or a different export format from 1C.
Often the right answer is a mix: a support agent looks up return rules in the base and the order status in the CRM. If you are not sure whether you need an agent at all or a workflow with fixed steps is enough, compare the options on the page agent or workflow.