Step 1: documents
A RAG system takes seven steps: collect documents, split them into chunks, turn the chunks into vectors, store them in a vector database, find chunks for a question, pass the question and the chunks to the model, and get an answer with a source link. The model is not trained; it answers from what was found.
The first step is to collect documents. Take what employees and clients ask about most often: internal rules, the price list, contract terms, support answers. Remove outdated versions right away, or search will find them too.
If the knowledge already sits in a CRM, an SQL database or an internal wiki, you do not have to move it. The LangChain documentation advises connecting such a system to the agent as a tool.
1. No code. In Flowise, open “Document Stores”, add a store and choose a loader for the file format, such as “PDF Loader”. Add your own metadata tag: later you can use it to pick chunks from exactly this set.
2. With code. In the LangChain example, each PDF page becomes a Document object with text and metadata: the file name in the source field and the page number in the page field. You will need these fields in step 6 for the source link.
The same no-code setup can also be built in Dify. It is distributed under Apache 2.0 with two conditions: without permission you may not run a multi-user cloud service like Dify or remove the logo from the console.
If the documents contain clients’ personal data, 152-FZ, Russia’s personal data law, applies: the data is collected and stored in databases in Russia, and Roskomnadzor, the Russian data regulator, is notified before any transfer abroad. So the vector database and the models for such documents go on your own server; more on the page on data security.
Step 2: chunks
The second step is to split the documents into chunks. The model gets not the whole file but a few pieces on the topic of the question. This makes the answer more accurate and cheaper, because models are paid by the volume of text.
The chunk size is set in characters. The overlap repeats the end of the previous chunk at the start of the next one, so a thought at the boundary is not lost.
1. No code. In Flowise, choose the “Recursive Character Text Splitter”: it looks for natural boundaries, such as the ends of sentences and sections. For Markdown texts there is the “Markdown Text Splitter”, and for web pages the “HTML-to-Markdown Text Splitter”.
2. Look at the chunk preview. In the Flowise documentation example, a chunk is 1,500 characters with an overlap of 750. Each piece should read on its own, and you can delete extra ones by hand after processing.
3. With code. In LangChain the same method is set with the line RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200, add_start_index=True). The add_start_index parameter remembers where the chunk sits in the source document.
There is no ready size for every case. The Flowise documentation directly advises trying different splitters, sizes and overlaps on your own documents.
Step 3: embeddings
The third step is to turn the chunks into vectors. An embedding is a set of numbers that records the meaning of a chunk: texts with similar meaning get similar sets. A separate embedding model calculates the vectors, not the one that writes the answers.
The vector length depends on the model, usually from 384 to 1,024 numbers. Documents and questions are processed by the same model. The dimension of the model and of the vector database index must match, or the upload will fail with an error.
Russian documents need a multilingual model. For example, bge-m3 in the Ollama catalog supports more than 100 languages and accepts text of up to 8,192 tokens, the pieces of words a model counts in.
1. No code. In Flowise, click the “Select Embeddings” card and choose “Ollama Embeddings” so that the vectors are calculated on your computer or server.
2. With code. Download the model with the command ollama pull bge-m3. To check: ollama run bge-m3 “Hello” prints a vector as an array of numbers.
3. In LangChain, connect the model with the langchain-ollama package: pip install -qU langchain-ollama, then OllamaEmbeddings(model=”bge-m3”) in the code.
Step 4: vector database
The fourth step is to store the vectors in a vector database. This is a store that quickly finds vectors close to the vector of the question. Which open databases exist and how to choose one is covered on the page about vector databases.
1. No code. In Flowise, click “Select Vector Store” and choose a database.
2. Add a “Record Manager”. It keeps a fingerprint of each chunk and skips what is already in the database when you upload again.
3. Click “Upsert” to load the chunks into the database.
4. With code. For a trial, LangChain has InMemoryVectorStore: a database in the program’s memory, with nothing to install. Chunks are loaded with the line vector_store.add_documents(documents=all_splits).
To work with people, replace the in-memory database with a persistent one. LangChain has ready packages for Chroma, Milvus, Postgres and Qdrant, and the search code barely changes.
Step 5: search
The fifth step is to find chunks for the question. The same model turns the question into a vector, and the database returns the nearest chunks. For search by meaning, the Ollama documentation advises cosine similarity, the angle between vectors.
The top K parameter sets how many chunks to return; in Flowise the default is 4. With too few chunks, the answer will miss some facts. With too many, the answer costs more and the prompt gets extra text.
1. No code. In Flowise, click “Retrieval Query” in the store and enter a test question. You will see which chunks were found.
2. With code. In LangChain, search runs with the vector_store.similarity_search method and the question text.
3. The similarity_search_with_score method also returns a similarity score. Its meaning depends on the database: in the LangChain documentation example, a smaller number means closer similarity.
Step 6: answer with a source link
The sixth step is to pass the question and the found chunks to the model. In the model’s instructions, write: answer only from the chunks, and if the answer is not in them, say so and pass the question to a person. Why an agent without this rule makes mistakes is covered in the article on why an agent answers wrongly.
The result of the scheme is an answer with a source link. The link shows the file and page where the fact comes from, and an employee can check the answer without rereading the whole document.
1. No code. In Flowise Agentflow V2, connect the vector database to the “Agent” block and turn on “Return Source Documents”: together with the answer, the agent returns information about the sources.
2. If you need only search without the agent’s reasoning, add a “Retriever” block and choose the “Text with Metadata” format: the chunks will arrive with the document names.
3. With code. Label each found chunk with the source and page fields from step 1 and pass it to the model with the question. Ask the model to put the chunk number in the answer, and the program will replace the number with the file name and page.
How such an agent answers employees and clients from the company’s knowledge base is covered on the page about the RAG agent.
Step 7: quality check
Check the answer with its source link before launch: is the answer right, and does the link lead to where the fact is written? Collect a set of real questions with correct answers and sources, including questions the documents cannot answer.
Check search and answer separately. If the right text is not among the found chunks, change the splitting, the overlap or top K. If the chunks are right but the answer is wrong, fix the model’s instructions or take a stronger model.
In n8n, light evaluations run such a test. On your own server they are available in the Community edition after free registration.
1. Create a data table or a Google Sheet with columns: question, expected answer and an empty column for the agent’s answer.
2. Add an “Evaluation Trigger” node to the workflow: it feeds questions from the table one by one.
3. After the agent, add an “Evaluation” node with the “Set outputs” action and write the answer into the empty column.
4. Click “Execute workflow” to the left of the evaluation trigger and compare the answers with the expected ones.
Repeat the test after every document update. How to build the question set and which metrics to count is covered on the page about agent testing.