RAG for company knowledge: AI search with sources and access rules

What RAG adds to search

Retrieval-augmented generation (RAG) means finding relevant document passages and giving them to a language model as the basis for an answer. Ordinary search returns results for you to read and combine. In this example, the assistant combines a reservation rule with a collection location from another document. Both source passages must remain accessible from the answer.

RAG does not retrain the model on company documents for this purpose. It provides the retrieved text when each question is asked. The benefit is a readable combined answer; the model can still omit information or connect it incorrectly. Someone looking for a familiar form may not need this extra step.

Before the question: make documents searchable

The system reads approved documents and splits longer texts into searchable passages. Each passage needs a link to its title, version, original location and access rights. In our example, the reservation deadline must be retrievable together with the instruction to use the ticket form. If splitting the text breaks this connection, the answer may miss a condition.

The search index is a processed view of the documents. When version 3 of the IT handbook is replaced, that index must also be updated. Decide who handles changes, deleted documents and changed permissions, and how to check that old content no longer appears as a valid basis for answers.

At question time: apply access controls before supplying context

The application needs to identify the signed-in person and respect which sources they may access. For our employee, both general IT documents may be used as answer context. A pricing agreement restricted to purchasing must not reach the model as a text extract or appear as a search result or source reference.

This boundary needs technical enforcement. Telling the language model “Do not show confidential information” does not replace access control. The original document link also needs the appropriate permissions. Test the same question with two accounts that have different access rights and inspect which passages were actually passed to the model.

When answering: stay within what the sources support

The model drafts an answer from the permitted search results and connects statements to their source passages. In the example, the deadline and collection hours are supported. Neither document says whether a laptop is actually available on the requested date: that would require an authorised query of the current device inventory.

For the Sunday question, the source does not establish an exception. The system should state that gap. If two current documents give different collection hours, it should show the conflict and refer to the responsible team instead of choosing one time. Incorrect answers remain possible even with citations; the cited passage must actually support the statement.

Find errors before releasing the assistant

Ask the document owner or a subject-matter expert to record the expected source passage and a correct answer for recurring questions. Use the example for three separate checks:

• Wrong source: does retrieval return an old reservation deadline? Check versions, document preparation and result selection.
• Wrong wording: does the system find “five working days” but answer “five days”? Information was lost when drafting the answer.
• Missing evidence: does it invent a Sunday exception or device availability? These questions must remain visibly unanswered.

When the additional setup is worthwhile

Compare the same typical questions using your existing search and the assistant. Can employees find the correct answer more easily and check it against the original? The table below helps you choose an approach. An extract from a handbook cannot establish authoritative current figures.

The work includes document preparation, technical implementation, permission checks and subject-matter testing. Depending on the solution, running costs include hosting, model usage and maintenance. Also appoint a document owner and provide a simple way to report incorrect answers. Adding a chat interface does not resolve outdated or conflicting documents.