The everyday task
An employee needs to prepare a supplier approval. They know their question, but not the title of the relevant policy. The information sits in different sections; after finding it, they still have to bring it together.
The aim is an answer whose individual claims can be checked against the original. The fictional example below shows the difference between a document search result and a draft answer.
- Knowledge is spread across several locations.
- Versions and ownership are not always clear.
- Not every colleague may access every document.
From a document excerpt to an answer you can check
All document names, rules and questions in this example are fictional. They do not describe an actual company policy or client implementation.
- Document excerpt
- “Supplier Approval Guide”, version 2, section 3 “Documents”: A completed supplier information form and a signed code of conduct must be submitted for a new supplier approval. Section 4 “Review”: Procurement checks that the documents are complete.
- Employee question
- “Which documents do I need for a new supplier, and who checks that everything has been submitted?”
- Draft answer with section references
- “You need a completed supplier information form and a signed code of conduct [Supplier Approval Guide, version 2, section 3]. Procurement checks completeness [same source, section 4].” The employee opens both sections in the original and checks the statements.
- A deliberately unanswerable question
- “Can I submit the code of conduct later for an urgent order?” — “The available sections contain no rule about urgent orders or later submission. I cannot infer an exception from them. Please check with the person responsible for the policy.”
Permissions, retrieval and drafting have different jobs
In this RAG workflow, relevant document passages are retrieved and then used to draft an answer. Access checks first establish which content the person is allowed to use.
Restrict access before retrieval
The system limits retrievable content to the signed-in person’s permissions. An instruction to the language model does not replace this check.
Find relevant passages
In this example, search returns sections 3 and 4 with the title and version. Whether those results cover the question is assessed separately from the wording of the answer.
Draft an answer from the sources
The language model summarises the supported information and assigns a source passage to each statement. It leaves the undocumented exception unresolved.
Check statements against the original
The person opens the source and checks its content and validity. The draft grants neither supplier approval nor permission for an exception.
When search is enough and when an assistant fits
Someone who only wants to open the current supplier information form mainly needs an unambiguous document result. A maintained collection, clear file names and existing search may be enough. Even the short question in this example can be answered by reading the policy directly.
A knowledge assistant may fit when employees repeatedly ask questions in their own words and need to bring information together from several sections. Its added contribution is a draft answer with references for the individual claims. Use these questions to check whether assembling an answer actually takes less work once checking the original is included.
Prepare document quality and permissions
Check extraction of text, tables and attachments. A scanned form may need different preparation from clean text. Decide which metadata must survive so a source can be located again.
Keep the title, version, section and person responsible for the content. Start with a limited, maintained collection. Historical versions must remain identifiable. Test permissions using different roles: content a person cannot access must be unavailable both as a search result and as material for drafting an answer.
Test unanswered questions and outdated sources
Use the supported question and the unanswerable exception question in the example as two separate checks. Add an old policy version and a test user without access. The assistant should state that evidence is missing without inventing a prohibition or permission.
Record separately: did search return the correct sections? Does the draft represent them accurately? Is unauthorised content excluded? Repeat these checks after document or permission changes. A citation alone does not confirm that an answer is correct.
How to tell whether it helps
Compare the same specialist questions using your existing search and the assistant. Record the time to a checked answer, including corrections. Also consider setup, document maintenance and running costs.
- Does retrieval find the relevant, valid sections?
- Is every claim in the draft supported by its cited passage, and does the exception question remain unresolved?
- Is unauthorised content excluded for every role tested?
- Does the draft reduce the overall effort compared with reading the documents directly?