Insights — Artificial intelligence
A useful RAG is not judged on its demo, but on its third week.
Most document assistant projects fail for three reasons that have nothing to do with the model: chunking that does not match the documents, answers without citations, and evaluation that does not exist. Here is the chain we deploy, and why it lasts.
01 — The problem
The model does not forget your data: it never read it.
A language model knows nothing about your procedures, your contracts or the history of your interventions. It produces a plausible answer from whatever it is handed at the moment of the question. The whole point of RAG is to hand it the right passages, at the right time, with enough context that it does not have to guess.
When an assistant gets things wrong, the cause is almost always upstream: a badly chunked document, an index mixing several versions, a search that cannot find the right passage because it uses different words.
01Chunk by document structure, not by character count
The reflex is to split every document at a thousand characters. That is exactly what produces wrong answers: a procedure cut in two means nothing, and a table separated from its header means less.
We chunk by structure: section, clause, step, table row keeping its header. Every chunk carries its provenance — document, version, date, page — because that provenance is what allows the answer to cite its source.
02Search with both vocabularies
Your teams do not use the words of your documents. Semantic search bridges formulations; lexical search finds exact references — part numbers, codes, proper names. Using both and merging the results does more for assistant quality than any change of model.
We add a reranking step: the fifteen best passages are reassessed so only the five genuinely answering the question survive. That selection is what turns an impressive prototype into a used tool.
03Cite, or stay silent
An answer without a source is unusable in a company: nobody will change a contract or a setting on the basis of an unsourced sentence. We therefore enforce an answer format that always carries the document, its version and the excerpt used.
And when the corpus does not contain the answer, the assistant must say so. An assistant replying “I cannot find this in your documents” is infinitely more useful than one improvising with confidence.
04Measure, then optimise the cost
An evaluation set of one to two hundred real questions, with the expected answer and expected document, is enough to steer quality. We replay it on every change of model, chunking or retrieval strategy: without it, any claimed improvement is an impression.
Cost is then controlled through the gateway: automatic selection of the cheapest model good enough, caching of frequent questions, and a spending cap per assistant and per user. An AI bill must fit on one predictable budget line.
- Model chosen according to the difficulty of the question
- Caching of answers to recurring questions
- Switch to a local or European model when sensitivity requires it
- Caps and logging per assistant and per user
The full chain: sources, index, hybrid search, reranking, generation with citation, evaluation and a cost gateway.
02 — Common questions
What management asks before we start
Can our documents be used to train a model?
No, and that is contractual: your data feeds a search index that belongs to you and is never used to train a model. No provider receives a training set built from your documents.
Should we start with all our documents?
No — that is the best way to fail. We start with a closed scope, such as one family of procedures or one type of contract, where quality is verifiable, then we extend. A large, badly prepared corpus produces a confident, wrong assistant.
What does one question cost, roughly?
A few cents for a short question on a small corpus, with a cap that falls as caching and routing take effect. The line to watch is not generation but repeated retrieval: that is exactly what caching and reranking bring down.
Would you like an assistant that cites its sources?
We scope the perimeter, prepare the corpus, deploy the index and measure quality against your real questions — before any go-live.