Insights — Artificial intelligence
A RAG is cheap to demonstrate. It is expensive to make reliable.
A vector store, a model, an interface: the demonstration takes three days. The cost is decided afterwards — in indexing your real documents, evaluating the answers and managing access rights. Here are the cost lines, and what genuinely moves the bill.
01 — The problem
The bill is decided by what the demonstration never shows.
Almost every document-assistant project we take over began with a successful proof of concept: fifty documents, a few simple questions, correct answers in two seconds. Six months later, three people open the tool, because the answers rely on outdated procedures and nobody can say whether they are right.
The real cost therefore does not sit in the language model, but in three lines you only discover in production: the quality of the index, access control, and the continuous measurement of accuracy.
01The five cost lines that really make up the price
Document indexing comes first. Properly converting scans, spreadsheets and procedures written by ten different people, cutting them up without breaking the meaning, versioning what changes: this is the least glamorous part of the project, and the most decisive. A mediocre index can never be rescued at answer time.
Evaluation comes next: a set of real questions with the expected answer, and a measurement of the rate of correct answers, the share of invented answers, and the correct refusals when the information does not exist.
- Indexing and document preparation — often half the budget
- Evaluation: real questions, measured accuracy, correct refusals
- Hosting of the model and of the vector store
- Guardrails: access rights, logging, human validation
- Operations: quality monitoring, re-indexing, supplier watch
02What divides the bill by two
The first lever costs nothing: reuse the infrastructure you already have. An existing PostgreSQL database becomes a vector store with the pgvector extension — no extra service to run, back up or secure.
The second is intelligent routing: a mid-sized model answers everyday questions very well, and the most expensive models are reserved for the genuinely difficult cases. That is exactly the role of the multi-model gateway we install by default: one application, several engines, a single calling contract.
- Reuse PostgreSQL rather than adding a dedicated database
- A mid-sized model for everyday questions
- A specialised model only for the hard cases
- Incremental index: re-index only what changed
- Logs kept to improve, never without a defined retention period
03The three questions to ask before signing
Who owns the index? If the answer is the provider, you are renting your own knowledge, and you will lose it the day you change supplier. Where do queries run, and where are the logs stored? The answer must be written in the contract, not promised verbally. And finally: how do you measure that an answer is right? A provider who does not talk about evaluation is selling a demonstration, not a tool.
02 — Common questions
What we are asked about budget
What budget should we plan for an internal document assistant?
On a scope of 200 to 2,000 documents and around thirty users, allow a few weeks of setup, then a monthly running cost in the hundreds when hosting is shared. The factor that changes the price threefold is not the model: it is the state of the documents to be indexed.
Do we need the most powerful model on the market?
No. For document work, a mid-sized model answers correctly as soon as the index is clean and the retrieved passages are relevant. The most expensive model never compensates for poor retrieval; it only makes the mistake more elegant.
Can we start small?
That is the right method. A narrow scope — one procedure, one standard contract, one technical reference — measured on real questions, is worth more than an assistant that answers everything approximately. You then widen it, one family of documents at a time.
Do you want a costing on your own documents?
RAG scoping: we examine your sources, build a set of real questions with your teams and hand over a costed plan, annual running cost included.