Databases were cleaned up, duplicates deleted, schemas validated and mandatory fields enforced in the source systems. The guiding principle was: once the raw data is clean and structured in the data pool, the quality is, by definition, correct. Anyone attempting today to operate production-ready RAG (Retrieval-Augmented Generation) systems according to precisely this old model often experiences a crash landing after the go-live. The reason? With generative AI, this static way of thinking falls short.
Even a perfectly curated, error-free corporate knowledge base often delivers incomplete or misleading results in practice. The problem here rarely lies with the data itself, but rather at the interface with humans. Imprecise user search queries, the retrieval system’s lack of understanding of deeper linguistic nuances, or the ‘noise’ within large sets of results all lead to the context becoming diluted on its way to the language model.
In modern AI architectures, data quality must no longer be viewed as a static state – it is a dynamic process that takes place directly during the query at runtime. To make the leap from a shaky AI prototype to a stable enterprise application, we must refine data streams in real time. The key levers of modern RAG systems are therefore situated immediately before and after the vector database: forward optimisation and backward optimisation.
1. Forward optimisation: building bridges before the search
Employees in their day-to-day business lives are not information retrieval experts. They use informal slang and abbreviations, type in vague questions, or simply use terms that do not appear in the documents in exactly that form. If these initial prompts are sent directly to the database as a mathematical vector, the hit rate remains low, as the embedding of an imprecise query often fails to fully capture relevant semantic aspects.
Forward optimisation therefore specifically places specialised query rewriters, smaller LLMs, distilled models or rule-based methods in front of the actual search to semantically enrich the query.
Query Expansion & Paraphrasing
For example, if a user
enters the question “How does the termination process work at our company?”, the system simultaneously transforms this into several precise search queries that examine the topic from different angles (e.g. “What notice periods apply to the termination of an employment contract?” or “Termination agreement and remaining holiday entitlement according to HR policy”). By running these queries in parallel, the system can significantly increase recall (the hit rate). This is supported by upstream metadata filtering to narrow the search directly to relevant departments or time periods.
HyDE (Hypothetical Document Embeddings)
The HyDE method is
a particularly elegant approach. Instead of searching using the user’s imprecise query, the pipeline generates a purely hypothetical response in advance. The mathematical rationale is that the hypothetical document often has a similar semantic structure to the actual target documents. As a result, the generated embedding is significantly closer to the relevant documents in the database than the embedding of the original query. The artificially generated response serves as a highly efficient search magnet and is simply discarded after retrieval.
2. Backward optimisation: filtering out noise after the search
Modern enterprise systems usually rely on hybrid retrieval (a combination of classic keyword search via BM25 and dense retrieval) as a first step, or use multi-vector approaches such as ColBERT. These initial retrievers filter extremely quickly – often within milliseconds when dealing with millions of documents. However, this speed comes at the expense of fine-grained relevance.
The result: whilst the database returns the top 25 text fragments (chunks), the top results frequently include partial semantic overlaps or superficial similarities that do not actually answer the question. This is where backward optimisation comes into play.
The cross-encoder as the ultimate quality filter
A downstream cross-encoder model (such as a BGE reranker) takes the original user query and analyses it alongside each of the 25 retrieved text fragments in a deep computational step. It evaluates the logical fit sentence by sentence.
This gives rise to the golden rule of modern RAG architectures: cast a wide net (top 25–50 via query expansion for maximum recall), but narrow down the results extremely precisely (top 3–5 via re-ranking for maximum precision). The re-ranker radically reorders the documents, bringing the strongest matches in terms of content right to the top. A subsequent context compression step further condenses the chunks, filters out redundant phrases and passes only the absolute essence to the final language model.
The aim: to overcome the ‘Lost in the Middle’ phenomenon
Why do we go to such immense architectural lengths after retrieval? Because large language models have an empirically proven weakness: the ‘Lost in the Middle’ phenomenon. When we feed an LLM a huge context prompt with 20 or 30 chunks, it places extremely heavy weighting on the information at the beginning and end. Important details buried in the middle are simply overlooked. By filtering and compressing the data via backward optimisation, we achieve three things:
- We massively reduce token costs.
- We drastically minimise the application’s latency.
- We significantly reduce the risk of hallucinated responses, as the model is not distracted by noise.
Making data quality measurable rather than flying blind
To ensure this dynamic cycle is sustained in the long term, continuous RAG evaluation within the development pipeline is essential. Traditional information retrieval metrics such as Recall@k, nDCG (Normalised Discounted Cumulative Gain) and MRR (Mean Reciprocal Rank) help us to monitor the quality of the search algorithms with mathematical precision. Coupled with automated ‘LLM-as-a-Judge’ frameworks that continuously check ‘faithfulness’ (factual accuracy in relation to the context), we ensure that the system remains stable even as new documents are added daily.
Conclusion
Companies that view data quality in the age of generative AI in isolation – as a downstream cleansing of source systems – are squandering the true potential of their technology. Only by implementing intelligent forward optimisation during the query and precise backward optimisation after retrieval can inaccurate human inputs be transformed into reliable, business-critical answers. For future-proof enterprise applications, Dynamic Data Quality is no longer an optional feature – it forms the qualitative foundation for operational success.
Download white paper "Whitepaper_Dynamic_Data_Quality_RAG_V2"