The classic answer is: "We have tested randomly, it looks good."
The pain point: "Looks good to me" does not scale.
Manual testing may still work in the POC phase. But in operation with thousands of requests per week? Nobody has the time to manually read 5,000 chat histories and check them for facts. Anyone who develops RAG (Retrieval Augmented Generation) based solely on gut feeling is playing vabanque with data quality.
The deep dive: Automated evaluation (LLM-as-a-Judge)
We do not rely on chance. We use frameworks such as RAGAS or Arize Phoenix to evaluate our pipelines automatically. The principle: a strong LLM (the "judge") evaluates the output of the smaller production LLM.
We look at two metrics in particular:
- Faithfulness
Here, the evaluator checks: "Does every piece of information in the answer really come from the retrieved documents?"
If the score is low, the bot hallucinates facts that were not in context. An absolute no-go in the enterprise environment. - Answer Relevance
Here the evaluator checks: "Does the answer actually answer the user's question?"
A bot can say factually correct things but completely miss the point. This metric exposes this.
The result: measurable quality
Instead of saying "The bot works better today", we say: "After updating the chunking algorithm, our faithfulness score has risen from 0.78 to 0.92." This is how AI tinkering becomes reliable software engineering. Quality must be measurable - otherwise it's just a coincidence. How do you safeguard your RAG systems? Team "manual sampling" or team "auto-eval"?