RAG access control is the part of retrieval systems that nobody demos, and document permission leaks are what appear when it is missing. The demo indexes a folder of documents, asks a question, and gets a good answer. The thing nobody checks is whether the intern who asked is allowed to see the…
Evaluate a RAG system in two separate parts : retrieval (did the right chunks come back?) and generation (is the answer supported by those chunks?). Measure retrieval with recall\@k and MRR against a small hand-labelled golden set of questions; measure generation with a groundedness check that…
Introduction Building an AI application that can retrieve company information and generate useful responses is exciting. But getting the model to produce a relevant answer is not the only engineering challenge. There is another question that matters just as much: How do we control what users send…
Most RAG systems get tested the same way: someone asks a few questions, the answers look fine, and the system ships. Then a document changes, a chunking setting gets tweaked, and nobody notices that retrieval got worse until a user complains. You can catch most of that in CI with two checks: does…
Most teams that ask me about the fine-tuning vs RAG decision matrix have quietly already chosen fine-tuning. The reasoning is always the same: we have company knowledge, the model does not have it, so we should teach the model. That instinct is wrong for the vast majority of small business…
Da ideia à arquitetura de uma aplicação que responde com base em documentos Modelos de linguagem são excelentes para produzir respostas convincentes. O problema é que uma resposta convincente não é necessariamente verdadeira. Foi dessa preocupação que surgiu o Research Quest , uma aplicação para…
I've been working on Cairnly, an open-source AI engineering learning platform that also serves as a reference implementation for building RAG applications. GitHub: https://github.com/CherukuriSiva/Cairnly Live demo: https://cairnly-blond.vercel.app Why I built it There are many tutorials explaining…
A PDF table can survive extraction with every number intact and still answer a question incorrectly. Consider two rows with the same measurement. Only the first has a superscript a , and the note says that its value was measured under a reduced load. If a pipeline attaches that note to every row…