Kala Atelier

From “seems good” to “measurably good” to “live” For two episodes I’ve been repeating the same nag: measure it, don’t assume it. Now we build the thing that measures — and then we put Nimbus HelpDesk AI online. Here’s why evaluation matters more in RAG than in normal software. In a normal app, a function…

“Works on my questions” isn’t “works” Episode 6 gave us a real, grounded, cited bot. Now let’s try to break it — because your customers certainly will. Break #1 — the exact term. A customer types: “I’m getting error ERR_402 at checkout.” Semantic search struggles here. Embeddings are brilliant at meaning, but “ERR_402” has no…

The last mile Five episodes of groundwork come together here. We can load documents, chunk them, embed them, store them, and — as of last episode — retrieve() the most relevant, quality-checked passages for any question (or honestly return nothing). But retrieval just hands us raw chunks of text. A customer doesn’t want three passages…

Our search is fast, but it isn’t smart After Episode 4, MySQL ranks our chunks quickly and returns the top 3. Let’s test it with a question Nimbus’s knowledge base can’t possibly answer: php episode-04/search.php "What’s the weather in Mumbai today?" #1 score=0.081 [refund-policy.md] Customers may request a full refund within 14 days of purchase……

The problem hiding in our search Episode 3 ended on a high: ask “how do I get my money back?” and the refund policy comes back as the top match. Retrieval works. But look closely at what search.php actually does: $rows = db()->query(‘SELECT source, content, embedding FROM chunks’)->fetchAll(); That line says: “give me every single…

The gap we have to close At the end of Episode 2 we had chunks.json: our documents sliced into clean, tagged passages. But those chunks are still just text, and text has a frustrating limitation for search. A customer might ask: “How do I get my money back?” The relevant chunk says: “Customers may request…

Where we left off In Episode 1 we cheated. We pasted Nimbus’s refund policy into the prompt by hand and watched the model answer from it. That proved the core idea of RAG — but it doesn’t scale. Nimbus has dozens of help articles, a refund policy, an FAQ, and a features guide. You can’t…