What started as a simple documentation assistant became a Slack-native helper powered by live community answers and governed APIs. Here’s how we built it, what we learned, and how you can replicate the approach.
Chat-based use cases are rapidly becoming the default way people access knowledge at work. Retrieval-augmented generation (RAG) is a practical way to ground answers in your organization’s content by retrieving relevant passages first, then letting an LLM compose the response from that evidence. In plain terms: search your docs, stuff the right excerpts into the prompt, and get contextualized answers back.
For Anaplan model builders and consultants, the “docs” aren’t just one site. They include official product documentation, API references, and the ever-active Community Forum. Our aim was a single assistant that understands all three—and meets people where they work (Slack).
We built the end-to-end solution in Dataiku because it couples no-code components with full-code flexibility, and because it ships production-grade ways to expose what you build as governed APIs. Dataiku Answers (a packaged, scalable web app) lets teams stand up RAG chat interfaces quickly, with source citations and governance built in.
And when it’s time to operationalize, Dataiku’s API Node & API Deployer make it straightforward to publish services (like our augmented LLM or an agent) as real-time endpoints behind a centralized Deployer.
Source data: Official Anaplan documentation and API/library references exported to a two-column dataset: url, text.
Embed & chunk: Use Dataiku’s “Embed dataset” recipe to chunk the text and generate vector embeddings (we associated the url as metadata so answers can cite their sources).
Result: A knowledge bank object in the Flow we can query from downstream components.
Dataiku’s RAG tutorials mirror this pattern and emphasize starting your Answers app from a knowledge bank created by the Embed recipes.
App creation: Open the knowledge bank → “Create your Dataiku chat application.”
LLM + storage: Select your LLM, then point Answers at two SQL datasets: one for conversation history and one for user profiles (required).
Retrieval behavior: Disable “Let Answers decide when to use the knowledge bank” so it always grounds in Anaplan content; enable display of source extracts for transparency.
Governance: Answers includes quotes/sources in-line and sits on Dataiku’s LLM Mesh for model choice, routing, and guardrails.
This gave us a fast, governed web UI to validate retrieval quality before wiring up Slack.
We packaged the retrieval-augmented model as a real-time API using Dataiku’s API Deployer so any channel (Slack, web, service desk) can call it securely.
We used an integration workflow to:
Listen for new messages in a help-desk channel.
Fetch thread history for context.
Call the Dataiku API with the user’s question (plus thread context).
Format & post the reply with citations and links back to sources.
This keeps knowledge in the flow of work, while Answers remains our governed “workbench” for evaluation.
Static documentation alone won’t cover edge cases, breaking changes, or the latest best practices. Our first attempt was to crawl tens of thousands of forum pages—accurate but hard to maintain. The better approach: an agent that searches the Anaplan Community live.
With AI agents in Dataiku, you can build both visual (no-code) and code agents; we used a Python-based code agent so we could:
Query our static knowledge bank;
Call the forum’s search;
Fetch top results;
Extract question/answer text;
Build a grounded prompt combining static + live content;
Return an answer with citations.
Dataiku’s agent capability is designed for this: central creation, multi-model choice, orchestration, and governance—so “live” tools don’t turn into an unmonitored script zoo.
Data layer: Documentation/URLs → Embed dataset → Knowledge bank
RAG service: Retrieval + prompt construction → Augmented LLM
API: Deployed via API Deployer for real-time use
Interfaces:
Dataiku Answers (testing, admin UI, governed logs)
Slack bot (thread-aware help desk)
Agent: Code agent that blends static knowledge with live Community Forum answers, then cites both.
Always-on citations: Show the snippets and URLs used to compose each answer to fight hallucinations and build trust. (Answers does this out of the box.)
Grounding policy: Force retrieval for Anaplan-tagged questions; only allow “general LLM” mode when explicitly requested.
Conversation logging: Store queries, responses, feedback, and metadata in SQL to analyze drift, common gaps, and cost.
APIs over ad-hoc scripts: Use the Deployer so every change is reviewed, versioned, and auditable.
Start with a clean corpus. A simple two-column url/text dataset—and putting url into embedding metadata—pays dividends when you need crisp citations.
Use Answers to iterate fast, then promote the same logic as an API for Slack and other channels.
Don’t hoard the forum—or the bills. Crawling everything is expensive and stale; a targeted agent keeps the bot current without monthly re-ingests.
Governance isn’t optional. Centralized deployment and conversation logs made it easy to answer “who asked what, and why did the bot say that?”
Aggregate your sources (official docs, API refs, FAQs).
Embed dataset → Knowledge bank in Dataiku; attach source URLs as metadata.
Create a Dataiku Answers app from the knowledge bank; wire an LLM and two SQL datasets (history + profiles); require knowledge-bank use; enable source extracts.
Package the augmented LLM as an API with Dataiku’s API Deployer.
Connect your channel of choice (Slack, ServiceNow, etc.) to that API.
Add an agent for live sources (e.g., vendor community) to blend static + fresh context.
Evaluation datasets for regression-style testing of prompts and retrieval settings.
Role-aware routing so the same bot answers differently for novice vs. expert users.
Broader agent toolset (ticketing, change logs) to move from “answers” to “actions.”