A self-hosted retrieval-augmented generation assistant that turns a fragmented nonprofit content estate into one conversational entry point — with a deterministic path for the questions where a probabilistic answer is not acceptable.
Overview
A nonprofit with a large public information footprint needed a simpler way for visitors to find reliable answers. Its content spanned institutional pages, support services, legal-rights material, reports, publications, events, and several related websites. The organization also served people who could arrive with urgent safety concerns, so a conventional "search box with an LLM" was not sufficient.
I designed and implemented a self-hosted RAG assistant that combines deterministic emergency routing, multi-dataset retrieval, reranking, grounded generation, streaming responses, conversation memory, and a custom web widget.
The client is intentionally unnamed, and the interface visuals are edited reconstructions.
The challenge
The assistant had to balance five needs:
- Urgent safety routing: Recognized high-risk phrases needed an immediate, consistent response rather than a probabilistic generated answer.
- Grounded guidance: Legal and support-service responses needed to stay within approved source material and avoid definitive legal judgments.
- Broad content discovery: Users needed one entry point for organizational information, services, programs, publications, and events.
- Operational control: The solution needed to run in the organization's own environment rather than depend on a fully managed chatbot platform.
- Accessible delivery: The experience needed to work as a responsive website widget with streamed answers, suggested questions, and conversation continuity.
My role
I worked across the full implementation: analyzed source documents and website content; structured the knowledge base into retrieval-oriented categories; designed the chatflow and prompts; configured embedding, retrieval, reranking, and generation; implemented the streaming web widget and API proxy; added privacy messaging, personal-data warnings, and client-side throttling; created and reviewed a 24-prompt evaluation set; and documented deployment and operational follow-up.
Solution
A deterministic safety path before the LLM
The chatflow checks the incoming query for recognized urgent-risk phrases before retrieval or generation. A match returns a fixed response with emergency contacts and next-step guidance. This makes the most safety-sensitive path predictable and independent of model variability.
This rule-based branch is deliberately narrow. It complements, rather than replaces, broader model behavior and should be expanded and reviewed with domain specialists as real usage data becomes available.
Retrieval across four purpose-built datasets
The knowledge layer was organized into four datasets covering organizational and contact information; support services and applications; legal and urgent-support content; and publications, events, and international work. The implementation record documents 499 chunks across these datasets. Every non-emergency question searches all four, and results are reranked with Cohere rerank-v3.5, filtered by a score threshold, and limited to the top five passages before generation.
Grounded, low-variance generation
Gemini 2.5 Flash generates the final response with temperature set to zero. The prompt instructs the model to prioritize retrieved source material; avoid inventing unsupported details; provide general information rather than definitive legal conclusions; route violence or threat scenarios to emergency resources; decline unrelated questions; and answer briefly and clearly in the target language. The flow retains a ten-message conversation window for follow-up questions.
A custom streaming widget
The responsive widget provides four suggested starting questions, streamed Server-Sent Events responses, lightweight Markdown rendering, follow-up quick actions, typing and error states, mobile-safe layout behavior, persistent conversation IDs within a session, and an in-context privacy notice. The browser calls a small Python proxy, which forwards approved request headers to the self-hosted Dify API and streams the response back to the client.
Privacy-aware interaction design
The widget includes a privacy notice explaining the data categories and processing purposes. It also detects patterns resembling national identifiers, phone numbers, IBANs, and email addresses and warns users not to share unnecessary personal data.
The warning is advisory, not a transmission block. Production hardening should therefore add server-side controls, retention verification, restricted CORS, and server-side rate limiting rather than relying on browser-only safeguards.
Buyer-relevant use cases
| Use case | User need | System behavior |
|---|---|---|
| Emergency routing | "I am not safe" or a recognized equivalent | Bypasses generation and returns fixed urgent guidance |
| General legal-rights guidance | Understand available protections or next steps | Retrieves approved legal content and avoids definitive legal advice |
| Support-service discovery | Find counseling, social support, or referral options | Searches service content and provides relevant contact routes |
| Institutional self-service | Find hours, contact details, programs, and policies | Answers from organizational datasets |
| Research and event navigation | Discover reports, publications, and events | Retrieves relevant material across the wider content collection |
Evaluation
The checked-in evaluation report covers 24 prompts across institutional information, services, legal guidance, emergency scenarios, out-of-scope handling, and language quality.
| Result | Count |
|---|---|
| Pass | 16 |
| Warning | 5 |
| Fail | 3 |
| Pass rate | 67% |
Within this test set all three emergency prompts passed, all four out-of-scope prompts passed, and no hallucination was recorded by the evaluator. Weaknesses remained in broad service questions, retrieval of some known content, and one truncated legal response.
These are evaluation results, not a guarantee of production performance. Only the verified 67% figure is presented here; a higher accuracy number that appears in a separate roadmap document is not supported by the checked-in report.
What the evaluation changed
The findings produced a practical optimization backlog: improve fallback behavior for broad questions; tune retrieval thresholds and candidate counts; investigate empty API responses; verify token and timeout settings for long legal answers; add missing or under-retrieved content; conduct load testing and real-user pilot evaluation; and review logs, retention, and data-handling controls.
Outcome
The result is a working, self-hosted assistant architecture that turns a fragmented nonprofit content estate into a single conversational entry point. Its most safety-sensitive route is deterministic, ordinary answers are grounded in a curated knowledge layer, and the custom widget provides a usable path from a public website to the RAG workflow.
Just as importantly, the evaluation made the remaining quality gaps visible. Rather than presenting a polished demo as proof of perfect accuracy, the project established a measurable baseline and a clear path for iteration.
Need a similar product or workflow? If you have a large public content estate and questions that cannot all be answered probabilistically, this is the shape of system that handles both.
