Why route emergency questions around the language model entirely?

Because the value of a generated answer is that it varies with the question, and that is exactly the wrong property here. Someone disclosing they are in danger needs the same verified contacts and next steps every time, regardless of phrasing, model version or retrieval luck. Matching recognized phrases before retrieval makes the most safety-sensitive path predictable and independent of model variability — it is the one route where determinism beats relevance.

Does a keyword branch not make the system brittle?

It is deliberately narrow, and narrowness is the point. The branch complements broader model behaviour rather than replacing it: a phrase it does not recognise still gets the grounded RAG path, whose prompt independently instructs the model to route violence or threat scenarios to emergency resources. The branch should be expanded and reviewed with domain specialists as real usage data arrives — which is a commitment, not a disclaimer.

What did the evaluation actually show?

A checked-in report over 24 prompts: 16 pass, 5 warning, 3 fail — a 67% pass rate. Within that set all three emergency prompts passed, all four out-of-scope prompts passed, and the evaluator recorded no hallucination. Weaknesses remained in broad service questions, retrieval of some known content, and one truncated legal response. A higher accuracy figure appears in a separate roadmap document and is not supported by the checked-in report, so it is not quoted.

The question you must not let a model answer

Nonprofit RAG — a self-hosted support assistant with a deterministic branch ahead of its grounded answering path

The failure mode

A nonprofit's public information footprint had grown across institutional pages, support services, legal-rights material, reports, publications, events and several related websites. Visitors could not find reliable answers, and the obvious remedy is a search box with a language model behind it.

That remedy is right for almost every question the organization receives, and wrong for the ones that matter most.

The organization also serves people who may arrive in an urgent situation. Someone typing that they are not safe is not asking a question in the sense the retrieval pipeline understands. They need a specific, verified set of contacts and next steps — the same ones, every time, regardless of how they phrased it, which model version is deployed, or what the reranker happened to surface that afternoon.

Why the obvious fix breaks

The instinct is to handle this in the prompt: instruct the model that if a user discloses danger, it should provide emergency resources. That instruction is worth having, and it is in the prompt. It is not sufficient on its own.

A generated answer's defining property is that it varies with its input. That is the feature everywhere else in the product and the defect here. Two people describing the same emergency in different words can get differently complete answers, and neither of them is in a position to notice that they got the weaker one.

Retrieval adds a second failure surface. If the emergency contacts live in a document that has to be retrieved and reranked above four other candidates before the model can quote it, then the correctness of the most important response in the system depends on a similarity score.

And a model that is right 95% of the time is a good assistant and an unacceptable emergency responder — because the 5% is not distributed evenly across questions that matter equally.

The frame that resolved it: decide which questions are allowed to have a probabilistic answer, and route the rest before the model is ever called.

The mechanism

The safety branch runs first, and it is not a model call. The chatflow checks the incoming query for recognized urgent-risk phrases before retrieval and before generation. A match returns a fixed response with emergency contacts and next-step guidance. Nothing about that answer depends on a similarity score or a sampling temperature.

Everything else is retrieved across four purpose-built datasets — organizational and contact information; support services and applications; legal and urgent-support content; and publications, events and international work. The implementation record documents 499 chunks across them. Every non-emergency question searches all four; results are reranked with Cohere rerank-v3.5, filtered by a score threshold, and limited to the top five passages before generation.

Generation is deliberately low-variance. Gemini 2.5 Flash answers at temperature zero under a prompt that instructs it to prioritise retrieved source material, avoid inventing unsupported details, give general information rather than definitive legal conclusions, route violence or threat scenarios to emergency resources, decline unrelated questions, and answer briefly in the target language. A ten-message conversation window carries follow-ups.

Note the redundancy: the prompt also routes threat scenarios to emergency resources. The deterministic branch is not a replacement for that instruction — it is the layer that does not depend on it holding.

The assistant's welcome state offering four starting questions, including one for urgent safety information — The four suggested openers do real work — they teach visitors what the assistant is for, and put the urgent-safety route one tap from the welcome screen rather than behind a phrasing the matcher has to recognise. An edited reconstruction: the organization is intentionally unnamed and the copy is replaced.

The edge cases that shaped it

Self-hosting was a requirement, not a preference. The assistant runs in the organization's own environment on a self-hosted Dify deployment rather than a fully managed chatbot platform. The browser calls a small Python proxy, which forwards approved request headers to the Dify API and streams the response back.

The widget is where the product is judged. Four suggested starting questions, Server-Sent Events streaming, lightweight Markdown rendering, follow-up quick actions, typing and error states, mobile-safe layout, and persistent conversation IDs within a session. An assistant that answers well but renders badly on a phone has not answered.

The personal-data warning is honest about being advisory. The widget detects patterns resembling national identifiers, phone numbers, IBANs and email addresses, and warns users not to share unnecessary personal data — alongside an in-context privacy notice explaining data categories and processing purposes. It warns; it does not block transmission. Calling that a control would be a misrepresentation.

The in-context privacy notice listing the data controller, the categories of data handled, the processing purposes, and a warning against entering unnecessary personal data — Controller, data categories, purposes, and the warning in red — shown in the conversation rather than linked from a footer. Also an edited reconstruction.

The evaluation set was built before the launch narrative. A 24-prompt evaluation across institutional information, services, legal guidance, emergency scenarios, out-of-scope handling and language quality produced 16 passes, 5 warnings and 3 failures — a 67% pass rate. All three emergency prompts passed, all four out-of-scope prompts passed, and the evaluator recorded no hallucination.

The failures became the backlog, not a footnote. Improve fallback behaviour for broad questions; tune retrieval thresholds and candidate counts; investigate empty API responses; verify token and timeout settings for long legal answers; add missing or under-retrieved content; run load testing and a real-user pilot; review logs, retention and data-handling controls.

Limits, and what I would do differently

67% is a real number and a small sample. Twenty-four prompts establish a baseline, not a performance guarantee. A higher accuracy figure appears in a separate roadmap document; it is not supported by the checked-in report, so it is not quoted here. Reporting the number I can defend rather than the number that reads better is the whole point of having an evaluation.

The deterministic branch is narrow by construction and needs specialists to widen it. Phrase matching catches the phrasings someone anticipated. Widening it well is not an engineering exercise — it needs domain review against real usage data, and until that happens the branch's coverage is an open question rather than a solved one.

Browser-side safeguards need server-side counterparts. The personal-data detection and client-side throttling live in the widget, which means they protect users who are using the widget as intended. Production hardening should add server-side controls, retention verification, restricted CORS and server-side rate limiting.

No production usage is claimed. This is a delivered architecture with a measured evaluation baseline, not a system with observed traffic behind it.

If some fraction of your users can arrive in a situation where a merely likely answer is not good enough, that fraction deserves its own path — decided before retrieval, independent of the model, and boring by design.