Nursing and Law pilots
Two real-user persona pilots on the HAI orchestrator: an evidence-based-practice tutor for nursing students and a U.S. law research assistant, with retrieval, clinical safety checks, and defect fixes driven by actual use.
RAG persona-based AI evidence tooling hallucination mitigation retrieval architecture privacy by design user-driven QA
The short version
I ran two persona pilots on the lab's orchestrator with real users who had actual study and research needs. Two nursing students used an evidence-based-practice tutor for their coursework. A third user worked a law-research persona against state statutes and court opinions.
In my lab these are live systems in real use, and I am always changing them: people depend on them for graded coursework, where a fabricated citation costs marks, so they have to hold up, not just demo well. What they prove is a capability that transfers. The method that grounds an assistant in a nursing curriculum or a state's statutes is the same one that would ground it in a company's own knowledge, which is why I treat these as pilots for exactly that kind of work. There are five provisioned assistant profiles, three of them the pilot users and two of them friends on the general persona; the rest of my friend group uses the media services and never touches the AI.
The pilots surfaced four defects that months of my own testing had not. The most instructive was a privacy leak: an admin preview path had been pointed at one student's private corpus. All four were fixed. The nursing pipeline now includes a PubMed evidence tool, hybrid BM25-plus-embeddings retrieval, self-serve document upload, and an optional independent clinical critique from a second 27B medical model.
What a nursing user gets
A nursing student picks a mode, and the choice replaces the general-purpose model selection. Four modes are available:
| Mode | Model | Purpose |
|---|---|---|
| Reference | Quick 8-9B tier | Brief, cited lookup. |
| Tutor | Automatic cascade | Teach and quiz. Grounds every fact in retrieved context. |
| EBP Research (default) | 27B on the 24 GB card | Synthesize, critically appraise, format in APA 7th, step by step. |
| Educator | 27B | Writes NCLEX-style test items in the student's own blueprint format. Each item carries a Bloom's level tag, an NCLEX category, a marked answer with rationale, and a real citation from the evidence tool. |
EBP Research is the workhorse. The Educator mode was added as a fourth mode, built from a format specification the student provided: their course blueprint tables, item structure, and marking conventions.
Building these modes meant learning enough of the domain to encode it. Some of it I already had: I have a medical background from time as an EMT and a medic and two years of nursing school, and I knew APA. The evidence-based-practice structure, PICOT, the levels of evidence and how they are appraised, and how the NCLEX blueprint categories map to item types I learned from the student, who is the clinician on this project. Those became the persona's shared reference corpus and the prompt rules each mode runs under, and the clinical critique's review criteria were specified the same way.
The evidence tool
Citation fabrication is the first failure mode in academic work. I built the evidence pipeline so references come from a live lookup instead of the model's memory: a three-step PubMed E-utilities chain runs search, summary, then fetch. Results are filtered to PubMed-indexed, free-full-text papers within five years of the query date. Each hit returns the real abstract, a copy-ready APA 7th reference, and the PMID or PMC link.
If nothing matches, a relaxation ladder widens the net: drop the free-full-text filter, then widen the year window, then run unrestricted with an "older than your window" warning. Rate limits trigger automatic backoff. The model is instructed to cite only what the tool returned. If a paper is paywalled, the student uploads the PDF and it enters their personal corpus.
Self-serve upload
A user drops a PDF, DOCX, text, or markdown file into the interface and tags its intent: "a source to cite," "my own draft," or "an assignment rubric." The server extracts the text and promotes section labels to markdown headers. It prepends a title and an intent-based provenance note so the model treats the content correctly, then writes the file into the user's corpus and reloads their namespace. Files above 20 MB are rejected. A scanned PDF with no text layer fails with a clear "needs OCR" error instead of silently indexing nothing.
What a law user gets
The law persona is U.S. law with a state focus. Its primary method is live web search of authoritative government and court sources: the state legislature's statute site, the federal code sites, and official court opinions. It cites what it finds and is framed as information, not advice. Its shared reference corpus covers the state's criminal-procedure statutes and the 4th, 5th, and 6th Amendments, along with the landmark Supreme Court opinions in that area and a guide to where the law can be found online. The user keeps a private corpus on top of that, the same two-layer design the nursing persona uses.
How it stays honest
Three mechanisms keep outputs grounded in what exists.
Where citations come from
The model is never asked to produce a reference from memory. Every citation it offers carries the PMID or PMC link the tool returned, so a student can click through and see the paper. The prompt rule is strict: cite only what the tool returned. When a user uploads their own document, the server's provenance tag tells the model whether to treat it as evidence or as a draft.
Medical critique
In deep-think mode a nursing user can add an independent clinical safety check. A separate 27B medical model reviews the work for drug/dose/route/interaction accuracy, lab-range correctness, scope-of-practice violations, over-certainty language, and fabricated citations, and its findings merge into the refinement pass.
The cost is real. The 24 GB card cannot hold both 27B models at once, so each pass requires one or two cold model swaps, and nursing deep-think gets a larger budget of four passes and 900 seconds to cover them. If the medical model is absent or returns bad JSON, the normal critique and the first pass still stand.
The privacy guarantee
Persona and identity are separate concepts in this system. Persona controls behavior and domain framing, and identity is the user's own namespace, corpus, and memory. Any permitted user can wear any persona and stay themselves, because the orchestrator resolves the two separately. The identity namespace is the privacy anchor, and the persona comes from the request.
Grounding searches the union of a user's private corpus and their persona-shared reference corpus. A nursing student's private drafts are never visible to another nursing student, even though they share the same reference materials.
What real use found
Four defects came out of real coursework use, and each one traces to a specific thing a user did.
| # | Defect | Trigger | Fix |
|---|---|---|---|
| 1 | Degenerate output on the 8B tier | A large retrieval plus system prompt and tool schemas filled the model's 8,192-token window, leaving roughly one token to generate. | Hard caps on injected context: 1,600 characters per chunk, 6,000 total. |
| 2 | Privacy leak in admin preview | The "preview a nursing mode" path pointed at one student's private corpus, so an admin preview could see that student's drafts. | Repointed to the persona-shared reference corpus. Previews never touch private data. |
| 3 | Stock refusals about file access | The model said "I cannot access your files" and "I do not keep anything between sessions" to users who had a persistent, loaded library. | A get-whole-document tool plus a prompt rule: the assistant must fetch what is already in the user's library and never ask them to paste it. |
| 4 | No chat switching | A user wanted to pause one conversation and start another without losing either. | Became the Conversations Suite (chats as first-class objects): per-chat history, auto-generated titles from a background 8B call, pin a document like a rubric to one chat for priority retrieval there only, export to markdown. |
The last item shows how user-driven design works here: a single request became its own project with its own deliverables. The other three were fixes to the nursing pipeline that both students benefit from.
The nursing pilots are functional but not formally evaluated. The one outcome signal is anecdotal: a student reported A grades on the coursework the tutor supported, which is one person's result, not a measurement. With two users there is no sample set and no formal measure of learning outcomes. The users came in through the friend onboarding process, and I support every account by hand, which is workable at this size.
Retrieval architecture details
The retrieval stack uses hybrid BM25-plus-embeddings ranking fused by Reciprocal Rank Fusion. That catches paraphrase hits that keyword search alone would miss while keeping exact-token matches ranked properly.
Embedding pipeline
A local embedding model served by Ollama computes vectors per chunk at load time. Results are cached on disk keyed by the hash of the chunk text, so only new chunks get re-embedded. Each chunk is prefixed with its document title so mid-document chunks stay attributable to their source.
Thresholds and fallback
A chunk enters results if its BM25 score reaches at least 1.5 or its cosine similarity reaches at least 0.55. Both thresholds are absolute values, not relative rankings, so an off-topic query returns nothing instead of grounding in irrelevant context. If no embedding host responds within three seconds, retrieval falls back to BM25-only.
Why no vector database
The corpora are small. Keeping vectors in memory avoids operational complexity without meaningful benefit at five user profiles and modest reference libraries.