A multi-agent pipeline that watches regulatory sources, scores relevance per customer, and drafts a grounded impact assessment, with mandatory human approval before any of it counts.
Healthcare compliance teams manually watch a shifting body of federal and state regulation. Today that means reading the source, noticing a change (or not), judging whether it applies to their facility, and writing an impact memo from scratch, every time. Miss one and the consequence is a failed survey or a repayment demand, not a missed Slack message.
Diffing a document is trivial. The expensive human work is deciding whether a change actually applies to a specific facility, and then translating dense regulatory text into a plain-language, correctly cited recommendation. Relevance has to be scored against an explicit customer profile (programs, facility types, jurisdictions), not judged globally, or the signal drowns in noise.
Instead of one large prompt trying to do everything, the pipeline splits into five single-purpose agents: ingestion detects and diffs changes, relevance scores each one against a customer profile (acting as a cost gate before the expensive step), detection classifies severity, synthesis drafts a cited impact assessment grounded only in retrieved source text, and a conversational agent answers ad-hoc questions the same way.
This is a solo proof of concept, not a production deployment, and I'm explicit about that rather than overstating it. It runs fully end to end, including an offline mock-AI mode so the whole flow is demoable with zero cloud credentials. The design target is a sub-15% edit rate: the share of AI-drafted assessments a human has to meaningfully rewrite before approving, the single number that proves whether the AI is actually trustworthy or just theater.
In high-stakes AI, guardrails are the product. Human-in-the-loop isn't a fallback bolted on afterward; it's the reason the tool is adoptable at all in a domain where being confidently wrong is worse than being slow.
A single trust metric beats a dozen vanity ones. Edit rate is a direct proxy for "is this actually good enough to use," and designing the whole pipeline around moving that one number kept the scope honest.