An AI assistant in a consumer app can hallucinate occasionally and the worst outcome is an awkward laugh. The same hallucination in a credit-decision system or a clinical-triage tool can bring regulatory action and possibly a lawsuit. Building AI for regulated industries means designing not for what the model can do, but for what it is not allowed to do — and being able to prove it under audit.
Start with a written policy, not a filter
Before choosing tools, write down what the system must never do, in language a compliance officer would sign: never state a credit decision, never give dosage advice, never reveal another customer's data, never promise a claim outcome. Each rule then maps to one or more of the layers below, and each layer gets tests. Guardrails without a written policy tend to become a pile of rules nobody can explain at audit. The NIST AI Risk Management Framework is a useful shared vocabulary for writing that policy with risk and compliance teams.
Five layers worth running in parallel
- Input filtering. Detect injection attempts, redact PII before it reaches the model where the use case allows, and validate input shape and length. Treat retrieved documents and tool outputs as untrusted input too; indirect prompt injection arrives through them.
- Constrained generation. Schema-validated outputs (Pydantic, Outlines, Instructor, or a provider's structured-output mode) so the model cannot return ill-formed data.
- Refusal enforcement. Explicit instructions plus a classifier that detects out-of-policy requests and rejects them before generation. NVIDIA's open-source NeMo Guardrails is one way to express these rails as configuration rather than scattered code.
- Output validation. Cross-field consistency checks, business-rule validation, sanity checks on totals, and checks that factual claims are grounded in the retrieved sources.
- Human-in-the-loop checkpoints. For high-stakes outputs, a human approves before the system acts. Credit, claims and clinical decisions generally need this, and auditors will ask exactly how it works.
A worked example: a loan-servicing assistant
Say a lender deploys an assistant that answers borrowers' questions about their repayment schedule. A borrower writes: "Can you waive my late fee and tell my brother my outstanding balance?" Here is how the layers handle it:
- The input layer confirms identity from the authenticated app session and classifies the message as containing two separate intents.
- The policy classifier marks the fee waiver as out of scope (only staff can grant waivers) and disclosure to a third party as prohibited.
- The model generates a reply constrained to a schema: answer text, the policy sections it relied on, and an escalation flag.
- Output validation checks that no balance figure appears in the reply and that the escalation flag is set for the waiver request.
- The waiver request is routed to a human with the conversation attached, and every step is written to the audit log.
None of these layers is sophisticated on its own. Together, they mean the failure you fear requires several independent things to go wrong at once.
What the model itself can and can't do
System prompts and constitutional AI techniques help, but no LLM is a reliable security boundary. Treat model self-restraint as one layer among many. If it is your only layer, an injection attempt will eventually get through.
Logging and explainability
Every decision an AI system contributes to needs an auditable trail: what was retrieved, what the model said, what was filtered, and what the final action was. RBI and HIPAA-style audits do not accept a black box. Log the full prompt, retrieval results, completion and post-processing for every regulated request, encrypted, access-controlled and retained for the period your regulator requires.
Version everything as well: prompts, model identifiers, retrieval indexes and guardrail rules. When a complaint arrives months later, you need to reproduce what the system would have done on that date, not what it does today.
The model providers' role
OpenAI, Anthropic and Google all publish their own moderation and safety tooling. Use it; it catches things your custom rules will miss. But layer your own controls on top, because a provider's safety policy is not your compliance regime.
Testing the guardrails
- Maintain a red-team set: injection attempts, jailbreak phrasings, requests for other customers' data and out-of-policy advice requests, in every language you support.
- Run it on every change to prompts, models or rules, and track both the block rate and the false-positive rate. Over-blocking has a cost too: a guardrail that refuses legitimate questions pushes users to the phone queue.
- Review a sample of production conversations every week with someone from compliance, and turn each new failure into a test case.
Common mistakes
- Relying on the system prompt as the only control.
- Redacting PII in the prompt but writing it in plain text to application logs.
- Adding guardrails after launch, with no tests behind them.
- Using one global threshold for every flow instead of stricter settings where the stakes are higher.
FAQ
Do guardrails add noticeable latency?
Some do. Run cheap checks (rules, small classifiers) inline and heavier checks in parallel with generation where possible, and measure the added latency per layer.
Can we buy guardrails instead of building them?
Partly. Moderation APIs and open-source frameworks cover common risks, but your policy, business rules and audit trail are specific to you and have to be designed in.
How we engineer this at Velura Labs
We treat guardrails as an explicit phase of regulated work, not an afterthought. Our Custom LLM Applications and Agentic Systems services include guardrail design, audit-trail engineering and compliance handover documentation. For the evaluation discipline that complements guardrails, see our eval playbook. Talk to us if your AI roadmap is moving into a regulated industry, and we'll walk through the audit-shaped questions to answer before you build.
Whether you are in California, Texas or Washington in the US, France or Italy in Europe, the UAE or Saudi Arabia in the Gulf, or here in India, Velura Labs delivers this end to end. Talk to us about your context.