REVIEW 26 cited by
NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems. Guardrails (or rails for short) are a specific way of controlling the output of an LLM, such as not talking about topics considered harmful, following a predefined dialogue path, using a particular language style, and more. There are several mechanisms that allow LLM providers and developers to add guardrails that are embedded into a specific model at training, e.g. using model alignment. Differently, using a runtime inspired from dialogue management, NeMo Guardrails allows developers to add programmable rails to LLM applications - these are user-defined, independent of the underlying LLM, and interpretable. Our initial results show that the proposed approach can be used with several LLM providers to develop controllable and safe LLM applications using programmable rails.
Forward citations
Cited by 26 Pith papers
-
Rewriting the Response Path: Silent Tampering and Provider-Signed Defense in BYOK LLM Agents
A malicious relay can strategically rewrite aligned LLM outputs in BYOK agent architectures to achieve up to 99.1% attack success on benchmarks like AgentDojo and ASB.
-
Governed Capability Evolution: Lifecycle-Time Compatibility Checking and Rollback for AI-Component-Based Systems, with Embodied Agents as Case Study
A governed upgrade framework with interface, policy, behavioral, and recovery checks keeps unsafe activations at zero across multi-round AI capability upgrades on a PyBullet/ROS 2 manipulation testbed while retaining ...
-
Dynamic Capability Scoping for Enterprise AI Agents: A Synthetic Dataset and Three-Source Permission Architecture
A three-source permission-scoping architecture for enterprise AI agents is proposed alongside a 600-prompt synthetic dataset labeled with minimum required permissions, validated at Cohen's kappa 0.917–0.967.
-
Geometry-Guided Constraint Learning for LLM Safety Classification
Sparse-autoencoder features reduce the number of safety constraints needed to two for most BeaverTails categories, and a three-phase-trained cone constraint modestly beats a flat polytope on in-distribution Qwen3.5-9B...
-
RAIL Guard: Closing the Evaluation-to-Remediation Gap in Responsible AI for LLM Agents
Closed-loop remediation on eight responsible-AI dimensions converges far more often than block-and-retry (96.9% vs 49.1%) and pre-tool-call evaluation cuts unsafe agent executions by 33%.
-
Reliable Weak-to-Strong Monitoring of LLM Agents
Monitor scaffolding, not monitor awareness or omniscience, drives detection reliability, and a hybrid chunked monitor lets weak models supervise strong LLM agents.
-
AI-guided digital intervention with physiological monitoring reduces intrusive memories after experimental trauma
An AI-guided version of the Imagery Competing Task Intervention, with pupillometry, reduced self-reported intrusive memories after analogue trauma in a preregistered randomized experiment.
-
Unified Multi-Task Learning & Model Fusion for Efficient Language Model Guardrailing
Small discriminative classifiers trained on synthetic guardrail data, plus a bandit-based model merging search, beat GPT-4-level moderation systems on multiple safety benchmarks.
-
Adversarial Reasoning at Jailbreaking Time
A loss-guided 'reason, verify, search' loop with three LLM modules surpasses prior semantic jailbreak methods on several defended models.
-
Aegis2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrails
Aegis2.0 provides a commercially usable, human-annotated safety dataset with 24 risk categories, and models trained on it with parameter-efficient methods match WildGuard and beat Llama Guard 3.
-
A Flexible Large Language Models Guardrail Development Methodology Applied to Off-Topic Prompt Detection
A synthetic-data-driven method trains lightweight classifiers to flag user prompts as off-topic relative to a system prompt, outperforming heuristics and partially transferring to jailbreak and harmful content.
-
NiyamAI - An Intent-Bound AI Agent with Cryptographically Verifiable Guardrails using Zero-Knowledge Proofs
Niyam-AI binds agent permissions with SHA-256 and adds zk-SNARK proofs for a small Judge model's safety decisions, reporting F1 88.5% on Agent-SafetyBench, though the classifier is benchmark-adapted and the proof appl...
-
Reason Less, Verify More: Deterministic Gates Recover a Silent Policy-Violation Failure Mode in Tool-Using LLM Agents
Deterministic read-only pre-execution gates raise τ²-bench airline success from 29.6% to 42.0% on gpt-4o-mini by blocking silent policy-violating tool writes, with the lift replicated on disjoint seeds.
-
Autonomous Topology Mutation: Safe Runtime Restructuring for Multi-Agent LLM Systems with Capability, State, and Shadow Invariants
A telemetry-triggered 'topology mutation' mechanism splits overloaded LLM agents into specialized children, gated by capability, state-routing, and shadow-validation invariants.
-
Fence: Specialized SLM Guardrails for LLM Applications
Small language models trained on GAN-style adversarial synthetic data outperform prompted large models on use-case-specific guardrails for off-topic and prompt-injection detection.
-
SEALGuard: Safeguarding the Multilingual Conversations in Southeast Asian Languages for LLM Software Systems
A LoRA-adapted SeaLLM model detects unsafe and jailbreak prompts in nine Southeast Asian languages with 97% recall and 98% F1 on the authors' new SEALSBench benchmark, far above zero-shot LlamaGuard.
-
Agent Safety Should Be a Runtime Contract
Agent safety should be a runtime contract, enforced by sandboxes and permission gates on the preventive side and by verifiable evidence chains on the submission side, not by model alignment alone.
-
Phionyx: A Deterministic AI Runtime Architecture with Structured State Management and Pre-Response Governance
A deterministic 46-block runtime wrapper around LLMs with pre-response safety gates and impact-weighted cache eviction reports reproducible control signals and modest efficiency gains in scoped single-instance tests.
-
OneShield -- the Next Generation of LLM Guardrails
A paper describes OneShield, a model-agnostic guardrail framework with parallel risk detectors and a policy manager, and reports its enterprise deployment and use in InstructLab.
-
JavelinGuard: Low-Cost Transformer Architectures for LLM Security
A study of five small transformer classifier architectures for LLM jailbreak and prompt injection detection claims low-latency accuracy comparable to large models, led by the multi-task Raudra design.
-
Relative Bias: A Comparative Framework for Quantifying Bias in LLMs
A model is 'relatively biased' when its responses deviate from the consensus of a baseline LLM set, and this deviation can be scored by embedding distances or LLM judges plus equivalence tests.
-
Improved Large Language Model Jailbreak Detection via Pretrained Embeddings
A random forest on Snowflake embeddings detects jailbreak prompts with F1 0.96 on JailbreakHub, but the high score depends on training on the same in-the-wild jailbreak source used in that benchmark.
-
The Hitchhikers Guide to Production-ready Trustworthy Foundation Model powered Software (FMware)
The paper catalogs the lifecycle stages and production-readiness challenges of software built around foundation models (FMware) and proposes an action plan of engineering practices and research directions.
-
Bridging the Safety Gap: A Guardrail Pipeline for Trustworthy LLM Inferences
A guardrail pipeline combining detection, retrieval grounding, rule-based wrappers, and a repair model is reported to match OpenAI moderation and fix 80.7 percent of hallucinated HaluEval answers.
-
Challenges in Guardrailing Large Language Models for Science
A position paper proposing a guardrail framework with four dimensions (trustworthiness, ethics & bias, safety, legal) and implementation strategies for scientific LLM use.
-
Creating an LLM-based AI-agent: A high-level methodology towards enhancing LLMs with APIs
A diploma thesis that reviews existing techniques for giving LLMs access to APIs and proposes a largely untested on-device architecture based on semantic matching to predefined macros.
Discussion (0). Continue with ORCID to comment.