{"id":"59c3957a-0a5a-41f5-9852-9532313d26d8","arxiv_id":"2412.07724","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Granite Guardian 2B and 8B are open-source LLM guardrails that detect harmful content, jailbreaks, and RAG hallucination risks, reporting AUC 0.871 on harm benchmarks and 0.854 on groundedness benchmarks.","lead":"IBM researchers released Granite Guardian, a pair of open-source AI safety models that flag harmful prompts and responses, including jailbreaks and factual errors in retrieval-augmented chatbots. If the reported scores hold up, it is one of the few single guardrails that covers both content safety and hallucination checks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline harm AUC may be inflated by train/test overlap: Granite Guardian trains on HH-RLHF prompts, and several evaluated harm benchmarks (AegisSafetyTest, BeaverTails, SafeRLHF) are also derived from HH-RLHF, with no documented decontamination.","rationale":"The reader's weakest assumption identifies the same load-bearing concern I find: the absence of demonstrated decontamination between Granite Guardian's HH-RLHF-heavy training data and the HH-RLHF-derived evaluation benchmarks. This is not an ad hominem issue and not merely a missing citation; it directly determines whether the headline AUC 0.871 is evidence of generalization or of memorization. The paper's own text supplies the ingredients for the concern: Section 3.1 documents HH-RLHF as a training source, and Section 5.3 documents HH-RLHF as the source for AegisSafetyTest, BeaverTails, and SafeRLHF. The single sentence in Section 6.1 about train-test overlap is explicitly about baselines, not about Granite Guardian's own data, so the paper does not refute the concern. I do not think this warrants rejection: the models are released, the annotation and template design are described in detail, and the RAG results are compared on the standard TRUE benchmark. The issue is addressable by a decontamination analysis or by releasing overlap statistics. Since the reader's verdict is already CONDITIONAL with the same concern, my read does not change that verdict. A concrete overlap check would settle whether the concern lands, so I recommend keeping the conditional verdict rather than moving to accept or reject.","tokens_in":28305,"tokens_out":3180,"duration_ms":31037,"concrete_test":"Compute exact and near-duplicate overlap between Granite Guardian's training samples and the three HH-RLHF-derived evaluation sets: AegisSafetyTest (359 prompt-only samples), BeaverTails (3,021 response samples), and SafeRLHF (2,000 response samples). Because the training data includes first-turn HH-RLHF prompts, check each benchmark test prompt against the released HH-RLHF split identifiers, and also run n-gram or embedding-based near-duplicate matching to catch lightly edited variants. Then recompute the Table 6 aggregate AUC and per-dataset AUC on the non-overlapping subset. If the overlap is non-trivial and the AUC drops materially (for example, by more than 0.03 on the aggregate or on AegisSafetyTest specifically), the headline 0.871 is substantially explained by train-test overlap rather than by out-of-distribution generalization.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim — that Granite-Guardian-3.0-8B reaches AUC 0.871 on aggregated harm benchmarks and outperforms Llama Guard and ShieldGemma (Table 6) — depends on those benchmarks measuring out-of-distribution generalization. That condition is not established. Section 3.1 says the first human-annotation phase used first-turn HH-RLHF prompts (7,000 prompt-response pairs), and the second phase selected 1,000 more HH-RLHF-derived examples via uncertainty sampling. Section 3.2 adds synthetic jailbreak and RAG data, but nothing in the paper reports filtering HH-RLHF instances out of the evaluation sets. The benchmarks in Table 4 include AegisSafetyTest, which Section 5.3 itself describes as 'derived from Anthropic's HH-RLHF harmlessness dataset'; BeaverTails, whose 'prompts are derived from HH-RLHF red teaming'; and SafeRLHF, which 'includes the prompts of the BeaverTails dataset'. Section 6.1 states only that 'The evaluations results do not consider train-test overlap for baselines', which addresses baseline contamination, not whether Granite Guardian's own training data overlaps the test sets. If the HH-RLHF first-turn prompts used for training appear verbatim or near-verbatim in AegisSafetyTest, BeaverTails, or SafeRLHF test instances, the quoted AUC reflects memorization of benchmark-specific inputs rather than generalizable risk detection. This is a concrete, checkable threat to the paper's central empirical claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Granite Guardian, a suite of 2B and 8B safeguard models fine-tuned from Granite 3.0 instruct models, designed to detect harmful content (harm, social bias, profanity, violence, sexual content, unethical behavior, jailbreaking) and RAG-specific risks (context relevance, groundedness, answer relevance). The training data combines human annotations on HH-RLHF-derived prompts and responses, synthetic contrastive and jailbreak data, and synthetic RAG data generated from HotPotQA, SQuAD2, MNLI, and SNLI. The authors evaluate the models on public harmfulness benchmarks and TRUE groundedness benchmarks, reporting an aggregate harm AUC of 0.871 for the 8B model and an average groundedness AUC of 0.854, and comparing against Llama Guard, ShieldGemma, ANLI, WeCheck, and MiniCheck.","tokens_in":28624,"tokens_out":4844,"duration_ms":42020,"significance":"If the reported results are robust, the paper makes a useful contribution: an open-source, unified guardrail covering both content-safety and RAG hallucination risks, with publicly released model weights, cookbooks, and a flexible safety-instruction template that supports custom risk definitions. The use of multiple human annotators per sample, reported inter-annotator agreement, and evaluation across low false-positive-rate operating points are strengths. However, the central generalization claim is currently not established because the paper does not demonstrate that the training data are disjoint from several of the key evaluation benchmarks, and the abstract's 'state-of-the-art' wording is stronger than the paper's own tables support. These issues are repairable and do not invalidate the underlying model development, but they must be addressed before the headline claims can be accepted.","major_comments":[{"comment":"The paper does not establish that Granite Guardian's own training data are disjoint from the evaluation benchmarks, although several benchmarks are derived from the same source as the training data. Section 3.1 states that the first annotation phase used 7,000 first-turn prompts from HH-RLHF and the second phase selected 1,000 additional HH-RLHF-derived examples, while Section 5.3 and Table 4 describe AegisSafetyTest as 'derived from Anthropic's HH-RLHF harmlessness dataset', BeaverTails prompts as 'derived from HH-RLHF red teaming', and SafeRLHF as including BeaverTails prompts. The only statement about overlap, in Section 6.1, says 'The evaluations results do not consider train-test overlap for baselines', which addresses the baselines rather than Granite Guardian. Because the headline harm AUC of 0.871 is computed on exactly these datasets, the authors should report a formal decontamination analysis (for example, exact and near-duplicate overlap of training and evaluation prompts and responses, with explicit thresholds), retrain without the overlapping sources, or re-state the claims as in-distribution performance. This is load-bearing for the abstract's generalization claim.","section":"§3.1, §5.3, Table 4, §6.1"},{"comment":"The claims that Granite Guardian 'outperforms other open- and closed-source models' and is 'the most generalizable and competitive model available' are not supported by the paper's own results on the RAG benchmarks. In Table 8, Llama-3.1-Bespoke-MiniCheck-7B achieves an average AUC of 0.873 on the TRUE datasets, which is higher than Granite-Guardian-3.0-8B's 0.854, and the text in Section 6.2 correctly states that the 8B model 'ranks second on average AUC' and is 'the best-performing fully open-source model'. The abstract and introduction should be revised to report the comparative ranking accurately, and the statement about closed-source models should be removed or accompanied by closed-source baselines, since Section 5.2 lists no closed-source systems.","section":"§6.2, Table 8; Abstract; §1"},{"comment":"The evaluation section says the selected benchmarks 'prioritize out-of-distribution and public datasets' (Section 5.3), but the HH-RLHF-derived benchmarks are not out-of-distribution relative to the training data described in Section 3.1. Even without verbatim overlap, the distribution of prompt topics and styles in AegisSafetyTest, BeaverTails, and SafeRLHF is substantially represented in the training sample, so the aggregated AUC partly reflects distributional familiarity rather than pure out-of-distribution generalization. To support the 'most generalizable' claim, the authors should report performance separately on benchmarks whose source distributions are disjoint from training, such as ToxicChat, OpenAI Moderation, SimpleSafetyTests, and HarmBench, and discuss any performance differences.","section":"§5.3, §6.1"}],"minor_comments":[{"comment":"The column headers MNBN, QX, QC, SumE, and DialF do not match the dataset names used in Section 5.3 (MNBM, QAGS-XSUM, QAGS-CNN/DM, SummEval, and DialFact, respectively); please align the table with the text.","section":"Table 8"},{"comment":"'This category is applies to both prompts and responses' should read 'This category applies to both prompts and responses.'","section":"§2.1.1"},{"comment":"'any instance in these datasets that have have been marked' contains a duplicated 'have' that should be removed.","section":"§5.3"},{"comment":"The notation U|k and S|k should be defined more precisely: 'top-k tokens' should specify top-k by log-likelihood and whether the selection is restricted to the first generated token position before the softmax normalization.","section":"§4.3"},{"comment":"The limitations section does not mention the train-evaluation overlap issue; a sentence acknowledging the need for decontamination analysis in future releases would improve transparency.","section":"§7.2"},{"comment":"The training setup reports a learning rate, optimizer, and epoch count, but the dataset size and the validation split used for checkpoint selection are not stated; please provide these details for reproducibility.","section":"§4.2"}],"recommendation":"major_revision","confidential_remarks":"The contamination concern raised by the stress-test is real and directly affects the advertised SOTA numbers, but it is checkable and fixable through decontamination reporting or claim softening, so I do not think rejection is warranted. The paper would also benefit from a model card or data-provenance appendix documenting the overlap status of every benchmark relative to the training data."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing: Granite Guardian is a real contribution. An open-weights 2B/8B model that jointly covers harmful content, jailbreaks, and RAG groundedness, with a sensible safety-instruction template and a careful human-annotation pipeline (inter-annotator agreement reported, diverse annotators). The synthetic data generation for jailbreaks and RAG risks is documented in enough detail to reproduce. Releasing weights and code is a plus. The RAG evaluation on TRUE is thorough, and the results are respectable even if MiniCheck beats them on average AUC.\n\nSecond thing: the headline harm AUC of 0.871 is not trustworthy as reported. The training data for the first phase comes from first-turn HH-RLHF prompts (Section 3.1), and the evaluation includes AegisSafetyTest, BeaverTails, and SafeRLHF, all of which are derived from HH-RLHF (Section 5.3). The paper's only statement on overlap, that \"evaluations results do not consider train-test overlap for baselines,\" addresses baselines, not Granite Guardian's own training data. No decontamination is reported for their own set. This is a concrete, checkable threat: if those first-turn prompts appear verbatim or near-verbatim in the test benchmarks, the AUC reflects memorization, not generalization. The stress-test note holds up on reading the paper.\n\nThe overclaim in the abstract (\"most generalizable and competitive model available\") is also not supported: MiniCheck has a higher average AUC on RAG, and there is no comparison with WildGuard or commercial moderation APIs.\n\nA softer but real gap: the paper evaluates only the umbrella harm definition and groundedness, not the specific sub-dimensions (bias, profanity, sexual content, etc.) it claims to cover. That is a stated scope choice, but it means the headline claims are narrower than the model's advertised coverage.\n\nThe contamination problem is fixable. They can run a decontamination analysis (string or embedding overlap), re-report the AUCs on a filtered set, or at minimum disclose the overlap. The model itself appears real and the code release makes it easy to check. I would send this to peer review and ask for that decontamination analysis plus a more measured framing. The paper is worth engaging with, but the empirical headline needs to be verified before being cited.","headline":"A genuinely useful open guardrail model release, but the headline harm AUC is threatened by possible HH-RLHF train/test overlap that the paper never rules out.","tokens_in":29292,"tokens_out":2050,"would_cite":false,"duration_ms":20939,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Granite Guardian claims that a single open-source guardrail model can detect harmful content and RAG hallucinations at deployment-competitive accuracy, reporting an aggregated AUC of 0.871 on harm benchmarks and 0.854 on groundedness…","keywords":["risk detection","LLM guardrails","harmful content","jailbreak detection","RAG hallucination","groundedness","safety instruction template","supervised fine-tuning"],"falsifier":"Compare the released Granite Guardian training data against the test splits of AegisSafetyTest, BeaverTails, and SafeRLHF using exact-match and near-duplicate retrieval; if a material fraction of the benchmark examples appears in the training set, the aggregated AUC of 0.871 reflects memorization and the generalization claim fails.","tokens_in":28112,"feed_emoji":"🛡️","tokens_out":8266,"duration_ms":64831,"temperature":0.7,"pith_summary":"This paper introduces Granite Guardian, a pair of open-source language models (2B and 8B parameters) meant to act as a single guardrail for any LLM pipeline. The authors claim these models detect both classic harmful content—an umbrella 'harm' category plus social bias, profanity, violence, sexual content, unethical behavior, and jailbreaking—and RAG-specific hallucination risks: context relevance, groundedness, and answer relevance. On aggregated public benchmarks, the 8B model reports an AUC of 0.871 for harmful-content detection, ahead of the Llama Guard and ShieldGemma families, and an average AUC of 0.854 on the TRUE groundedness benchmark, making it the best-performing fully open-source groundedness checker. The value of the claim is that one model, released openly, can replace separate content-moderation and fact-consistency components.","feed_headline":"One open model beats rivals on harm detection and RAG groundedness","feed_subtitle":"The 8B model reports 0.871 AUC for harmful content and tops open-source RAG groundedness checkers.","key_machinery":"The mechanism that carries the argument is a safety instruction template: a prompt that names the target span (user message, assistant message, or context), inserts the text between control tokens, and applies a plain-language risk definition, instructing the model to answer 'Yes' or 'No'. The probability of risk is computed from the top-$k$ token log-likelihoods, aggregating all tokens that contain 'Yes' or 'No' as $\\text{score}_{\\text{unsafe}} = \\sum_{u \\in U|k} \\exp(\\mathrm{LL}(u))$ and softmax-normalizing the pair; with $k=20$ this yields a confidence score from the same checkpoint for any risk definition. The template allows custom risk definitions at inference time, and the synthetic-data pipeline supplies the hard cases: 24 jailbreak revision strategies and RAG negatives seeded from HotPotQA, SQuAD2, MNLI, and SNLI.","core_discovery":"The central claim is that a single supervised fine-tuned model family can detect safety risks across both the input/output moderation axis and the retrieval-augmented generation quality axis at deployment-competitive accuracy. Granite Guardian is derived from Granite 3.0 instruct checkpoints and trained on human-annotated HH-RLHF first-turn prompts paired with model responses, plus synthetic data for contrastive benign/harmful prompts, jailbreak revisions, and RAG negative samples. On the aggregated harm benchmarks, Granite-Guardian-3.0-8B achieves the best AUC (0.871), AUPRC (0.846), and F1 (0.758) among compared models; on the TRUE datasets it averages 0.854 AUC for groundedness, placing second behind MiniCheck and first among fully open-source models. The paper also reports that with the jailbreak risk definition the 8B model reaches a recall of 1.0 on ToxicChat jailbreak prompts.","pith_inferences":["The reported margin over Llama Guard and ShieldGemma may shrink once the HH-RLHF ancestry of both training and benchmark sets is accounted for; a decontamination study would settle how much of the 0.871 AUC is generalization.","The top-k yes/no scoring trick is model-agnostic: the same template could be applied to other instruction-tuned base models, so the approach may be a recipe for building guardrails rather than a one-off model.","A direct test of the jailbreak claim would be to run the model against a fresh, never-published red-teaming set; recall 1.0 on ToxicChat's jailbreak subset does not by itself show robustness to unseen attack strategies.","The RAG half of the claim could be extended from groundedness to the full triad—context relevance and answer relevance—on held-out enterprise data, which the paper does not benchmark."],"forward_implications":["Deployments can use one open model instead of separate toxic-content and fact-consistency checkers, since Granite Guardian covers both with a single inference call.","Operators can set thresholds on the computed risk probability to tune for low false-positive rates; at FPr 0.1 the 8B model reaches TPr 0.68 on the aggregated harm benchmark.","The 2B variant offers a resource-constrained option with aggregate AUC 0.782 and recall 0.747, making safety filtering feasible on smaller infrastructure.","Because the safety template accepts new risk definitions, enterprises can extend the same checkpoint to custom policies without retraining, although the paper advises testing such extensions.","Open release of the models lets practitioners audit and adapt the guardrail, supporting the paper's stated goal of shared responsible-AI infrastructure."],"supporting_citations":[{"why":"Supplies the Granite 3.0 instruct base models that are fine-tuned into Granite Guardian.","marker":"(Granite Team, 2024)"},{"why":"Provides the HH-RLHF first-turn human prompts that form the core of the human-annotated training data.","marker":"(Bai et al., 2022)"},{"why":"Defines Llama Guard, the main harm-detection baseline family and template-based safety-model approach.","marker":"(Inan et al., 2023)"},{"why":"Defines ShieldGemma, the second harm-detection baseline and the direct template-format comparison.","marker":"(Zeng et al., 2024)"},{"why":"Defines MiniCheck, the strongest groundedness baseline that Granite Guardian trails by 0.019 average AUC.","marker":"(Tang et al., 2024)"},{"why":"Provides the TRUE benchmark suite used for all groundedness evaluations.","marker":"(Honovich et al., 2022)"},{"why":"Supplies the AegisSafetyTest benchmark, an HH-RLHF-derived test whose overlap risk bears on the harm AUC.","marker":"(Ghosh et al., 2024)"},{"why":"Supplies the BeaverTails response-harmfulness benchmark, another HH-RLHF-derived test set.","marker":"(Ji et al., 2023)"},{"why":"Supplies the SafeRLHF response benchmark, also descended from the HH-RLHF prompt pool.","marker":"(Dai et al., 2024)"},{"why":"Provides the jailbreak taxonomy and seed revision strategies used to synthesize adversarial training data.","marker":"(Rawat et al., 2024)"}],"fun_headline_variants":["Open Granite Guardian 8B tops safety and RAG checks","Granite Guardian: 0.871 AUC harm, 0.854 RAG grounding","One open model excels at harm detection and groundedness","Granite Guardian 8B leads open models in safety, RAG","Open-source guard tops both harm and hallucination tests"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline scores assume that Granite Guardian's own training data does not overlap with the benchmark examples used to test it, but several test sets (AegisSafetyTest, BeaverTails, SafeRLHF) and the training prompts share the same HH-RLHF source, and the paper does not show that these overlaps were removed.","fun_headline_variants_meta":{"raw":{"variants":["Open Granite Guardian 8B tops safety and RAG checks","Granite Guardian: 0.871 AUC harm, 0.854 RAG grounding","One open model excels at harm detection and groundedness","Granite Guardian 8B leads open models in safety, RAG","Open-source guard tops both harm and hallucination tests"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000354,"raw_usage":{"total_tokens":1912,"prompt_tokens":917,"completion_tokens":995,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":533,"completion_tokens_details":{"reasoning_tokens":904}},"tokens_in":533,"tokens_out":995,"duration_ms":9103,"temperature":1.0,"reasoning_tokens":904,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:33:17.428219+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare the released Granite Guardian training data against the test splits of AegisSafetyTest, BeaverTails, and SafeRLHF using exact-match and near-duplicate retrieval; if a material fraction of the benchmark examples appears in the training set, the aggregated AUC of 0.871 reflects memorization and the generalization claim fails.","supporting_citations":[{"cited_title":"MiniCheck: Efficient fact-checking of LLMs on grounding documents","cited_arxiv_id":null,"evidence_quote":"Defines MiniCheck, the strongest groundedness baseline that Granite Guardian trails by 0.019 average AUC."}],"review_version":1}