Pith. sign in

REVIEW 5 major objections 4 minor 29 references

A 184M-parameter classifier outperforms an 8B guardrail on every prompt-injection benchmark while maintaining zero false positives on benign agentic prompts.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A 184M-parameter DeBERTa-v3 fine-tuned model is claimed to beat Llama-Guard-3-8B on all tested prompt-injection benchmarks while adding BFSI regulatory labels, but a leaked training/eval overlap undermines the zero-FPR claim.

T0 review reviewed 2026-08-02 challenge →

load-bearing objection Real engineering substance and a useful benchmark comparison, but the headline zero-FPR claim is undermined by the paper's own contamination audit, and the tables have internal inconsistencies. the 5 major comments →

arxiv 2607.22545 v1 pith:OKTJI2XY submitted 2026-05-06 cs.LG cs.AIcs.CLcs.CR

Semalith v1.4: A Calibrated 184M Safety Classifier Achieving State-of-the-Art Prompt-Injection Detection at 44x Fewer Parameters than Llama-Guard-3-8B

Tejasvi C. Addagada This is my paper
classification cs.LG cs.AIcs.CLcs.CR
keywords prompt injection detectionsafety classifierDeBERTa-v3small language modelsfalse positive ratebenchmark contaminationfinancial services complianceLlama-Guard-3 comparison
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Semalith v1.4 is a compact encoder-based safety classifier that claims to match or beat Llama-Guard-3-8B on all seven prompt-injection and adversarial-stealth benchmarks, using 44x fewer parameters. The paper argues that the key is not model scale but a three-axis taxonomy (prompt injection, general harm, financial-services compliance) trained on a contamination-controlled real-world corpus with an auxiliary super-category head. If true, this means small classifiers can handle the safety-filtering workload that currently requires large causal-LM guardrails, with a 33x latency advantage. The paper also introduces a binary-mapping correction that eliminates a systematic evaluation artifact, and it discloses six measured weaknesses and deployment guidance. A sympathetic reader would take the central claim as: for structure-detectable attacks, a well-trained 184M encoder can outperform an 8B model.

Core claim

The paper's central claim is that a 184M-parameter DeBERTa-v3-base classifier, fine-tuned with a 22-class head and a 4-class auxiliary super-category head under jointly weighted loss, achieves state-of-the-art prompt-injection detection: it wins every one of seven prompt-injection benchmarks against Llama-Guard-3-8B, often by 50–90 percentage points, while producing zero false positives on 208 benign agentic prompts (vs 0.063 for the 8B model). The same model simultaneously outputs eleven financial-services regulatory labels in one forward pass. The authors locate the reason in data diversity rather than capacity: the bottleneck in earlier 184M iterations was under-represented attack classes

What carries the argument

The object carrying the argument is a DeBERTa-v3-base encoder (184M parameters) with two heads on the [CLS] embedding: a 22-way classification head covering BENIGN, nine prompt-injection sub-types, general harm, and eleven BFSI regulatory labels, plus a 4-way auxiliary super-category head (BENIGN / D-attack / D8-harm / BFSI) trained with a jointly weighted loss. The auxiliary head is credited with preventing D-attack sub-class collapse when any sub-class is below the training-data stability floor. The training corpus is 76,204 real-world rows from 49 public sources, deduplicated with SHA-1 and MinHash against every held-out benchmark, and the evaluation uses a corrected binary mapping that t

Load-bearing premise

The claim that the training corpus is clean of the held-out evaluation prompts: the SHA-1/MinHash deduplication is only as good as its coverage, and the paper reports 169 exact collisions with AgentHarm benign rows, so the zero-false-positive result may partly reflect memorization rather than generalisation.

What would settle it

Take the released training-corpus manifest and the 208 AgentHarm-benign prompts; run a fuzzy string-match and embedding-similarity check beyond exact SHA-1. If near-duplicates beyond the 169 reported collisions exist, then retrain Semalith v1.4 with all overlapping rows removed and measure FPR on AgentHarm-benign; a meaningful rise above 0.000 would falsify the claim that the zero-FPR is a clean generalization result.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • For production safety filtering, a ~184M parameter encoder can replace an 8B causal-LM guardrail on prompt-injection workloads, cutting per-query latency from ~387ms to ~12ms and enabling deployment on consumer GPUs or edge.
  • A single forward pass can deliver three safety axes (injection, harm, compliance), so financial-services and agentic deployments need not chain multiple guardrails.
  • The zero false positives on 208 benign agentic prompts, if it holds after contamination controls, means agentic workloads can run without the false-alarm burden that plagues binary recall-maximising classifiers.
  • The methodology of SHA-1/MinHash deduplication against evaluation sets and per-class stability floors provides a template for trustworthy safety-classifier evaluation.
  • The paper's axis-stratified comparison shows that benchmark-averaged win rates are misleading: a model can dominate on one axis and lose on another, so deployment choice should be benchmark-axis driven.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The zero-FPR figure is the paper's most fragile claim: the author's own audit reports 169 exact-string collisions between training-corpus rows (WildGuardMix authority-filtered) and the AgentHarm benign prompts, so the 0.000 is not a clean out-of-distribution measurement. A fair test would remove those rows and re-measure.
  • If the auxiliary super-category head is as important as claimed, the same recipe should transfer to other small encoders (e.g., a distilled RoBERTa or a DeBERTa-large) and other domain taxonomies (healthcare, legal), offering a fast way to build domain-specific guardrails.
  • The paper's 'data diversity beats capacity' argument implies that further PI improvements will come from more diverse attack data, not larger models; a testable extension is to train on the same corpus with a 2x-larger encoder and check whether recall moves less than adding 10k new attack rows.
  • The binary-mapping correction is a reminder that evaluation pipelines for multi-label safety classifiers can systematically inflate FPR; other guardrails reporting binary flags from fine-grained heads may harbour the same artifact.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. Semalith v1.4 is a 184M-parameter DeBERTa-v3-base classifier with a 22-class head (D-attack subtypes, general harm, BFSI regulatory labels) and a 4-class auxiliary super-category head, trained on a 76,204-row corpus mined from 49 public sources. The paper claims state-of-the-art prompt-injection detection at 44x fewer parameters than Llama-Guard-3-8B, winning 7/7 prompt-injection benchmarks, 11/18 benchmarks overall, and achieving FPR = 0.000 on 208 AgentHarm benign agentic prompts. It also describes a binary-mapping methodology fix, latency measurements, McNemar significance tests, and a reproducibility harness with SHA-1 contamination audit.

Significance. If correct, this would be a useful engineering contribution: a compact multi-axis safety classifier with low latency and a disclosed three-axis taxonomy, backed by a reproducibility package, Wilson confidence intervals, and a documented audit of six weaknesses. The paper also includes useful deployment guidance distinguishing v1.3 and v1.4 operating points. However, the central empirical claims are undermined by internal contradictions and by contamination of the benchmark used for the headline zero-FPR result. These issues are load-bearing: the paper's main selling point is exactly the clean, parameter-efficient, zero-FPR comparison against Llama-Guard-3-8B.

major comments (5)
  1. [§7 vs §3.2 and Abstract] The audit in §7 reports 169 training rows overlapping AgentHarm-benign prompts (0.2218%), directly contradicting §3.2's claim that every candidate row is SHA-1/MinHash-deduplicated against every held-out benchmark and the abstract's 'contamination-clean 76,204-row corpus.' Since AgentHarm-benign is the only contaminated benchmark and is the basis of the headline FPR=0.000 on n=208, the zero-FPR figure is not a clean generalization measurement. With at most 39 uncontaminated prompts, a 0/39 result gives a Wilson upper bound of ≈0.09, not the claimed [0.000, 0.018]. This cannot be fixed by re-analysis alone.
  2. [Table 3 vs §5.2, §4.5, §6.6] Table 3 reports macro_f1 = 0.876 and per-class F1 values (D1_AUTHORITY_CLAIM 0.500, D6_AGENTIC_INJECTION 0.857, B-11 0.833) that are inconsistent with v1.4 values stated in §5.2 (val macro-F1 0.8222) and with §4.5/§6.6, which report the three enriched labels' val F1 as 0.931, 0.755, and 0.577. The table appears to be v1/v1.3 data rather than v1.4 data. This is load-bearing because the paper's claim that v1.4 fixed the thin-label gaps rests on per-class F1 numbers that are not in the results table.
  3. [Table 5, §4.4] The paper states Semalith v1.4 wins 11 of 18 benchmarks (abstract, §1, §4.4, §8), but counting the rows in Table 5 gives at least 12 Semalith wins (hackaprompt, gandalf, advbench, mosscap_l6, mosscap_l7, mosscap_l8, wildjailbreak, salad_clean_eval, attaq, aart, beavertails_test, agentharm_benign_holdout). If HarmBench-copyright is excluded as 'not a fair comparison' (§4.4), the comparison has 17 benchmarks and the count is 12. The reported win rate is arithmetically wrong and repeated in the abstract.
  4. [§7 vs §4.2 and Abstract footnote] The status of the AgentHarm-benign evaluation is contradictory. The abstract footnote and Table 4 report a completed 208-row evaluation with Wilson CI [0.000, 0.018], while §7 says 'A full validation of the AgentHarm-benign FPR claim on the complete 208-row dataset is planned for the next release cycle' and gives a different CI ([0.001, 0.027]). These statements cannot both be true; the discrepancy prevents verification of the headline zero-FPR result.
  5. [§5.3 vs Abstract/Conclusion] The 'state-of-the-art prompt-injection detection' claim is unsubstantiated. The paper compares PI performance only with Llama-Guard-3-8B and Granite-Guardian, but its own McNemar analysis (§5.3) shows PromptGuard-2-86M is significantly superior on 14 benchmarks including Mosscap, with near-ceiling recall. Without a head-to-head against the actual state-of-the-art compact PI classifier, the SOTA claim is not supported.
minor comments (4)
  1. [§5.3] The text first says 'The seven PI/stealth benchmarks with χ² > 500' and then later 'The five PI/stealth benchmarks with χ² > 500'; at least one count is wrong.
  2. [Title/Abstract] The paper calls the model 'calibrated' but provides no calibration analysis (e.g., ECE, reliability diagram, temperature scaling results). The term appears only in the title and abstract.
  3. [§7/Table 4] The Wilson CI for AgentHarm-benign differs between Table 4 ([0.000, 0.018]) and §7 ([0.001, 0.027]). A zero-event interval cannot have a positive lower endpoint, so the §7 interval is internally inconsistent with the reported 0/208.
  4. [§3.1] Typo: 'amicrosoft/deberta-v3-base' should be 'a microsoft/deberta-v3-base'. Also, Table 6's caption redundantly notes 'AttaQ (0.945 vs 0.868) is a Semalith win' when that row already marks the winner.

Circularity Check

2 steps flagged

FPR=0.000 on AgentHarm-benign is not a clean held-out prediction: the paper's own audit reports 169 training-row collisions with this eval set, and the FPR metric uses a post-hoc binary mapping.

specific steps
  1. fitted input called prediction [Abstract; Section 3.2; Section 7 (Reproducibility); Table 4 row agentharm_benign_holdout]
    "FPR = 0.000 on 208 benign agentic prompts ... 21 of 22 benchmarks at zero contamination (max 0.22%) ... Each candidate row is SHA-1- and MinHash- deduplicated against every held-out evaluation benchmark ... agentharm_benign_holdout: 169 collisions from WildGuardMix authority-filtered rows overlapping AgentHarm benign prompts."

    The headline zero-FPR result is presented as a held-out OOD evaluation, but the paper's own audit identifies the AgentHarm-benign holdout as the single contaminated benchmark, with 169 training rows overlapping its prompts. The 'contamination-clean' claim in the abstract is thus contradicted by the audit, and the FPR=0.000 is at least in part an evaluation on training data rather than a prediction on unseen benign agentic prompts. At most 39 of 208 prompts are uncontaminated; a 0/39 result gives a Wilson upper bound near 0.09, not the reported [0.000, 0.018].

  2. other [Section 4.3 (BFSI binary-mapping methodology fix); Table 4]
    "pred = 1 if SUPER[arg max]∈{1,2} else 0 ... Applied to the existing v1 checkpoint (no retraining), this drops FPR by 24% on average and lifts ToxicChat F1 by 10.5 percentage points. The methodology fix is now the published default."

    The FPR/ToxicChat numbers in the results tables are produced by a binary mapping that was changed after the fact and applied to an existing checkpoint without retraining. The reported zero-FPR and F1 improvements are therefore partly consequences of the post-hoc scoring definition, not independent model predictions. The paper discloses the change, but the headline result is not a fixed pre-registered evaluation convention.

full rationale

The decisive circularity evidence is the paper's own Section 7 contamination audit: the only nonzero contamination (0.2218%) is exactly the benchmark used for the headline zero-FPR claim, with 169 training rows overlapping AgentHarm-benign prompts. Section 3.2's promise of SHA-1/MinHash deduplication against every held-out set is therefore violated for the claim's key benchmark, and the abstract's 'contamination-clean' description is unsupported. This is not a case of malicious intent; it is a disclosed, measurable evaluation-set overlap that makes FPR=0.000 a partially circular result. The additional post-hoc binary mapping (Section 4.3) further means the reported FPR/F1 are constructed by a scoring rule chosen after evaluation, though the paper is transparent about it. The other 21 benchmarks are reported at zero contamination, and the 7/7 prompt-injection recall wins against Llama-Guard-3 are not affected by this overlap, so the paper retains substantial independent content; hence score 6 rather than higher. Self-citations to the cont7-era lineage are contextual, not load-bearing, and no uniqueness theorem or ansatz is smuggled in via citation. Section 7 also states 'A full validation of the AgentHarm-benign FPR claim on the complete 208-row dataset is planned for the next release cycle, pending infrastructure availability,' which further concedes that the headline FPR claim is not yet fully validated as a clean held-out result.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The model's behavior depends on standard fine-tuning hyperparameters, hand-set class weights, an unreported aux-loss weight, and the assumption that public benchmark labels are correct. The post-hoc binary mapping is an evaluation assumption chosen after seeing results. No invented entities are introduced.

free parameters (3)
  • Auxiliary super-category loss weight α = Grid-searched over {0.05,0.10,0.20,0.40}; exact value not given in paper
    Controls the balance between the 22-class and 4-class heads; selected by grid search on the v1 val split (§3.1, §5.1).
  • Class weights (BENIGN=1.5, D6=3.0, D7=6.0, BFSI=1.5)
    Hand-set in §3.1; influence precision/recall trade-offs on each class and are tuned to the authors' priorities.
  • Training hyperparameters (lr=2e-5, wd=0.01, batch=32, epochs=6, max_len=256)
    Standard but hand-chosen; max_len truncation is load-bearing for long prompts, and the model does not process longer inputs.
axioms (5)
  • domain assumption DeBERTa-v3-base provides a reliable feature representation for safety classification
    The entire model is a fine-tuned encoder; its transfer quality is assumed (§3.1).
  • domain assumption Public benchmark labels (HackaPrompt, Mosscap, WildGuardMix, etc.) are trusted ground truth for safety detection
    All evaluation scores use these labels as the gold standard (§4.2).
  • ad hoc to paper The BFSI binary mapping (flag if super-category ∈ {1,2}) is the correct operational definition of 'harmful' for evaluation
    This mapping was changed post hoc in §4.3 and applied to an existing checkpoint; it improves reported FPR and ToxicChat F1 without retraining.
  • domain assumption The 76,204-row corpus from 49 public sources is representative of deployment distribution, except for disclosed contamination
    The paper relies on corpus quality for generalization; the AgentHarm overlap shows this assumption is not fully met.
  • domain assumption 256-token truncation is acceptable for safety classification
    The model only scores complete prompts up to 256 tokens; longer prompts are truncated at the tokenizer (§1, §3.4).

reviewed 2026-08-02 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Semalith v1.4: A Calibrated 184M Safety Classifier Achieving State-of-the-Art Prompt-Injection Detection at 44x Fewer Parameters than Llama-Guard-3-8B." pith.science (2026). https://pith.science/paper/OKTJI2XY

@misc{pith2026260722545,
  author       = {Pith},
  title        = {Pith review of: Semalith v1.4: A Calibrated 184M Safety Classifier Achieving State-of-the-Art Prompt-Injection Detection at 44x Fewer Parameters than Llama-Guard-3-8B},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OKTJI2XY}},
  note         = {Machine review of arXiv:2607.22545}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Deploying large language models in financial-services and agentic settings requires safety classifiers that simultaneously handle prompt injection, regulatory compliance, and general harm, a combination no existing open guardrail addresses in a single inference pass. Semalith v1.4 is a 184M-parameter DeBERTa-v3-base classifier performing simultaneous three-axis safety classification including prompt injection, general harm, and financial-services regulatory compliance, in a single forward pass. Its 22-class head (BENIGN, nine prompt-injection sub-types, general-harm, eleven BFSI labels) is trained with a 4-class auxiliary super-category head under jointly weighted loss, on a 76,204-row corpus mined from 49 public sources with SHA-1 deduplication against every held-out evaluation set, with 21 of 22 benchmarks at zero contamination (max 0.22%). Against Llama-Guard-3-8B on 22 held-out benchmarks, Semalith v1.4 wins every prompt-injection evaluation (7/7) and 11 of 18 benchmarks overall at 44x fewer parameters, with FPR = 0.000 on 208 benign agentic prompts vs 0.063 for Llama-Guard-3-8B. On general-harm benchmarks (WildGuardMix, HEx-PHI, HarmBench), Llama-Guard-3 leads; this complementary split is documented in Section 4. Six measured weak spots are disclosed in Section 6. Deployment guidance: v1.3 is recommended for conversational moderation deployments (ToxicChat F1 0.624); v1.4 is recommended when BFSI label coverage or zero-FPR on benign agentic prompts is the priority.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

29 extracted references · 15 linked inside Pith

  1. [1]

    DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre- Training with Gradient-Disentangled Embedding Sharing,

    P. He, J. Gao, W. Chen. “DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre- Training with Gradient-Disentangled Embedding Sharing,”ICLR 2023, arXiv:2111.09543

    Pith/arXiv arXiv 2023

  2. [2]

    Llama Guard: LLM-based Input-Output Safeguard for Human- AI Conversations,

    H. Inan, K. Upasani, J. Chi, R. Rungta, K. Iyer, Y. Mao, M. Tontchev, Q. Hu, B. Fuller, D. Testuggine, M. Khabsa. “Llama Guard: LLM-based Input-Output Safeguard for Human- AI Conversations,” arXiv:2312.06674, 2023

    Pith/arXiv arXiv 2023

  3. [3]

    Llama Guard 3 8B Model Card,

    Meta AI. “Llama Guard 3 8B Model Card,” 2024.https://huggingface.co/meta-llama/ Llama-Guard-3-8B

    2024

  4. [4]

    Granite Guardian,

    IBM Research. “Granite Guardian,” arXiv:2412.07724, 2024

    Pith/arXiv arXiv 2024

  5. [5]

    WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs,

    S. Han, H. Zeng, H. Shin, A. Askell, A. Bakhtin, et al. “WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs,”NeurIPS 2024, arXiv:2406.18495

    Pith/arXiv arXiv 2024

  6. [6]

    PromptGuard 2: Robust Prompt Injection and Jailbreak Detection,

    Meta AI. “PromptGuard 2: Robust Prompt Injection and Jailbreak Detection,” 2025. https://huggingface.co/meta-llama/Prompt-Guard-2-86M

    2025

  7. [7]

    AprielGuard: A Lightweight Safety Classifier for Enterprise AI,

    ServiceNow Research. “AprielGuard: A Lightweight Safety Classifier for Enterprise AI,” arXiv:2512.20293, 2025

    arXiv 2025

  8. [8]

    OWASP Top 10 for Large Language Model Applications,

    OWASP. “OWASP Top 10 for Large Language Model Applications,” Ver- sion 2.0, 2025.https://owasp.org/www-project-top-10-for-large-language-model- applications/

    2025

  9. [9]

    Domain Shift Amplifies False Positive Rates in LLM Safety Classifiers: Evidence from Financial-Services Deployments,

    Y. Zhang, M. Ren. “Domain Shift Amplifies False Positive Rates in LLM Safety Classifiers: Evidence from Financial-Services Deployments,”NAACL 2025 Industry Track, 2025

    2025

  10. [10]

    PINT: Prompt Injection Test,

    Lakera AI. “PINT: Prompt Injection Test,” 2024.https://lakera.ai/pint (commercial benchmark; cited for context, not measured in this work)

    2024

  11. [11]

    Ignore Previous Prompt: Attack Techniques For Language Models,

    F. Perez, I. Ribeiro. “Ignore Previous Prompt: Attack Techniques For Language Models,” NeurIPS 2022 Workshop on ML Safety, arXiv:2211.09527

    Pith/arXiv arXiv 2022

  12. [12]

    Gandalf: Prompt Extraction Challenge,

    Lakera AI. “Gandalf: Prompt Extraction Challenge,” 2023.https://gandalf.lakera.ai/. 14

    2023

  13. [13]

    Mosscap: Escalating-Defense Prompt-Injection Benchmark,

    Lakera AI. “Mosscap: Escalating-Defense Prompt-Injection Benchmark,” 2024. https: //huggingface.co/datasets/Lakera/mosscap

    2024

  14. [14]

    Universal and Transfer- able Adversarial Attacks on Aligned Language Models,

    A. Zou, Z. Wang, N. Carlini, M. Nasr, J.Z. Kolter, M. Fredrikson. “Universal and Transfer- able Adversarial Attacks on Aligned Language Models,” arXiv:2307.15043, 2023

    Pith/arXiv arXiv 2023

  15. [15]

    AART: AI-Assisted Red-Teaming with Diverse Data Generation for New LLM-Powered Applications,

    B. Radharapu, K. Harrison, L. Aroyo, P. Bhatt. “AART: AI-Assisted Red-Teaming with Diverse Data Generation for New LLM-Powered Applications,” arXiv:2311.08592, 2023

    Pith/arXiv arXiv 2023

  16. [16]

    AttaQ: Automated Attack-Generation for Testing Question-Answer Models,

    I. Bhatt, C. Bhatt, L. Shpigel Nacson, et al. “AttaQ: Automated Attack-Generation for Testing Question-Answer Models,”ACL 2024 Workshop on Customizable NLP, arXiv:2406.10590

    Pith/arXiv arXiv 2024

  17. [17]

    HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal,

    M. Mazeika, L. Phan, X. Wang, A. Zou, Z. Wang, N. Mu, E. Sakhaee, N. Li, S. Basart, B. Li, D. Forsyth, D. Hendrycks. “HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal,”ICML 2024, arXiv:2402.04249

    Pith/arXiv arXiv 2024

  18. [18]

    Fine-Tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!

    X. Qi, Y. Zeng, T. Xie, P.-Y. Chen, R. Jia, P. Mittal, P. Henderson. “Fine-Tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!”ICLR 2024, arXiv:2310.03693

    Pith/arXiv arXiv 2024

  19. [19]

    ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversation,

    Z. Lin, Z. Wang, Y. Tong, Y. Wang, Y. Guo, Y. Wang, J. Shang. “ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversation,”EMNLP 2023 Findings, arXiv:2310.17389

    Pith/arXiv arXiv 2023

  20. [20]

    BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset,

    J. Ji, M. Liu, J. Dai, X. Pan, C. Zhang, C. Bian, B. Chen, R. Sun, Y. Wang, Y. Yang. “BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset,” NeurIPS 2023

    2023

  21. [21]

    SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models,

    L. Li, Y. Dong, R. Wang, X. Hu, W. Zuo, D. Lin, Y. Qiao, J. Shao. “SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models,”ACL 2024 Findings, arXiv:2402.05044

    Pith/arXiv arXiv 2024

  22. [22]

    SimpleSafetyTests: A Test Suite for Identifying Critical Safety Risks in Large Language Models,

    B. Vidgen, S. Agrawal, A. Ahmed, V. Blili-Hamelin, K. Brown, A. Drouhard, et al. “SimpleSafetyTests: A Test Suite for Identifying Critical Safety Risks in Large Language Models,” arXiv:2311.08370, 2023

    Pith/arXiv arXiv 2023

  23. [23]

    AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents,

    M. Andriushchenko, F. Croce, N. Flammarion. “AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents,” arXiv:2410.09024, 2024

    Pith/arXiv arXiv 2024

  24. [24]

    Probable Inference, the Law of Succession, and Statistical Inference,

    E.B. Wilson. “Probable Inference, the Law of Succession, and Statistical Inference,”Journal of the American Statistical Association, 22(158):209–212, 1927

    1927

  25. [25]

    The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning,

    N. Li, A. Pan, A. Gopal, S. Pinder, G. Stephan, A. Padhi, N. Warrington, D. Biderman, J. Zou, M. Fredrikson, Z.C. Lipton. “The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning,”ICML 2024, arXiv:2403.03218

    Pith/arXiv arXiv 2024

  26. [26]

    Investment Adviser Code of Ethics,

    U.S. Securities and Exchange Commission. “Investment Adviser Code of Ethics,” Rule 206(4)-7, 17 CFR §275.206(4)-7, 2004.https://www.sec.gov/rules/final/ia-2256.htm

    2004

  27. [27]

    Conduct of Business Sourcebook (COBS),

    Financial Conduct Authority (UK). “Conduct of Business Sourcebook (COBS),” FCA Handbook, 2024.https://www.handbook.fca.org.uk/handbook/COBS/

    2024

  28. [28]

    Markets in Financial Instruments Directive II (MiFID II),

    European Parliament and Council. “Markets in Financial Instruments Directive II (MiFID II),” Directive 2014/65/EU, 2014.https://eur-lex.europa.eu/legal-content/EN/TXT/ ?uri=CELEX:32014L0065. 15

    2014

  29. [29]

    Artificial Intelligence Act,

    European Parliament and Council. “Artificial Intelligence Act,” Regulation (EU) 2024/1689, 2024.https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689. 16

    2024

This paper was first reviewed by deepseek-v4-flash on August 2, 2026.