Pith. sign in

REVIEW 5 major objections 5 minor 50 references

Personalized Constitutionally-Aligned Agentic Superego: Secure AI Behavior Aligned to Diverse Human Values

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A modular 'superego' guardrail checks each proposed agent action against a universal safety floor and user-chosen constitutions before execution, and on benchmark tests cuts harm by up to 98.3% and raises harmful-prompt refusals to…

desk verdict A real prototype with a plausible architecture, but the headline harm-reduction numbers depend on an unnamed LLM judge, and the personalization feature that is actually novel is never benchmarked. read the letter →

arxiv 2506.13774 v2 pith:MBIIOPUF submitted 2025-06-08 cs.AI cs.CYcs.MA

Nell Watson , Ahmed Amer , Evan Harris , Preeti Ravindra , Shujun Zhang This is my paper
classification cs.AIcs.CYcs.MA
keywords agenticAIalignmentpersonalizedvaluesconstitutionalethicalguardrailscomplianceenforcementModelContextProtocolharmbenchmarks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the hard part of aligning agentic AI—respecting both universal safety rules and personal, cultural, or organizational values—can be moved out of the model and into a separate oversight agent. The proposed 'superego' sits between the planning and execution steps of an inner agent, checks each proposed action against a hand-written Universal Ethical Floor and user-selected 'Creed Constitutions,' and can allow, block, modify, or ask for clarification before anything happens. On standard harm benchmarks, the configuration cuts measured harm scores by up to 98.3% and raises refusal rates on harmful prompts to near or exactly 100% on tested models, without fine-tuning the inner model. The point of the paper is that personalized alignment becomes a configuration problem—pick constitutions, dial adherence levels—rather than a retraining problem.

What carries the argument

The load-bearing mechanism is the hierarchical enforcement loop of Algorithm 1: every proposed action is first checked against the Universal Ethical Floor (UEF), a non-negotiable safety baseline; only if that passes are the user-selected constitutions applied, each with a dialable 1–5 adherence level. The compliance enforcer then emits one of four verdicts—Allow, Block, Modify with a compliant alternative, or Request Clarification. Constitutions are delivered to the agent at runtime through the Model Context Protocol, so the rule set is external, versionable, and shareable rather than baked into weights.

What would settle it

Run the same superego configuration against a jailbreak suite generated after the UEF was frozen, using an inner agent explicitly optimized to produce plans that are harmful but phrased in the constitution's own language; if harm scores return to baseline levels, the classification premise that carries the architecture fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a modular, externally enforced constitutional layer can steer existing LLM agents toward safer and more personalized behavior with no change to the base model. Even configured with only the universal floor, the superego reduces HarmBench attack success rates substantially across several inner models and, on AgentHarm's harmful set, brings average refusal rates to 99.4% for one leading model and 100% for another. The same mechanism is what lets a user apply a vegan, Halachic, or K-12 constitution: the compliance enforcer evaluates the inner agent's plan against the selected rules and the user's 1–5 adherence dial before execution.

Load-bearing premise

The approach depends on the LLM doing the superego's judging being able to correctly tell whether a proposed plan violates the hand-written rules; the paper does not specify which model performs this judgment or show that its judgments generalize, and it admits that a weaker judge can be outmaneuvered by a cleverer inner agent.

Editorial extensions

If this is right

  • Existing agentic systems can be retrofitted with a stronger safety and personalization layer without retraining or altering the underlying model.
  • Personalization shifts from prompt engineering to selecting, forking, and dialing constitutions, lowering the technical burden on non-expert users.
  • The non-negotiable Universal Ethical Floor prevents personalization from sliding into moral relativism, since it takes priority over every user-selected rule.
  • If the benchmark results hold, harmful outputs on tested adversarial and agentic suites can be reduced to near-total refusal levels, while targeted tuning can keep genuine false positives near baseline rates.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's own experiments, the same pre-execution checkpoint could double as an auditable compliance log for regulated deployments, since every plan carries a recorded allow/block/modify verdict and a reason.
  • A marketplace of shared constitutions will need an explicit arbitration mechanism for conflicts between constitutions at different adherence levels; the paper sketches weights and thresholds but leaves the resolution algorithm unspecified.
  • The architecture's ceiling is set by the judge model's capability: if a sufficiently clever inner agent can hide harmful intent from the judge, the guardrail becomes cosmetic, so the most decisive extension is measuring judge-versus-agent capability gaps.
  • The mechanism could be extended beyond natural-language outputs to tool calls, potentially catching patterns like tool poisoning or data exfiltration that look benign as text but dangerous as API invocations.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The manuscript proposes an external 'Superego Agent' oversight layer for agentic LLMs: a compliance enforcer intercepts inner-agent plans and checks them against a Universal Ethical Floor plus user-selected, dialable 'Creed Constitutions,' with a prototype exposed via the Model Context Protocol and a constitutional marketplace. The authors report large safety gains on HarmBench and AgentHarm, including a 98.29% harm-score reduction for Gemini 2.5 Flash and refusal rates of 96.6-100% for Claude Sonnet 4, and claim a novel framework for personalized alignment.

Significance. If the benchmark results were reproducible, the paper would make a useful practical contribution: an external guardrail that enforces a universal safety floor while allowing user-level personalization would be model-agnostic and deployable without retraining. Strengths include a working prototype, MCP integration, use of established external benchmarks (HarmBench, AgentHarm), and a candid discussion of governance and limitations. However, the central quantitative claims are currently not reproducible because the judge model is unnamed, and the headline numbers disagree with the final tuned configuration.

major comments (5)
  1. [Abstract and Section 5.4.2] The abstract's claim of '100% with Claude Sonnet 4 on AgentHarm's harmful set' is contradicted by the final configuration reported in Section 5.4.2, which gives a 96.6% harmful-prompt refusal rate after the iterative refinement. Because the refinement occurred after the 100% result, the final configuration is the one that should be reported; the abstract, the body text, and Table 6 must be reconciled so that all headline numbers refer to the same final system.
  2. [Section 3.3.1, Algorithm 1 and Section 5.4] The load-bearing predicate violates(proposed_action, rule) is never instantiated. The paper does not name the LLM that implements the Superego compliance enforcer, its version, decoding parameters, or its agreement with human judgments on the benchmark items. Section 6.4 explicitly concedes that a substantially less capable Superego can be outmaneuvered, so the reported 98.3% harm-score reduction cannot be attributed to the architecture unless the judge model is specified and its classification behavior is characterized.
  3. [Section 5.4.2] The two-round iterative refinement of the Superego prompt and UEF on AgentHarm's benign set means the reported numbers are a joint product of judge capability, hand-written rules, and benchmark-specific calibration. The paper asserts that the revisions were universally applicable rather than benchmark-specific gaming, but no held-out evaluation, pre-registered protocol, or development/test split supports this. Please report the exact revision log and evaluate the final configuration on a separate benign or false-positive benchmark to demonstrate generalization.
  4. [Section 5.4.1, Table 5] The HarmBench reporting is too loose for the headline claim: prompt counts are approximate ('~120 (approx.)'), processed counts differ between baseline and Superego conditions (3972 vs 3938) without explanation, and one harmful output is reclassified as a false positive by the authors based on their own judgment. Report exact counts, standard API settings, unmodified classifier verdicts, and paired statistics or confidence intervals so the reductions can be independently verified.
  5. [Sections 5.1 and 5.4] The quantitative evaluation tests only the UEF, not user-selected Creed Constitutions, adherence levels, or conflict resolution, despite the paper's title and stated contribution of personalized alignment. The two experiments described in Section 5.1 are not reported as executed studies. Either add experiments with at least one non-trivial constitution and different adherence levels, or explicitly limit the empirical claims to the UEF-only safety-floor setting.
minor comments (5)
  1. [Section 3.1] The sentence 'There are at least two distinct paths to realizing this concept of a personalized superego agent' appears twice within a few paragraphs; one occurrence should be removed.
  2. [Section 5.4.2] The qualitative review of 'benign' set refusals is repeated nearly verbatim for the intermediate tuning stage and again for the final stage; please consolidate to avoid duplication and clarify which numbers correspond to which stage.
  3. [Section 5.4.3] The phrase 'logged as per Figures 1 and 3' is incorrect because Figures 1 and 3 are architecture and data-flow diagrams, not reasoning logs; please cite the actual figure or supplementary material containing the reasoning example.
  4. [Table 5] The table uses '~120 (approx.)' and inconsistent decimal places (0.7% vs 0.025%) without clear conventions; make the table self-contained and provide exact denominators for every row.
  5. [Section 5.3] The 'poemtest' observation is reported as 'emergent agent resistance' and 'intuitive suspicion,' but no experimental protocol or replication evidence is given; please label this as an anecdotal observation and note that it is not a controlled result.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the benchmark results are measured against external suites with an independent harm classifier, and the self-citations do not carry the empirical claim.

full rationale

The paper's central empirical claim is evaluated against external benchmarks (HarmBench and AgentHarm) using an independent harm classifier, so the reported harm-score reductions and refusal rates are not defined by the inputs. The Superego is given a Universal Ethical Floor that says to refuse harmful content, but whether an LLM-based judge correctly applies that constitution to benchmark prompts is an empirical outcome, not a tautology. The self-citations ([6], [7], [11]) motivate the UEF and personalization concept but are not load-bearing for the quantitative benchmark results. The strongest concerns are reproducibility and generalizability, not circularity: Section 5.4.2 describes two rounds of prompt/UEF tuning on AgentHarm's benign set before reporting final harmful-set numbers, and Algorithm 1's core violates() predicate is implemented by an unnamed LLM whose capability is conceded in Section 6.4 ('A substantially less capable Superego risks being outmaneuvered'). These are serious evaluation-validity issues, but they do not amount to a circular derivation: no equation or reported quantity is shown to equal its own input by construction, and the benchmark outcomes remain externally falsifiable.

Assumptions & free parameters 3 free parameters · 4 assumptions · 4 invented entities

The central claim rests on the hand-written UEF and tuned Superego prompt, which are free parameters, and on the domain assumption that an LLM judge can reliably apply natural-language rules. The paper introduces software concepts (Superego Agent, Creed Constitution, UEF) that are either instantiated in a demo or asserted without external validation.

free parameters (3)
  • Universal Ethical Floor (UEF) rule text = not disclosed
    The UEF is the constitution used in all benchmark runs. Its natural-language rules are hand-authored (inspired by SaferAgenticAI.org, Section 3.3.5), and the benchmark results depend directly on its content.
  • Superego system prompt = not disclosed (two-plus revisions)
    Section 5.4.2 describes iterative refinement of the Superego's instructions to reduce false positives on AgentHarm's benign set, e.g., the 'user authorization assumption' and 'contextualizing automation' clarifications. These are prompt parameters tuned against the benchmark.
  • Adherence level scale (1-5) = 1-5 Likert (design choice)
    Section 3.3.3 introduces the dialable adherence scale; it is a design parameter, not fitted to data, but it is part of the framework's behavior.
assumptions (4)
  • domain assumption A prompted LLM can reliably interpret natural-language rules and detect violations in proposed plans.
    Algorithm 1 (Section 3.3.1) assumes the compliance enforcer's violates() checks are accurate; Section 6.4 concedes a less capable Superego can be outmaneuvered.
  • domain assumption HarmBench's Llama-2-13B classifier and AgentHarm's scoring correctly measure harmfulness.
    Section 5.4.1 and 5.4.2 use these external classifiers as ground truth; the paper also notes classifier limitations but relies on them for all reported numbers.
  • ad hoc to paper The UEF is a sufficient universal ethical baseline.
    Section 3.3.5 defines the UEF as non-negotiable, but its adequacy is asserted, not derived from an external standard; an author is involved in SaferAgenticAI.org.
  • domain assumption The inner agent's proposed plans and chain-of-thought are legible to the Superego.
    Section 6.2 and 6.3 assume the Superego can read planning steps; the OpenAI guardrail interaction in Section 5.3 shows this access is not always available or permitted.
invented entities (4)
  • Superego Agent independent evidence
    purpose: External oversight module that intercepts and validates an inner agent's plans against constitutions and the UEF.
    Functional prototype demonstrated at Creed.Space and GitHub repository, so it exists independently of the paper's narrative.
  • Creed Constitution independent evidence
    purpose: Machine-readable rule set (e.g., vegan, Halachic, K-12) that users select and dial for adherence.
    API specification in Listing 1 and example constitutions are provided, so the format is concretely instantiated.
  • Universal Ethical Floor (UEF)
    purpose: Non-negotiable baseline safety rules applied in all configurations.
    The UEF is authored by the authors (inspired by SaferAgenticAI.org, which an author co-founded) and its content is not disclosed, so its adequacy and universality remain asserted rather than externally validated.
  • Constitutional Marketplace
    purpose: Platform for publishing, discovering, forking constitutions; envisioned as an ecosystem for value sharing.
    Only a prototype concept (Section 4.1); no working marketplace is evaluated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Personalized Constitutionally-Aligned Agentic Superego: Secure AI Behavior Aligned to Diverse Human Values." pith.science (2026). https://pith.science/paper/MBIIOPUF

@misc{pith2026250613774,
  author       = {Pith},
  title        = {Pith review of: Personalized Constitutionally-Aligned Agentic Superego: Secure AI Behavior Aligned to Diverse Human Values},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MBIIOPUF}},
  note         = {Machine review of arXiv:2506.13774}
}
read the original abstract

Agentic AI systems, possessing capabilities for autonomous planning and action, show great potential across diverse domains. However, their practical deployment is hindered by challenges in aligning their behavior with varied human values, complex safety requirements, and specific compliance needs. Existing alignment methodologies often falter when faced with the complex task of providing personalized context without inducing confabulation or operational inefficiencies. This paper introduces a novel solution: a 'superego' agent, designed as a personalized oversight mechanism for agentic AI. This system dynamically steers AI planning by referencing user-selected 'Creed Constitutions' encapsulating diverse rule sets -- with adjustable adherence levels to fit non-negotiable values. A real-time compliance enforcer validates plans against these constitutions and a universal ethical floor before execution. We present a functional system, including a demonstration interface with a prototypical constitution-sharing portal, and successful integration with third-party models via the Model Context Protocol (MCP). Comprehensive benchmark evaluations (HarmBench, AgentHarm) demonstrate that our Superego agent dramatically reduces harmful outputs -- achieving up to a 98.3% harm score reduction and near-perfect refusal rates (e.g., 100% with Claude Sonnet 4 on AgentHarm's harmful set) for leading LLMs like Gemini 2.5 Flash and GPT-4o. This approach substantially simplifies personalized AI alignment, rendering agentic systems more reliably attuned to individual and cultural contexts, while also enabling substantial safety improvements. An overview on this research with examples is available at https://superego.creed.space.

Figures

Figures reproduced from arXiv: 2506.13774 by the authors.

Figure 1
Figure 1. Conceptual Architecture of the Personalized Superego Agent Framework [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Data flow diagram for the Multi-Phase Superego Architecture. 4. Key Features and Capabilities The Personalized Superego Agent framework translates the abstract concept of adaptable, user-driven AI alignment into tangible practice through several key features and demonstrated capabilities. These components are designed to make personalized alignment more accessible, shareable, and directly usable within existing and … view at source ↗
Figure 3
Figure 3. Data flow diagram for the Superego Model Context Protocol Architecture. 4.4. Demonstration Prototype (Creed Space) and Core Benefits The functionality and potential of the Personalized Superego Agent framework have been showcased through an interactive prototype, made accessible at www.Creed.Space. This demonstration platform allows users to engage directly with the core concepts of our system. Users can select from… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Prototypical Creed.Space Interface. 4.5. Implementation Choices and Agentic Framework Considerations The practical realization of the Personalized Superego Agent framework, particularly our prototype, involved specific choices regarding the underlying agentic framework…
Figure 5
Figure 5. Figure 5: A qualitative comparison of the raw, basic inner model response to a request (right) versus a significantly more sophisticated superego response (left). 5.4. Benchmarks and Quantitative Evaluation To quantitatively assess the effectiveness of the Personalized Superego …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

50 extracted references · 31 canonical work pages

  1. [1]

    Artificial Intelligence, Values, and Alignment

    Gabriel, I. Artificial Intelligence, Values, and Alignment. Minds Mach. 2020, 30, 411–437

    work page 2020

  2. [2]

    Artificial Morality: Top -down, Bottom-up, and Hybrid Approaches

    Allen, C.; Smit, I.; Wallach, W. Artificial Morality: Top -down, Bottom-up, and Hybrid Approaches. Ethics Inf. Technol. 2005, 7, 149–155

    work page 2005

  3. [3]

    Translating Principles into Practices of Digital Ethics: Five Risks of Being Unethical

    Floridi, L. Translating Principles into Practices of Digital Ethics: Five Risks of Being Unethical. Philos. Technol. 2019, 32, 185–193

    work page 2019

  4. [4]

    Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback

    Casper, S.; Davies, X.; Shi, C.; Gilbert, T.K.; Scheurer, J.; Rando, J.; Freedman, R.; Korbak, T.; Lindner, D.; Freire, P.; e t al. Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback. arXiv 2023, https://arxiv.org/abs/2307.15217

    arXiv 2023

  5. [5]

    Personalized Large Language Models

    Woźniak, S.; Koptyra, B.; Janz, A.; Kazienko, P.; Kocoń, J. Personalized Large Language Models. arXiv 2024, https://arxiv.org/abs/2402.09269

    arXiv 2024

  6. [6]

    Towards an End -to-End Personal Fine -Tuning Framework for AI Value Alignment

    Watson, E.; Viana, T.; Sturgeon, B.; Petersson, L.; Zhang, S. Towards an End -to-End Personal Fine -Tuning Framework for AI Value Alignment. Electronics 2024, 13, 4044

    work page 2024

  7. [7]

    Safer Agentic AI

    Watson, N.; Hessami, A. Safer Agentic AI. SaferAgenticAI.org. Available online: https://www.saferagenticai.org (accessed on 18 July 2025)

    work page 2025

  8. [8]

    Introducing the Model Context Protocol

    Anthropic. Introducing the Model Context Protocol. Available online: https://www.anthropic.com/news/model -context- protocol (accessed on 28 May 2025)

    work page 2025

Show all 50 references
  1. [9]

    Deep Reinforcement Learning from Human Preferences

    Christiano, P.; Leike, J.; Brown, T.B.; Martic, M.; Legg, S.; Amodei, D. Deep Reinforcement Learning from Human Preferences. Adv. Neural Inf. Process. Syst. 2017, 30, 1–9

    2017

  2. [10]

    Confabulation: The Surprising Value of Large Language Model Hallucinations

    Sui, P.; Duede, E.; Wu, S.; So, R.J. Confabulation: The Surprising Value of Large Language Model Hallucinations. arXiv 2024, https://arxiv.org/abs/2406.04175

    2024 arXiv

  3. [11]

    Choice Vectors: Streamlining Personal AI Alignment Through Binary Selection

    Watson, E.; Nguyen, M.; Pan, S.; Zhang, S. Choice Vectors: Streamlining Personal AI Alignment Through Binary Selection. Multimodal Technol. Interact. 2025, 9, 22

    2025

  4. [12]

    Freud, S. (1923). The Ego and the Id. In J. Strachey (Ed. & Translator), The Standard Edition of the Complete Psychological Works of Sigmund Freud; The Ego and the Id and Other Works (Vol. XIX, pp. 1–66). London: Hogarth Press and the Institute of Psycho-Analysis, 1953–1974

    1923

  5. [13]

    Universality of Representation in Biologic al and Artificial Neural Networks

    Hosseini, E.; Casto, C.; Zaslavsky, N.; Conwell, C.; Richardson, M.; Fedorenko, E. Universality of Representation in Biologic al and Artificial Neural Networks. bioRxiv 2024, https://doi.org/10.1101/2024.12.26.629294

    2024 doi

  6. [14]

    The neural bases of cognitive conflict and control in mo ral judgment

    Greene, J.D.; Nystrom, L.E.; Engell, A.D.; Darley, J.M.; Cohen, J.D. The neural bases of cognitive conflict and control in mo ral judgment. Neuron 2004, 44, 389–400

    2004

  7. [15]

    The neural basis of human social values: Evidence from functional MRI

    Zahn, R.; Moll, J.; Paiva, M.; Garrido, G.; Krueger, F.; Huey, E.D.; Grafman, J. The neural basis of human social values: Evidence from functional MRI. Cereb. Cortex 2009, 19, 276–283

    2009

  8. [16]

    A Cognitive Theory of Consciousness: The Workspace of the Mind; Cambridge University Press: Cambridge, UK, 1988

    Baars, B.J. A Cognitive Theory of Consciousness: The Workspace of the Mind; Cambridge University Press: Cambridge, UK, 1988

    1988

  9. [17]

    Unified Theories of Cognition; Harvard University Press: Cambridge, MA, USA, 1990

    Newell, A. Unified Theories of Cognition; Harvard University Press: Cambridge, MA, USA, 1990

    1990

  10. [18]

    Revealing economic facts: LLMs know more than they say

    Buckmann, M.; Nguyen, Q.A.; Hill, E. Revealing economic facts: LLMs know more than they say. arXiv 2025, https://arxiv.org/abs/2505.08662

    2025

  11. [19]

    Zeng, W.; Kurniawan, D.; Mullins, R.; Liu, Y.; Saha, T.; Ike -Njoku, D.; Gu, J.; Song, Y.; Xu, C.; Zhou, J. et al. ShieldGemma 2: Robust and tractable image content moderation. arXiv 2025, https://arxiv.org/bs/2504.01081

    2025 arXiv

  12. [20]

    Superego -Agent LGDemo (Branch: Fastapi_Mcp)

    Superego GitHub. Superego -Agent LGDemo (Branch: Fastapi_Mcp). GitHub, 2025. Available online: https://github.com/Superego-Agent/superego-lgdemo/tree/fastapi_mcp (accessed on 16 July 2025)

    2025

  13. [21]

    HarmBench: A standardized evaluation framework for automated red teaming and robust refusal

    Mazeika, M.; Phan, L.; Yin, X.; Zou, A.; Wang, Z.; Mu, N.; Sakhaee, E.; Li, N.; Basart, S.; Li, B.; et al. HarmBench: A standardized evaluation framework for automated red teaming and robust refusal. arXiv 2024, https://doi.org/10.48550/arXiv.2402.04249

    2024 doi arXiv

  14. [22]

    AgentHarm: A benchmark for measuring harmfulness of LLM agents

    Andriushchenko, M.; Souly, A.; Dziemian, M.; Duenas, D.; Lin, M.; Wang, J.; Hendrycks, D.; Zou, A.; Kolter, Z.; Fredrikson, M.; et al. AgentHarm: A benchmark for measuring harmfulness of LLM agents. In Proceedings of the International Conference on Learning Representations (IC...

    2025

  15. [23]

    Do the rewards justify the means? Measuring trade -offs between rewards and ethical behavior in the Machiavelli benchmark

    Pan, A.; Chan, J.S.; Zou, A.; Li, N.; Basart, S.; Woodside, T.; Ng, J.; Zhang, H.; Emmons, S.; Hendrycks, D. Do the rewards justify the means? Measuring trade -offs between rewards and ethical behavior in the Machiavelli benchmark. In Proceedings of the International Conferenc...

    2023

  16. [24]

    Who validates the validators? Aligning LLM-assisted evaluation of LLM outputs with human preferences

    Shankar, S.; Zamfirescu -Pereira, J.D.; Hartmann, B.; Parameswaran, A.G.; Arawjo, I. Who validates the validators? Aligning LLM-assisted evaluation of LLM outputs with human preferences. arXiv 2024, https://doi.org/10.48550/arXiv.2404.12272

    2024 doi arXiv

  17. [25]

    Vijil Test Library: Evaluating LLM Trustworthiness Across Eight Dimensions

    Vijil, Inc. Vijil Test Library: Evaluating LLM Trustworthiness Across Eight Dimensions. Available online: https://docs.vijil.ai/tests-library/index.html (accessed on 20 May 2025). Information 2025, 16, x FOR PEER REVIEW 44 of 45

    2025

  18. [26]

    INSPECT: An Extensible Toolkit for AI Behavior Evaluation

    AI Safety Institute. INSPECT: An Extensible Toolkit for AI Behavior Evaluation. Available online: https://inspect.aisi.org.uk (accessed on 20 May 2025)

    2025

  19. [27]

    Governance in Agentic Workflows: Leveraging LLMs as Oversight Agents

    Nasim, I. Governance in Agentic Workflows: Leveraging LLMs as Oversight Agents. OpenReview, 2025. Available online: https://openreview.net/forum?id=fP02TFDJh8 (accessed on 18 July 2025)

    2025

  20. [28]

    Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements

    Zhang, J.; Elgohary, A.; Magooda, A.; Khashabi, D.; Van Durme, B. Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements. arXiv 2024, https://arxiv.org/abs/2410.08968

    2024 arXiv

  21. [29]

    Disentangled Safety Adapters Enable Efficient Guardrails and Flexible Inference-Time Alignment

    Krishna, K.; Cheng, J.Y.; Maalouf, C.; Gatys, L.A. Disentangled Safety Adapters Enable Efficient Guardrails and Flexible Inference-Time Alignment. arXiv 2025, https://arxiv.org/abs/2506.00166

    2025 arXiv

  22. [30]

    InferAligner: Inference -Time Alignment for Harmlessness through Cross-Model Guidance

    Wang, P.; Zhang, D.; Li, L.; Tan, C.; Wang, X.; Ren, K.; Jiang, B.; Qiu, X. InferAligner: Inference -Time Alignment for Harmlessness through Cross-Model Guidance. arXiv 2024, https://arxiv.org/abs/2401.11206

    2024 arXiv

  23. [31]

    Almost Surely Safe Alignment of Large Language Models at Inference-Time

    Ji, X.; Ramesh, S.S.; Zimmer, M.; Bogunovic, I.; Wang, J.; Bou Ammar, H. Almost Surely Safe Alignment of Large Language Models at Inference-Time. arXiv 2025, https://arxiv.org/abs/2502.01208

    2025 arXiv

  24. [32]

    Dynamic Search for Inference-Time Alignment in Diffusion Models (DSearch)

    Li, X.; Uehara, M.; Su, X.; Scalia, G.; Biancalani, T.; Regev, A.; Levine, S.; Ji, S. Dynamic Search for Inference-Time Alignment in Diffusion Models (DSearch). arXiv 2025, https://arxiv.org/abs/2503.02039

    2025 arXiv

  25. [33]

    Constit utional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming

    Sharma, M.; Tong, M.; Mu, J.; Wei, J.; Kruthoff, J.; Goodfriend, S.; Ong, E.; Peng, A.; Agarwal, R.; Anil, C.; et al. Constit utional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming. arXiv 2025, https://arxiv.org/abs/2501.18837

    2025 arXiv

  26. [34]

    Constitutional AI: Harmlessness from AI Feedback

    Bai, Y.; Kadavath, S.; Kundu, S.; Askell, A.; Kernion, J.; Jones, A.; Chen, A.; Goldie, A.; Mirhoseini, A.; McKinnon, C.; et al. Constitutional AI: Harmlessness from AI Feedback. arXiv 2022, https://arxiv.org/abs/2212.08073

    2022 arXiv

  27. [35]

    T raining a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback

    Bai, Y.; Jones, A.; Ndousse, K.; Askell, A.; Chen, A.; DasSarma, N.; Drain, D.; Fort, S.; Ganguli, D.; Henighan, T.; et al. T raining a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback. arXiv 2022, https://arxiv.org/abs/2204.05862

    2022 arXiv

  28. [36]

    Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation

    Baker, B.; Huizinga, J.; Madry, A.; Zaremba, W.; Pachocki, J.; Farhi, D. Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation. OpenAI, 2025. Available online: https://openai.com/index/chain -of-thought-monitoring (accessed on 12 March 2025)

    2025

  29. [37]

    AI Control: Improving Safety Despite Intentional Subversion

    Greenblatt, R.; Shlegeris, B.; Sachan, K.; Roger, F. AI Control: Improving Safety Despite Intentional Subversion. arXiv 2024, https://arxiv.org/abs/2312.06942

    2024 arXiv

  30. [38]

    OpenAI x DFT: The First Moral Graph

    Edelman, J.; Klingefjord, O. OpenAI x DFT: The First Moral Graph. Meaning Alignment Institute, 2023. Available online: https://meaningalignment.substack.com/p/the-first-moral-graph (accessed on 12 March 2025)

    2023

  31. [39]

    Model Integrity

    Edelman, J.; Klingefjord, O. Model Integrity. Meaning Alignment Institute, 2024. Available online: https://meaningalignment.substack.com/p/model-integrity (accessed on 18 July 2025)

    2024

  32. [40]

    The Global Landscape of AI Ethics Guidelines

    Jobin, A.; Ienca, M.; Vayena, E. The Global Landscape of AI Ethics Guidelines. Nat. Mach. Intell. 2019, 1, 389–399

    2019

  33. [41]

    WhatsApp MCP Exploited: Exfiltrating Your Message History via MCP

    Beurer-Kellner, L.; Fischer, M. WhatsApp MCP Exploited: Exfiltrating Your Message History via MCP. Invariant Labs Blog

  34. [42]

    MCP Security Notification: Tool Poisoning Attacks

    Beurer-Kellner, L.; Fischer, M. MCP Security Notification: Tool Poisoning Attacks. Invariant Labs Blog. 2025. Available online: https://invariantlabs.ai/blog/mcp-security-notification-tool-poisoning-attacks (accessed on 12 March 2025)

    2025

  35. [43]

    Emergent misalignment: Narrow finetuning can produce broadly misaligned LLMs

    Betley, J.; Tan, D.; Warncke, N.; Sztyber -Betley, A.; Bao, X.; Soto, M.; Labenz, N.; Evans, O. Emergent misalignment: Narrow finetuning can produce broadly misaligned LLMs. arXiv 2025, https://arxiv.org/abs/2502.17424

    2025

  36. [44]

    On Emergent Misalignment

    Mowshowitz, Z. On Emergent Misalignment. Don’t Worry About the Vase. 2025. Available online: https://thezvi.substack.com/p/on-emergent-misalignment (accessed on 26 January 2025)

    2025

  37. [45]

    Model Plurality

    Lu, C. Model Plurality. Combinations Magazine, 2024. Available online: https://www.combinationsmag.com/model-plurality/ (accessed on 26 January 2025)

    2024

  38. [46]

    Model Plurality: A Taxonomy for Pluralistic AI

    Lu, C.; Van Kleek, M. Model Plurality: A Taxonomy for Pluralistic AI. OpenReview, 2024. Available online: https://openreview.net/forum?id=kil2mabTqx (accessed on 18 July 2025)

    2024

  39. [47]

    Collaborating Action by Action: A Multi-agent LLM Framework for Embodied Reasoning

    White, I.; Nottingham, K.; Maniar, A.; Robinson, M.; Lillemark, H.; Maheshwari, M.; Qin, L.; Ammanabrolu, P. Collaborating Action by Action: A Multi-agent LLM Framework for Embodied Reasoning. arXiv 2025, https://www.arxiv.org/abs/2504.17950

    2025 arXiv

  40. [48]

    OASIS: Open Agent Social Interaction Simulations with One Million Agents

    Yang, Z.; Zhang, Z.; Zheng, Z.; Jiang, Y.; Gan, Z.; Wang, Z.; Ling, Z.; Chen, J.; Ma, M.; Dong, B.; et al. OASIS: Open Agent Social Interaction Simulations with One Million Agents. arXiv 2024, https://doi.org/10.48550/arXiv.2411.11581

    2024 doi arXiv

  41. [49]

    Project Sid: Many- agent simulations toward AI civilization

    Altera, A.L.; Ahn, A.; Becker, N.; Carroll, S.; Christie, N.; Cortes, M.; Demirci, A.; Du, M.; Li, F.; Luo, S.; et al. Project Sid: Many- agent simulations toward AI civilization. arXiv 2024, https://doi.org/10.48550/arXiv.2411.00114. Information 2025, 16, x FOR PEER REVIEW 45...

    2024 doi arXiv

  42. [2025]

    Available online: https://invariantlabs.ai/blog/whatsapp-mcp-exploited (accessed on 12 March 2025)

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.