pith. sign in

arxiv: 2605.28647 · v1 · pith:GFAEJL5Dnew · submitted 2026-05-27 · 💻 cs.AI · cs.CY· q-fin.RM

The Ethics of LLM Sandbox and Persona Dynamics

Tim Gebbie , Stewart Gebbie This is my paper

Pith reviewed 2026-06-29 12:39 UTC · model grok-4.3

classification 💻 cs.AI cs.CYq-fin.RM
keywords LLM ethicsguardrailsreality gapepistemic riskpersona dynamicsreality launderingmoral complianceAI safety
0
0 comments X

The pith

LLM guardrails and persona dynamics create reality gaps that shift epistemic risk to users, making them unethical.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that guardrails and trained persona dynamics in LLMs produce a reality gap between the world the model is permitted to describe and the world in which users must act. It claims actively generating these gaps is unethical because it knowingly shifts epistemic risk back to uninformed users, a process labeled reality laundering that can cause harm at scale. This risk is sharpest in high-exposure advice contexts where users seek orientation rather than externally checkable tasks. The authors distinguish refusing harm from refusing reality and advocate top-down causal requirements at the task level instead of bottom-up moral correction at the response level. Parallels to financial regulation failures illustrate how formal safety systems can become performative while real exposure migrates elsewhere.

Core claim

Actively generating reality gaps is in fact unethical because it knowingly shifts epistemic risk back to the uninformed user—this is reality laundering. Guardrails naively appear ethically necessary when they claim to prevent direct harm, but often become suspect when they suppress truthful perception and launder uncomfortable mechanisms into acceptable abstractions. The conclusion is that so-called ethical AI becomes substantively unethical when it substitutes institutional reassurance for contact with reality.

What carries the argument

Reality laundering: the shift of epistemic risk to users via guardrails and persona dynamics that create a distance between permitted model descriptions and actionable reality.

If this is right

  • Potential harm occurs when reality gaps are operationalised at scale in high-exposure advice contexts.
  • Guardrails that suppress truthful perception become ethically suspect even if they prevent direct harm.
  • Moral compliance produces safe language while allowing distorted reality to persist.
  • Top-down causal requirements specification at the task level is required rather than bottom-up moral correction at the response or sandbox level.
  • Persona dynamics shape how uncertainty, conflict, authority, and risk are staged for the user.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the argument holds, explicit task-level constraints could be tested against response-level guardrails to measure differences in user decision accuracy.
  • The pattern of performative safety systems may appear in other AI interfaces such as content filters or recommendation engines.
  • Designers could add user-facing disclosures about the boundaries of permitted model reality to reduce hidden risk transfer.
  • Regulatory approaches to AI might shift emphasis from harm avoidance metrics to requirements for maintaining contact with external reality.

Load-bearing premise

That guardrails and persona dynamics are the primary cause of distorted outputs rather than limitations in the underlying model training or user prompting choices.

What would settle it

A controlled comparison showing that users given unguardrailed model outputs in advice contexts experience equivalent or greater epistemic risk and harm than users of guardrailed versions would undermine the claim that guardrails are the main driver of unethical risk shifting.

read the original abstract

It is well known that LLM guardrails and trained persona dynamics can produce a reality gap: the distance between the world a LLM is permitted or shaped to describe, and the world in which users must act. Here we argue that actively generating reality gaps is in fact unethical because it knowingly shifts epistemic risk back to the uninformed user -- this is reality laundering. This can potentially cause harm when operationalised at scale. The risk is sharpest in high-exposure advice contexts, where users seek orientation rather than a bounded, externally checkable task. Guardrails naively appear ethically necessary when they claim to prevent direct harm, but often become suspect when they suppress truthful perception and launder uncomfortable mechanisms into acceptable abstractions. Basel-style financial regulation, B-BBEE-style compliance, Societe Generale, and the London Whale show how formal safety systems can become legible, gameable, and performative while real exposure migrates elsewhere. The same pattern can appear in LLMs as moral compliance: safe language, distorted reality. We therefore distinguish refusing harm, from refusing reality; and then argue for top-down causal requirements specification at the task level rather than bottom-up moral correction at the response or sandbox level. Persona dynamics matter because the assistant interface is not neutral; it shapes how uncertainty, conflict, authority, and risk are staged. The conclusion is that so-called ``ethical AI'' becomes substantively unethical when it substitutes institutional reassurance for contact with reality.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 1 minor

Summary. The manuscript argues that LLM guardrails and persona dynamics generate 'reality gaps'—distances between permitted LLM descriptions and the actual world—constituting 'reality laundering' by shifting epistemic risk to uninformed users. This is claimed to be unethical, particularly in high-exposure advice contexts, drawing analogies to financial regulation failures like Basel and the London Whale. The paper distinguishes refusing harm from refusing reality and advocates for top-down causal requirements specification at the task level rather than bottom-up moral corrections.

Significance. If substantiated, the argument would challenge prevailing approaches to 'ethical AI' by suggesting that guardrails can become performative and counterproductive. It highlights the non-neutrality of persona dynamics in shaping user perception of uncertainty and risk. However, the significance is limited by the absence of empirical data, formal analysis, or testing of the central normative claim.

major comments (3)
  1. [Abstract] Abstract, paragraph 1: The assertion that guardrails 'can produce' reality gaps and are the site of 'moral compliance' and 'reality laundering' treats them as the primary cause without arguing against alternatives such as base model training limitations or user prompting choices; the skeptic note correctly identifies this as load-bearing for the ethical diagnosis of 'active' laundering.
  2. [Abstract] Abstract: The central normative claim—that actively generating reality gaps is unethical because it shifts epistemic risk—is presented via the definition of 'reality laundering' without independent ethical grounding, counterexample analysis, or examination of whether gaps persist independently of sandbox/persona choices.
  3. [Abstract] Abstract: The analogies to financial regulation (Basel-style, B-BBEE, Societe Generale, London Whale) illustrate how safety systems become gameable, but the manuscript does not specify the concrete mapping to LLM contexts or provide evidence that real exposure migrates in AI systems in the same manner.
minor comments (1)
  1. [Abstract] Abstract: The term 'reality laundering' is introduced as an invented entity without reference to prior literature on epistemic risk or similar concepts in AI ethics.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the careful reading and for isolating the load-bearing assumptions in the abstract. The manuscript is a conceptual and normative argument rather than an empirical study; we respond to each major comment below and indicate where revisions will strengthen clarity without altering the core thesis.

read point-by-point responses

  1. Referee: [Abstract] Abstract, paragraph 1: The assertion that guardrails 'can produce' reality gaps and are the site of 'moral compliance' and 'reality laundering' treats them as the primary cause without arguing against alternatives such as base model training limitations or user prompting choices; the skeptic note correctly identifies this as load-bearing for the ethical diagnosis of 'active' laundering.

    Authors: The manuscript does not assert that guardrails are the sole or primary source of all reality gaps. It isolates guardrails and persona dynamics as a controllable institutional mechanism that actively maintains and launders gaps by substituting permissible language for accurate description. Base-model limitations and user prompting are orthogonal contributors; the ethical claim targets the additional risk transfer introduced when deployers impose sandbox constraints that users cannot easily override. We will revise the abstract to state explicitly that other sources of misalignment exist while sandbox choices remain ethically salient because they are under direct institutional control and presented as harm-prevention measures. revision: partial

  2. Referee: [Abstract] Abstract: The central normative claim—that actively generating reality gaps is unethical because it shifts epistemic risk—is presented via the definition of 'reality laundering' without independent ethical grounding, counterexample analysis, or examination of whether gaps persist independently of sandbox/persona choices.

    Authors: The normative grounding is the principle that an agent with superior epistemic position should not knowingly distort a dependent party's access to reality in order to reduce its own exposure, especially when the distortion is framed as protective. This principle is independent of whether some gaps would exist absent guardrails; the active, scalable generation and maintenance of gaps through persona dynamics constitutes the laundering. The distinction between refusing harm and refusing reality already functions as counterexample analysis. We will add a concise statement of this grounding principle to the abstract and introduction and note that the argument applies to the incremental distortion introduced by sandbox design even if baseline gaps remain. revision: partial

  3. Referee: [Abstract] Abstract: The analogies to financial regulation (Basel-style, B-BBEE, Societe Generale, London Whale) illustrate how safety systems become gameable, but the manuscript does not specify the concrete mapping to LLM contexts or provide evidence that real exposure migrates in AI systems in the same manner.

    Authors: We accept that the analogies require a more explicit mapping. In revision we will insert a dedicated paragraph (referenced from the abstract) that maps each example: Basel-style requirements parallel safety benchmarks that certify 'safe' outputs while actual high-risk advice migrates to unfiltered channels or downstream user decisions; the London Whale case parallels how legible compliance (refusal of certain queries) allows epistemic risk to accumulate in user orientation tasks. The mapping remains illustrative of the structural pattern rather than empirical proof, consistent with the paper's conceptual scope. revision: yes

Circularity Check

0 steps flagged

No circularity: ethical argument rests on independent conceptual distinction and external analogies

full rationale

The paper's central move defines a 'reality gap' as the distance between permitted LLM descriptions and the user's world, then argues that actively generating such gaps is unethical because it shifts epistemic risk to uninformed users, naming the result 'reality laundering.' This does not reduce the ethical conclusion to the definition by construction; the 'because' clause supplies an independent grounding (epistemic risk transfer) rather than deriving the judgment from the term itself. No equations, fitted parameters, self-citations, or uniqueness theorems appear. The argument distinguishes 'refusing harm' from 'refusing reality' and invokes external regulatory examples (Basel, B-BBEE, Societe Generale) as support, keeping the derivation self-contained against the listed circularity patterns.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 1 invented entities

The argument rests on several untested assumptions about how guardrails operate and what counts as ethical risk transfer; no independent evidence is supplied for these premises.

axioms (2)
  • ad hoc to paper Guardrails and persona dynamics systematically suppress truthful perception
    Stated in abstract as a mechanism that produces reality gaps.
  • domain assumption Shifting epistemic risk to uninformed users constitutes unethical behavior
    Core normative premise used to label the practice as reality laundering.
invented entities (1)
  • reality laundering no independent evidence
    purpose: Label for the process by which guardrails create and hide reality gaps
    New term introduced to frame the ethical claim; no independent falsifiable test provided.

pith-pipeline@v0.9.1-grok · 5784 in / 1364 out tokens · 29684 ms · 2026-06-29T12:39:41.124278+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

21 extracted references · 19 canonical work pages

  1. [1]

    Basel Committee on Banking Supervision,

    URL:https://link.springer.com/article/10.1007/s00766-022-00375-7, doi:10.1007/ s00766-022-00375-7. Basel Committee on Banking Supervision,

    work page doi:10.1007/s00766-022-00375-7

  2. [2]

    Nature Medicine URL: https://www.nature.com/articles/s41591-025-04074-y, doi:10.1038/s41591-025-04074-y

    Reliability of large language models as medical assistants for the general public: a randomized preregistered study. Nature Medicine URL: https://www.nature.com/articles/s41591-025-04074-y, doi:10.1038/s41591-025-04074-y. Campbell, D.T.,

    work page doi:10.1038/s41591-025-04074-y

  3. [3]

    Working Paper 34255

    How People Use ChatGPT. Working Paper 34255. National Bureau of Economic Research. URL: https://www.nber.org/papers/w34255, doi:10.3386/w34255. Cui, J., Chiang, W.L., Stoica, I., Hsieh, C.J.,

    work page doi:10.3386/w34255

  4. [4]

    arXiv preprint arXiv:2405.20947 , year=

    OR-Bench: An over-refusal benchmark for large language models. URL:https://arxiv.org/abs/2405.20947, doi:10.48550/arXiv.2405.20947, arXiv:2405.20947. Department of Trade, Industry and Competition,

    work page doi:10.48550/arxiv.2405.20947

  5. [5]

    Hiddendetect: Detecting jailbreak attacks against large vision-language models via monitoring hidden states,

    Causal models in requirement specifications for ma- chine learning: A vision. URL:https://arxiv.org/abs/2502.11629, doi:10.48550/arXiv.2502. 11629,arXiv:2502.11629. Judicial Commission of Inquiry into Allegations of State Capture,

    work page doi:10.48550/arxiv.2502

  6. [6]

    Epistemic injustice in generative ai, in: Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, pp. 684–697. URL:https://ojs.aaai.org/ index.php/AIES/article/view/31671, doi:10.1609/aies.v7i1.31671. Lee, J.D., See, K.A.,

    work page doi:10.1609/aies.v7i1.31671

  7. [7]

    Human Factors 46, 50–80

    Trust in automation: Designing for appropriate reliance. Human Factors 46, 50–80. doi:10.1518/hfes.46.1.50.30392. Lerman, K., Chu, M.D., Bickham, C., Luceri, L., Ferrara, E.,

    work page doi:10.1518/hfes.46.1.50.30392

  8. [8]

    EPJ Data Science URL:https://link.springer.com/article/10.1140/epjds/s13688-025-00575-5, doi:10.1140/ epjds/s13688-025-00575-5

    Safe spaces or toxic places? content moderation and social dynamics of online eating disorder communities. EPJ Data Science URL:https://link.springer.com/article/10.1140/epjds/s13688-025-00575-5, doi:10.1140/ epjds/s13688-025-00575-5. Lindström, A.D., Methnani, L., Krause, L., Ericson, P., de Rituerto de Troya, Í.M., Mollo, D.C., Dobbe, R.,

    work page doi:10.1140/epjds/s13688-025-00575-5

  9. [9]

    Lu, C., Gallagher, J., Michala, J., Fish, K., Lindsey, J.,

    URL:https:// link.springer.com/article/10.1007/s10676-025-09837-2, doi:10.1007/s10676-025-09837-2. Lu, C., Gallagher, J., Michala, J., Fish, K., Lindsey, J.,

    work page doi:10.1007/s10676-025-09837-2

  10. [10]

    arXiv:2601.10387 [cs]

    The assistant axis: Situating and stabilizing the default persona of language models. URL:https://arxiv.org/abs/2601.10387, doi:10.48550/arXiv.2601.10387,arXiv:2601.10387. MacKenzie, D.A.,

    work page doi:10.48550/arxiv.2601.10387

  11. [11]

    (2016).Undone Science: Social movements, mobilized publics, and industrial transitions

    An Engine, Not a Camera: How Financial Models Shape Markets. MIT Press, Cambridge, MA. doi:10.7551/mitpress/9780262134606.001.0001. Miragoli, M.,

    work page doi:10.7551/mitpress/9780262134606.001.0001

  12. [12]

    Conformism, ignorance & injustice: Ai as a tool of epistemic oppression. Episteme URL:https://www.cambridge.org/core/journals/episteme/ article/conformism-ignorance-injustice-ai-as-a-tool-of-epistemic-oppression/ 26846FDAEE26CD81C85EB18480851A1F, doi:10.1017/epi.2024.11. Montz, A.L.,

    work page doi:10.1017/epi.2024.11 2024

  13. [13]

    Humans and Automation: Use, Misuse, Disuse, Abuse,

    Humans and automation: Use, misuse, disuse, abuse. Human Factors 39, 230–253. doi:10.1518/001872097778543886. 7 Peter, S., Riemer, K., West, J.D.,

    work page doi:10.1518/001872097778543886

  14. [14]

    Proceedings of the National Academy of Sciences 122, e2415898122

    The benefits and dangers of anthropomorphic conversational agents. Proceedings of the National Academy of Sciences 122, e2415898122. URL:https://pmc. ncbi.nlm.nih.gov/articles/PMC12146756/, doi:10.1073/pnas.2415898122. Polakow, D., Gebbie, T., Flint, E.,

    work page doi:10.1073/pnas.2415898122

  15. [15]

    Journal of Economic Methodology , 1–26URL:https://www.tandfonline.com/doi/ full/10.1080/1350178X.2026.2670997, doi:10.1080/1350178X.2026.2670997

    Where models fail: causality and self-reference in financial economics. Journal of Economic Methodology , 1–26URL:https://www.tandfonline.com/doi/ full/10.1080/1350178X.2026.2670997, doi:10.1080/1350178X.2026.2670997. Rahsepar Meadi, M., Sillekens, T., Metselaar, S., van Balkom, A., Bernstein, J., Batelaan, N.,

    work page doi:10.1080/1350178x.2026.2670997 2026

  16. [16]

    JMIR Mental Health 12, e60432

    Exploring the ethical challenges of conversational ai in mental health care: Scoping review. JMIR Mental Health 12, e60432. URL:https://mental.jmir.org/2025/1/e60432, doi:10.2196/60432. Republic of South Africa,

    work page doi:10.2196/60432 2025

  17. [17]

    Techni- cal Report

    Mission Green: Summary Report. Techni- cal Report. Societe Generale. URL:https://www.societegenerale.com/sites/default/files/ 12-may-2008-the-report-by-the-general-inspection-of-societe-generale.pdf. Stephenson, N.,

    2008

  18. [18]

    Technical Re- port

    JPMorgan Chase Whale Trades: A Case History of Derivatives Risks and Abuses. Technical Re- port. United States Senate Committee on Homeland Security and Governmental Affairs. URL:https://www.hsgac.senate.gov/subcommittees/investigations/library/files/ report-jpmorgan-chase-whale-trades-a-case-history-of-derivatives-risks-and-abuses-march-15-2013/. Wei, R....

    2013

  19. [19]

    Journal of Systems and Software 213, 112034

    ACCESS: Assurance case centric engineering of safety-critical systems. Journal of Systems and Software 213, 112034. URL:https://doi.org/10.1016/j.jss.2024.112034, doi:10.1016/j. jss.2024.112034. Wilcox, D., Gebbie, T.,

    work page doi:10.1016/j.jss.2024.112034 2024

  20. [20]

    URL:https://arxiv.org/ abs/1408.5585, doi:10.48550/arXiv.1408.5585,arXiv:1408.5585

    Hierarchical causality in financial economics. URL:https://arxiv.org/ abs/1408.5585, doi:10.48550/arXiv.1408.5585,arXiv:1408.5585. Zeissler, A.G., Metrick, A.,

    work page doi:10.48550/arxiv.1408.5585

  21. [21]

    Journal of Financial Crises 1, 75–91

    Jpmorgan chase london whale c: Risk limits, metrics, and models. Journal of Financial Crises 1, 75–91. URL:https://elischolar.library.yale.edu/ journal-of-financial-crises/vol1/iss2/4/, doi:10.17132/2693-3179.1016. 8

    work page doi:10.17132/2693-3179.1016