pith. sign in

arxiv: 2607.01510 · v1 · pith:W5IVFAFSnew · submitted 2026-07-01 · 💻 cs.AI · cs.CR

Janus: a Playground for User-Involved Agentic Permission Management

Pith reviewed 2026-07-03 20:04 UTC · model grok-4.3

classification 💻 cs.AI cs.CR
keywords AI agentspermission managementuser involvementprivacy and securitypermission fatigueagentic systemsevaluation frameworksynthetic responders
0
0 comments X

The pith

User input strengthens privacy in AI agent permissions but no design works best everywhere

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces Janus as a modular playground for testing how users can participate in managing permissions when AI agents act autonomously on their behalf. Through implementation of six different permission assistant designs and evaluation across multiple scenarios with synthetic responders, it establishes that direct user involvement can meaningfully improve privacy and security outcomes. The work also shows that AI can assist users to lower decision effort, yet designs must incorporate the reality of permission fatigue where repeated choices lead to poorer decisions. Because no single approach excels in every setting, the findings point toward context-aware permission systems rather than one-size-fits-all solutions.

Core claim

Janus demonstrates that user input is critical and can significantly strengthen privacy and security in agentic permission management, that AI augmentation of user decisions can help reduce cognitive load, and that realistic user behavior including permission fatigue must be accounted for in system design. No single design performs optimally across all contexts, motivating a more principled and context-sensitive approach to deploying permission assistants in agentic systems.

What carries the argument

Janus, a playground system with Janus-Core for implementing diverse permission management designs and Janus-Harness for automated evaluation, grounded in a conceptual model of key design axes for user involvement.

If this is right

  • User input is critical and can significantly strengthen privacy and security.
  • AI augmentation of user decisions can help reduce cognitive load.
  • Realistic user behavior including permission fatigue must be accounted for in system design.
  • No single design performs optimally across all contexts.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Future agentic systems could benefit from runtime selection among permission designs depending on detected context and user state.
  • Findings based on synthetic responders indicate that real-user validation studies are a necessary next step before deployment.
  • The modular structure of the playground enables systematic testing of additional designs and scenarios beyond those evaluated.

Load-bearing premise

The three synthetic responders used in evaluations sufficiently model realistic human user behavior, including permission fatigue, across the tested scenarios.

What would settle it

A controlled study with actual human participants performing permission decisions in the same scenarios that produces fatigue effects or security outcomes markedly different from those generated by the synthetic responders.

Figures

Figures reproduced from arXiv: 2607.01510 by Eugene Bagdasarian, Franziska Roesner, Natalie Grace Brigham, Tadayoshi Kohno.

Figure 1
Figure 1. Figure 1: Flowchart of permission management in our conceptual model, [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: High level overview of the Janus playground, consisting of Janus-Core (detailed in Section 4) and Janus-Harness (detailed in Section 5). [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Hull plot of permission assistants constructed from synthetic [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Hulls for permission assistants formed by runs with the [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Grid of dual-axis plots of outcomes versus risk tolerance across [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Hulls for permission assistants formed by runs with the [PITH_FULL_IMAGE:figures/full_fig_p016_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Hulls for permission assistants formed by runs with the [PITH_FULL_IMAGE:figures/full_fig_p016_7.png] view at source ↗
read the original abstract

AI agents that autonomously execute tool calls on a user's behalf raise pressing questions about permission management: what role could users play, and what role should they play? Despite many proposed approaches, the user's role in agentic permission management remains under explored. We introduce Janus, a playground system for implementing and evaluating user-involved agentic permission management designs. Janus consists of two components: Janus-Core, a modular agentic system supporting a diverse spectrum of permission management designs, and Janus-Harness, an automated evaluation framework. Grounded in a conceptual model that identifies key design axes for user involvement, we implement six permission assistants spanning the design space and evaluate them across three scenarios and three synthetic responders. We demonstrate that user input is critical and can significantly strengthen privacy and security, that AI augmentation of user decisions can help reduce cognitive load, and that realistic user behavior including permission fatigue must be accounted for in system design. No single design performs optimally across all contexts, motivating a more principled and context-sensitive approach to deploying permission assistants in agentic systems. Janus is publicly available to support future investigation into this dimension of agentic system design.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper introduces Janus, a modular playground consisting of Janus-Core (for implementing permission management designs in agentic systems) and Janus-Harness (for automated evaluation). Grounded in a conceptual model of design axes for user involvement, the authors implement six permission assistants spanning the space and evaluate them in three scenarios using three synthetic responders. They conclude that user input strengthens privacy/security, AI augmentation reduces cognitive load, permission fatigue must be modeled, and no single design is optimal across contexts.

Significance. If the evaluations are robust, the work supplies a publicly available testbed for systematic exploration of user roles in agentic permission management, a timely topic as autonomous tool-calling agents proliferate. The finding that context-sensitive designs are needed, together with the open-source release, could accelerate principled follow-on research.

major comments (2)
  1. [Evaluation] Evaluation (abstract and §4–5): The central claims about realistic user behavior and permission fatigue rest on three synthetic responders whose construction, parameterization, and validation against empirical human data on decision fatigue or variability are not described. Without such grounding, demonstrations that particular designs better handle fatigue or that context-sensitive approaches are required lose empirical support.
  2. [§3] §3 (conceptual model) and evaluation design: The six implemented designs are said to span the design space, yet the mapping from the identified axes to the concrete assistants is not shown in sufficient detail to allow readers to assess coverage or to reproduce the spectrum.
minor comments (1)
  1. [Abstract] Abstract: the phrase 'synthetic responders' is introduced without a forward reference to the section that defines them.

Simulated Author's Rebuttal

2 responses · 1 unresolved

We thank the referee for their constructive comments, which help clarify the scope and presentation of our work. We address each major comment below and outline the revisions we will make.

read point-by-point responses
  1. Referee: [Evaluation] Evaluation (abstract and §4–5): The central claims about realistic user behavior and permission fatigue rest on three synthetic responders whose construction, parameterization, and validation against empirical human data on decision fatigue or variability are not described. Without such grounding, demonstrations that particular designs better handle fatigue or that context-sensitive approaches are required lose empirical support.

    Authors: We acknowledge the need for greater transparency on the synthetic responders. In the revision we will expand §4 with a full description of their construction and parameterization, including the exact decision rules, fatigue thresholds, and variability parameters for each of the three responders. We will also add a limitations subsection clarifying that these responders are illustrative constructs designed to exercise the evaluation harness rather than validated models of human behavior. The central demonstration remains that the harness can surface differences across designs when fatigue-like behavior is present; we will adjust the abstract and §5 to avoid implying empirical validation of the responders themselves. revision: partial

  2. Referee: [§3] §3 (conceptual model) and evaluation design: The six implemented designs are said to span the design space, yet the mapping from the identified axes to the concrete assistants is not shown in sufficient detail to allow readers to assess coverage or to reproduce the spectrum.

    Authors: We agree that an explicit mapping is required for reproducibility and assessment of coverage. We will insert a new table in §3 that lists each of the six permission assistants together with the precise values assigned to every design axis (user involvement level, AI augmentation type, context sensitivity, etc.). This table will directly link the conceptual model to the implemented assistants and will be referenced in the evaluation design section. revision: yes

standing simulated objections not resolved
  • Empirical validation of the synthetic responders against human data on decision fatigue or behavioral variability

Circularity Check

0 steps flagged

No circularity; system-building paper with no derivations or fitted predictions

full rationale

The paper introduces an implemented playground system (Janus-Core and Janus-Harness) and evaluates six permission assistant designs across scenarios using synthetic responders. No equations, first-principles derivations, parameter fitting, or predictions appear in the abstract or described content. Claims about user input, AI augmentation, and permission fatigue rest on the described implementations and evaluations rather than any reduction to inputs by construction. No self-citation chains, ansatzes, or renamings of known results are load-bearing. This matches the default expectation for non-circular system papers.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claims rest on a conceptual model of design axes for user involvement and the assumption that synthetic responders capture key real-world behaviors; no free parameters or invented entities are described.

axioms (1)
  • domain assumption Synthetic responders accurately model realistic user behavior including permission fatigue
    Evaluations across three scenarios rely on three synthetic responders to stand in for human users.

pith-pipeline@v0.9.1-grok · 5734 in / 1211 out tokens · 22028 ms · 2026-07-03T20:04:43.811774+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

47 extracted references · 14 canonical work pages · 6 internal anchors

  1. [1]

    Levels of Autonomy for AI Agents,

    K. Feng, D. W. McDonald, and A. X. Zhang, “Levels of Autonomy for AI Agents,” Jul. 2025. [Online]. Available: https: //knightcolumbia.org/content/levels-of-autonomy-for-ai-agents-1

  2. [2]

    Design Patterns for Securing LLM Agents against Prompt Injections

    L. Beurer-Kellner, B. Buesser, A.-M. Cret ¸u, E. Debenedetti, D. Dobos, D. Fabian, M. Fischer, D. Froelicher, K. Grosse, D. Naeff, E. Ozoani, A. Paverd, F. Tram `er, and V . V olhejn, “Design Patterns for Securing LLM Agents against Prompt Injections,” 2025. [Online]. Available: https://arxiv.org/abs/2506.08837

  3. [3]

    AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents,

    E. Debenedetti, J. Zhang, M. Balunovic, L. Beurer-Kellner, M. Fischer, and F. Tram `er, “AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents,” inNeurIPS, 2024. [Online]. Available: https://openreview.net/forum? id=m1YY AQjO3w

  4. [4]

    ACM Transactions on Information Systems 43, 1–55

    L. Huang, W. Yu, W. Ma, W. Zhong, Z. Feng, H. Wang, Q. Chen, W. Peng, X. Feng, B. Qin, and T. Liu, “A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions,”ACM Trans. Inf. Syst., vol. 43, no. 2, Jan. 2025. [Online]. Available: https://doi.org/10.1145/3703155

  5. [5]

    Dissecting Human and LLM Preferences,

    J. Li, F. Zhou, S. Sun, Y . Zhang, H. Zhao, and P. Liu, “Dissecting Human and LLM Preferences,” inACL, 2024. [Online]. Available: https://aclanthology.org/2024.acl-long.99/

  6. [6]

    The protection of information in com- puter systems,

    J. Saltzer and M. Schroeder, “The protection of information in com- puter systems,”Proceedings of the IEEE, vol. 63, no. 9, pp. 1278– 1308, 1975

  7. [7]

    AI Agents May Always Fall for Prompt Injections

    S. Abdelnabi and E. Bagdasarian, “AI Agents May Always Fall for Prompt Injections,”arXiv preprint arXiv:2605.17634, 2026

  8. [8]

    Privacy reasoning in ambiguous contexts,

    R. Yi, O. Suciu, A. Gascon, S. Meiklejohn, E. Bagdasarian, and M. Gruteser, “Privacy reasoning in ambiguous contexts,”NeurIPS, 2026

  9. [9]

    You’ve Been Warned: An Empirical Study of the Effectiveness of Web Browser Phishing Warnings,

    S. Egelman, L. F. Cranor, and J. Hong, “You’ve Been Warned: An Empirical Study of the Effectiveness of Web Browser Phishing Warnings,” inSIGCHI Conference on Human Factors in Computing Systems, 2008

  10. [10]

    Progent: Securing AI Agents with Privilege Control

    T. Shi, J. He, Z. Wang, H. Li, L. Wu, W. Guo, and D. Song, “Progent: Programmable Privilege Control for LLM Agents,” arXiv:2504.11703, 2025

  11. [11]

    Defeating Prompt In- jections by Design,

    E. Debenedetti, I. Shumailov, T. Fan, J. Hayes, N. Carlini, D. Fabian, C. Kern, C. Shi, A. Terzis, and F. Tram `er, “Defeating Prompt In- jections by Design,” inIEEE Conference on Secure and Trustworthy Machine Learning (SaTML), 2026

  12. [12]

    Reframing LLM Agent Security as an Agent-Human Interaction Problem

    P. Wang, Y . Li, and Y . Tian, “Reframing LLM Agent Security as an Agent-Human Interaction Problem,” arXiv:2605.24309, 2026

  13. [13]

    ReAct: Synergizing Reasoning and Acting in Language Models,

    S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y . Cao, “ReAct: Synergizing Reasoning and Acting in Language Models,” in ICLR, 2023

  14. [14]

    Not What You’ve Signed Up For: Compromising Real- World LLM-Integrated Applications with Indirect Prompt Injection,

    K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz, “Not What You’ve Signed Up For: Compromising Real- World LLM-Integrated Applications with Indirect Prompt Injection,” inACM Workshop on Artificial Intelligence and Security, 2023

  15. [15]

    AirGapAgent: Protecting Privacy- Conscious Conversational Agents,

    E. Bagdasarian, R. Yi, S. Ghalebikesabi, P. Kairouz, M. Gruteser, S. Oh, B. Balle, and D. Ramage, “AirGapAgent: Protecting Privacy- Conscious Conversational Agents,” inCCS, 2024

  16. [16]

    Christodorescu et al.Systems Security Foundations for Agentic Computing

    M. Christodorescu, E. Fernandes, A. Hooda, S. Jha, J. Rehberger, K. Chaudhuri, X. Fu, K. Shams, G. Amir, J. Choi, S. Choudhary, N. Palumbo, A. Labunets, and N. V . Pandya, “Systems Security Foundations for Agentic Computing,” arXiv:2512.01295, 2026

  17. [17]

    Secure and Efficient Access Control Framework for Computer-Use Agents via Context Space,

    H. Gong, C. Li, R. Chang, and W. Shen, “Secure and Efficient Access Control Framework for Computer-Use Agents via Context Space,” arXiv:2509.22256, 2026

  18. [18]

    ACE: A Security Architecture for LLM-Integrated App Systems,

    E. Li, T. Mallick, E. Rose, W. Robertson, A. Oprea, and C. Nita- Rotaru, “ACE: A Security Architecture for LLM-Integrated App Systems,” inNDSS, 2026

  19. [19]

    Formal Policy Enforcement for Real-World Agentic Systems

    N. Palumbo, S. Choudhary, J. Choi, G. Amir, P. Chalasani, and S. Jha, “Formal Policy Enforcement for Real-World Agentic Sys- tems,” arXiv:2602.16708, 2026

  20. [20]

    CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents

    H. Foerster, T. Blanchard, K. Nikoli ´c, I. Shumailov, C. Zhang, R. Mullins, N. Papernot, F. Tram `er, and Y . Zhao, “CaMeLs Can Use Computers Too: System-level Security for Computer Use Agent,” arXiv:2601.09923, 2026

  21. [21]

    When Bots Take the Bait: Exposing and Mitigating the Emerging Social Engineering Attack in Web Automation Agent,

    X. Wu, G. Hong, Y . Chen, M. Liu, F. Jin, X. Pan, J. Dai, and B. Liu, “When Bots Take the Bait: Exposing and Mitigating the Emerging Social Engineering Attack in Web Automation Agent,” arXiv:2601.07263, 2026

  22. [22]

    IronCurtain: A Personal AI Assistant Built Se- cure from the Ground Up,

    N. Provos, “IronCurtain: A Personal AI Assistant Built Se- cure from the Ground Up,” Feb. 2026, https://www.provos.org/p/ ironcurtain-secure-personal-assistant/

  23. [23]

    Contextual Agent Security: A Policy for Every Purpose,

    L. Tsai and E. Bagdasarian, “Contextual Agent Security: A Policy for Every Purpose,” inHotOS, 2025

  24. [24]

    Privacy as Contextual Integrity,

    H. Nissenbaum, “Privacy as Contextual Integrity,”Washington Law Review, vol. 79, no. 1, pp. 119–157, 2004

  25. [25]

    Towards Automating Data Access Permissions in AI Agents,

    Y . Wu, K. Yang, F. Roesner, T. Kohno, N. Zhang, and U. Iqbal, “Towards Automating Data Access Permissions in AI Agents,” in IEEE Symposium on Security & Privacy, 2026

  26. [26]

    IsolateGPT: An Execution Isolation Architecture for LLM-Based Systems,

    Y . Wu, F. Roesner, T. Kohno, N. Zhang, and U. Iqbal, “IsolateGPT: An Execution Isolation Architecture for LLM-Based Systems,” in Network and Distributed System Security Symposium, 2025

  27. [27]

    AC4A: Access Control for Agents,

    R. K. Sharma and D. Grossman, “AC4A: Access Control for Agents,” arXiv:2603.20933, 2026

  28. [28]

    A Systematic Review of Access Control Models: Back- ground, Existing Research, and Challenges,

    N. Farhadighalati, L. A. Estrada-Jimenez, S. Nikghadam-Hojjati, and J. Barata, “A Systematic Review of Access Control Models: Back- ground, Existing Research, and Challenges,”IEEE Access, vol. 13, 2025

  29. [29]

    Protection,

    B. W. Lampson, “Protection,”SIGOPS Operating Systems Review, Jan. 1974

  30. [30]

    Programming semantics for multiprogrammed computations,

    J. B. Dennis and E. C. Van Horn, “Programming semantics for multiprogrammed computations,”Commun. ACM, vol. 9, no. 3, p. 143–155, Mar. 1966. [Online]. Available: https: //doi.org/10.1145/365230.365252

  31. [31]

    User-Driven Access Control: Rethinking Permission Granting in Modern Operating Systems,

    F. Roesner, T. Kohno, A. Moshchuk, B. Parno, H. J. Wang, and C. Cowan, “User-Driven Access Control: Rethinking Permission Granting in Modern Operating Systems,” inIEEE Symposium on Security and Privacy, 2012

  32. [32]

    Android Permissions Demystified,

    A. P. Felt, E. Chin, S. Hanna, D. Song, and D. Wagner, “Android Permissions Demystified,” inACM Conference on Computer and Communications Security, 2011

  33. [33]

    Android Permissions: User Attention, Comprehension, and Behav- ior,

    A. P. Felt, E. Ha, S. Egelman, A. Haney, E. Chin, and D. Wagner, “Android Permissions: User Attention, Comprehension, and Behav- ior,” inSymposium on Usable Privacy and Security, 2012

  34. [34]

    ‘I do (not) need that Feature!’ – Understanding Users’ Awareness and Control of Privacy Permissions on Android Smartphones,

    S. Prange, P. Knierim, G. Knoll, F. Dietz, A. D. Luca, and F. Alt, “‘I do (not) need that Feature!’ – Understanding Users’ Awareness and Control of Privacy Permissions on Android Smartphones,” in Symposium on Usable Privacy and Security, 2024

  35. [35]

    Do Windows Users Follow the Principle of Least Privilege? Investigating User Account Control Practices,

    S. Motiee, K. Hawkey, and K. Beznosov, “Do Windows Users Follow the Principle of Least Privilege? Investigating User Account Control Practices,” inSymposium on Usable Privacy and Security, 2010

  36. [36]

    Evolution of Android’s Permission-based Security Model and Challenges,

    R. K. Solanki, V . Laxmi, and M. S. Gaur, “Evolution of Android’s Permission-based Security Model and Challenges,” arXiv:2601.00252, 2026

  37. [37]

    The best of both worlds: Mitigating trade-offs between accuracy and user burden in capturing mobile app privacy preferences,

    D. Smullen, Y . Feng, S. A. Zhang, and N. Sadeh, “The best of both worlds: Mitigating trade-offs between accuracy and user burden in capturing mobile app privacy preferences,”Proceedings on Privacy Enhancing Technologies, vol. 2020, no. 1, 2020

  38. [38]

    Follow My Recommenda- tions: A Personalized Privacy Assistant for Mobile App Permissions,

    B. Liu, M. S. Andersen, F. Schaub, H. Almuhimedi, S. A. Zhang, N. Sadeh, Y . Agarwal, and A. Acquisti, “Follow My Recommenda- tions: A Personalized Privacy Assistant for Mobile App Permissions,” inSymposium on Usable Privacy and Security, 2016

  39. [39]

    Rethink web permissions: Seamless user control of powerful capabilities with Chrome’s new proposed<permission>element,

    M. Le, G. Cocchi, D. Renzulli, M. Viana, and T. Steiner, “Rethink web permissions: Seamless user control of powerful capabilities with Chrome’s new proposed<permission>element,” Jun. 2025, https: //developer.chrome.com/blog/rethinking-web-permissions

  40. [40]

    Rethinking Access Control and Authentication for the Home Internet of Things (IoT),

    W. He, M. Golla, R. Padhi, J. Ofek, M. D ¨urmuth, E. Fernandes, and B. Ur, “Rethinking Access Control and Authentication for the Home Internet of Things (IoT),” inUSENIX Security Symposium, 2018

  41. [41]

    User Comprehension and Comfort with Eye-Tracking and Hand-Tracking Permissions in Augmented Reality,

    K. Cheng, M. Sim, T. Kohno, and F. Roesner, “User Comprehension and Comfort with Eye-Tracking and Hand-Tracking Permissions in Augmented Reality,” inSymposium on Usable Security and Privacy (USEC), 2025

  42. [42]

    World-Driven Access Control for Continuous Sensing,

    F. Roesner, T. Kohno, and D. Molnar, “World-Driven Access Control for Continuous Sensing,” inACM Conference on Computer and Communications Security (CCS), 2014

  43. [43]

    Sketch-based Access Control: A Multimodal Interface for Translating User Preferences into Intent-Aligned Policies

    K. Monteiro and S. Das, “Sketch-based Access Control: A Multi- modal Interface for Translating User Preferences into Intent-Aligned Policies,” arXiv:2605.10012, 2026

  44. [44]

    Introducing ChatGPT agent: bridging research and action,

    OpenAI, “Introducing ChatGPT agent: bridging research and action,” Jul. 2025. [Online]. Available: https://openai.com/index/ introducing-chatgpt-agent/

  45. [45]

    Configure auto mode

    Anthropic, “Configure auto mode.” [Online]. Available: https: //code.claude.com/docs/en/auto-mode-config

  46. [46]

    Users are not the enemy,

    A. Adams and M. A. Sasse, “Users are not the enemy,”Communica- tions of the ACM, vol. 42, no. 12, pp. 40–46, 1999

  47. [47]

    AI agents are coming for your pri- vacy, warns Meredith Whittaker,

    M. Whittaker, “AI agents are coming for your pri- vacy, warns Meredith Whittaker,” Sep. 2025. [On- line]. Available: https://www.economist.com/by-invitation/2025/09/ 09/ai-agents-are-coming-for-your-privacy-warns-meredith-whittaker Appendix Additional Figures Figures 6 and 7 are versions of Figure 4 from Section 6.3 for thealways noandalways yessynthetic ...