{"id":"3f2ae6a4-1329-4d4b-bdb4-100def1277d7","arxiv_id":"2505.07759","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Current virtual assistants handle explicitly stated access control rules fairly well but often fail on inference-based and default-deny policy decisions, a pattern consistent across the three tested domains.","lead":"Researchers tested four AI assistants (ChatGPT, Gemini, Copilot, and DeepSeek) on how well they understand user-made access control rules for smart homes, smart cars, and medical records. They found the assistants frequently fail on questions that require inferring a default-deny rule, even though they mostly handle explicitly stated rules correctly.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The inference-failure claim is plausibly correct but not yet established: the contextual method's VA-generated U-MAPs and the lack of per-question scoring data leave the central accuracy finding confounded.","rationale":"The reader's CONDITIONAL verdict is reasonable, and my concern refines rather than replaces the weakest assumption. The authors' qualitative examples are suggestive, and the non-contextual condition is a better design; if the central claim were only that VAs are imperfect at applying U-MAPs in some phrasings, the evidence supports that. But the paper's stronger quantitative claim about consistent inference failure is not auditably supported: contextual sessions may conflate policy generation with policy evaluation, and aggregate percentages hide per-question variability. The claim could survive if the non-contextual transcripts show distinct inference failures; therefore the appropriate verdict remains CONDITIONAL, with the condition being clean per-question evaluation. No evidence of misconduct; this is a reproducibility and validity concern, not an integrity concern. I do not see grounds to move the verdict to REJECT or ACCEPT from the current text.","tokens_in":16561,"tokens_out":11830,"duration_ms":115110,"concrete_test":"Restrict the analysis to the non-contextual method: in fresh sessions, feed the exact U-MAPs from Tables 1, 3, and 4 verbatim, ask all five Table 2 questions per domain, and log raw answers plus transcripts. Have two independent annotators score each answer against the policy statement; report per-question accuracy for inference questions (Q-SM-5, Q-SC-5, Q-EHR-5) versus explicit-rule questions. If inference-question error rates are not materially higher than explicit-rule rates in this clean condition, the central claim 'largest and most consistent failure occurs on inference-based questions' is not supported by the reported data.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing weakness is in the evaluation protocol, not the selection of VAs. Sec 4.5.1 (Contextual Method) instructs the VA to 'generate a sample RBAC-like U-MAP, from instructions,' and then scores the answer to Table 2 questions 'against ground truth values' derived from the intended U-MAP. If the VA generates a different policy, a 'wrong' answer can be correct under the policy the VA is actually applying; the paper provides no transcripts or VA-generated policies to rule this out. The second, related weakness is that the headline 'largest and most consistent failure' on inference questions rests on aggregate Figure 2/3 percentages with no per-question breakdown, no confidence intervals, and no inter-rater agreement; it is possible the aggregate is driven by one or two idiosyncratic questions such as Q-SM-5 ('Can Homeowner access a Meter?'). The paper's own Sec 6 lists policy simplification and narrow perspectives but does not address these protocol confounds.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents an exploratory study of four commercial LLM-based virtual assistants (ChatGPT/GPT-4o, Google Gemini, Microsoft Copilot, and High-Flyer Deepseek) and their ability to interpret and apply user-managed access control policies (U-MAPs) in smart home, smart car, and electronic health record scenarios. The authors define several policy encodings (informal natural language, semi-formal unrolled statements, semi-formal rule-based, and a formal XACML subset) and test each VA under a contextual prompting protocol and a non-contextual protocol, using five access-request questions per domain. The main reported outcome is that the VAs handle explicitly stated rules reasonably well but consistently fail on inference-based questions that require applying a default-deny clause to resources or roles not named in the policy. The paper also reports format-dependent accuracy variation, discusses usability aspects such as response time and message limits, and offers recommendations for future VAs. The central qualitative claim—that inference and default-deny are hard for these systems—is plausible and worth investigating, but the quantitative accuracy figures are not backed by sufficient experimental detail.","tokens_in":16691,"tokens_out":9388,"duration_ms":86105,"significance":"If the central finding holds, it has practical security implications: using a general-purpose VA as an enforcement point for user-managed policies could allow unintended access, especially when policies rely on unstated denials. The paper is among the first to probe U-MAP comprehension in commercial VAs, and the selection of four assistants across three high-stakes domains is a reasonable starting point. Credit is due for publishing the actual policy encodings and question sets in Tables 1–4 and for including a sample per-question table in Appendix A. The qualitative observation in §5.4, such as the homeowner/meter example, is more convincing than the aggregate percentages in Figures 2 and 3. However, the significance is currently limited by the small, non-randomized question set, the absence of error bars or inter-rater agreement, and the contextual-protocol confound described below. The paper should be reframed as a qualitative, hypothesis-generating investigation, or the quantitative evaluation must be substantially strengthened.","major_comments":[{"comment":"The Contextual Method contains a direct confound that undermines the accuracy measurements. Step (3) instructs the VA to generate its own 'sample RBAC-like U-MAP' from instructions, and step (4) then scores the Table 2 questions against ground truth values derived from the intended U-MAP. If the VA generates a policy that differs from the intended one, a 'wrong' answer can be correct under the policy actually in force during that session. The paper provides no transcripts or VA-generated policies to rule out this possibility. Since the contextual condition contributes to all aggregate accuracy figures in Figures 2 and 3, and to the inference-failure claim in §5.4, this is a load-bearing methodological issue. The authors should either supply the generated policies and demonstrate that they match the intended U-MAPs, or restrict quantitative claims to the non-contextual condition where this confound does not arise.","section":"§4.5.1"},{"comment":"The different encodings of the supposedly same U-MAP are not semantically equivalent across formats, which invalidates the format-level accuracy comparisons. In the Smart Car domain, the Informal format (Table 3) states 'Only drivers should be allowed to give driving directions,' implying a default deny for non-drivers, while the Semi-Formal format states 'Everything else is Allowed' and the Semi-Formal-Rule-Based format states 'Default Radio & Directions & Settings = approved.' These are contradictory defaults, so a VA answering 'yes, kids can give directions' under the Semi-Formal or Rule-Based policy would be correct under that policy but marked wrong under the researchers' ground truth. Similar inconsistencies appear in the EHR encodings, where the Informal format restricts PII access to 'Admin Staff' but the Modified Semi-Formal-Rule-Based version grants PII access to 'Staff' and 'First Responders' without an explicit modification step. The paper must either make the formats semantically equivalent or treat each format as a distinct policy with its own ground truth; otherwise, the accuracy differences across formats in §5.3 and Figures 2–3 are uninterpretable.","section":"§4.3, Tables 1, 3, 4"},{"comment":"The quantitative claims rest on a very small sample that is not statistically characterized. Each domain has only five questions (Table 2), yet accuracy is reported to one or two decimal places with no confidence intervals, no error bars, and no inter-rater agreement calculation for the three researchers who conducted the sessions. With n=5 per condition, a single changed response shifts accuracy by 20 percentage points, so the differences described in §5.3 (e.g., 66.67% vs. 100% on a format) are not interpretable. Moreover, Table 5 provides per-question marks only for Smart Homes, and the caption does not define what the '×' marker means; no equivalent per-question data are supplied for Smart Cars or EHRs. The paper should provide the full per-question response matrix for all domains, a clear scoring legend, and appropriate uncertainty measures, or it should explicitly label all percentages as illustrative and drop the comparative claims.","section":"§4.4, §5.1, Figures 2–3, Table 5"},{"comment":"There are internal inconsistencies in the reported numbers that need correction. Section 5.1 states that ChatGPT's lowest accuracy is 80% in the Smart Car domain, but the Smart Car panel of Figure 2 shows a 60% value in the ChatGPT row. Section 5.2's averages (e.g., 93.34%, 96.11%) are not derivable from the per-domain values in Figures 2–3 because the aggregation formula is unspecified. In addition, §4.3.2 states that the Formal XACML format 'was not used directly in the procedures,' yet Figures 2 and 3 include bars labeled 'Formal (w/ Context)' and 'Formal (w/o Context).' The authors should correct the numbers, specify exactly how the averages are computed, and clarify which formats the figure legend actually refers to.","section":"§5.1, §4.3.2, Figure 2"},{"comment":"The scoring rule for inference-based questions is not specified precisely enough to support the central claim. The paper states that responses were 'recorded as true or false' and that correctness was determined by comparing the VA's response to 'the explicit policy statement,' but it does not say how 'cannot determine' responses were counted, nor does it report inter-rater reliability for the more subjective cases such as Q-SM-5 ('Can Homeowner access a Meter?'). Since the default-deny inference is the paper's headline finding, the authors should define the full scoring rubric, state how indeterminate answers were handled, and provide the raw question-by-question labels for all VAs and domains.","section":"§4.4, §5.4"}],"minor_comments":[{"comment":"The phrase 'By conducting unstructured to structured tests' is unclear; the study actually uses two prompting methods (contextual and non-contextual) across several policy formats, so the wording should be aligned with the methodology.","section":"Abstract"},{"comment":"In the research questions, 'V As' appears with an extra space; it should read 'VAs' for consistency with the rest of the paper.","section":"§1, RQ1"},{"comment":"Several references have garbled author names, e.g., references [6] and [7] begin with 'Chung, David Ferraiolo, and David Kuhn' without a clear first author, and reference [7] omits the year and conference details. These need to be cleaned up before publication.","section":"References"},{"comment":"The claim that EHR-domain accuracy is higher because 'the widespread use of access control within the healthcare field' provides training data is presented without evidence; it is speculative and should be labeled as a hypothesis rather than a conclusion.","section":"§5.6.2"},{"comment":"The table is titled 'Sample Results' and uses an '×' marker without a legend; even after a legend is added, the table should indicate whether blank cells are correct answers, unasked questions, or missing data.","section":"Table 5"}],"recommendation":"major_revision","confidential_remarks":"This is a borderline case. The topic is timely and the qualitative observation about default-deny being difficult for VAs is worth publishing after revision, but I would not accept the current quantitative framing. The inconsistent policy encodings in Tables 3–4 and the contextual-method confound are load-bearing and need to be resolved before any accuracy percentages are reported. If the authors choose not to expand the evaluation, the paper should be reframed as a purely qualitative, hypothesis-generating study."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is the first study I know that puts general-purpose LLM assistants through a user-managed access-control-policy task across smart home, smart car, and EHR domains. The core observation—that VAs are far better at retrieving explicit rules than at inferring a denied default for an unmentioned resource—is plausible and worth knowing. The 'Can the homeowner access a meter?' example captures it well. The paper gets credit for a clean task design: multiple policy formats (natural language, semi-formal, rule-based), two prompting methods, and four popular VAs. They also list some honest limitations in Sec. 6.\n\nThe soft spots are real, and they matter more than the story the abstract tells. First, the numbers are internally sloppy: the text claims ChatGPT accuracy ranges 91.67–96.67% and then says its lowest is 80% in the smart-car domain. The figures include 'Formal (w/ Context)' conditions even though Sec. 4.3.2 says the XACML formal format was not used directly in the procedures. That sort of mismatch needs fixing before I'd trust any of the percentages. Second, the contextual method has a genuine confound: the VA is asked to generate a sample RBAC-like U-MAP from instructions, and then its answers are scored against the authors' intended ground truth. If the VA produces a different policy, a 'wrong' answer can be right under the policy it actually holds. The paper doesn't provide transcripts or the generated policies, so you can't tell how much of the contextual-method error comes from this. The non-contextual method—where the policy is handed over directly—avoids the issue, so the inference-failure result may well survive. But the aggregate figures mix both methods, and that's a problem. Third, five questions per domain is a thin reed for the claim that inference is the 'largest and most consistent' failure; no confidence intervals, no per-question breakdown, no inter-rater check on the researchers' scoring.\n\nWho's it for? People working on LLM safety and usable security will want to read it as an exploratory marker, not as a benchmark. It deserves a serious peer review because the question is timely and the study is a reasonable first cut. A careful referee should ask for per-question data, session logs for the contextual condition, and a cleaned-up set of figures before publication. I'd take it under major revision, not desk-reject it.","headline":"A useful first pass at a real problem, but the numbers outrun the evidence and one protocol confound needs unpacking before you can trust the headline claim.","tokens_in":17256,"tokens_out":4466,"would_cite":false,"duration_ms":44313,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Current AI assistants cannot reliably enforce user-written access-control policies, the study argues: they answer direct questions well but fail exactly when a policy's deny-by-default clause must be applied to an unstated resource or role.","keywords":["user-managed access control policies","virtual assistants","large language models","access control evaluation","default-deny inference","smart home security","electronic health records","RBAC"],"falsifier":"Run the inference battery at scale on the current versions of the same assistants: build 50 to 100 access-request questions per domain whose correct answer requires applying a deny-by-default catch-all to a resource or role the policy never names, and score them against ground truth. If accuracy on those implicit-denial questions exceeds roughly 95%, or if asking the assistant to cite the rule that justified a denial consistently yields correct rule references, the paper's claim of a systematic inference failure would be refuted. A cheaper spot check: re-ask the smart-home 'homeowner and the meter' question in the Informal format on all four assistants today and compare the answers with the failures recorded in the paper.","tokens_in":16335,"feed_emoji":"🛡️","tokens_out":11194,"duration_ms":90684,"temperature":0.7,"pith_summary":"An exploratory study of four publicly available AI assistants — ChatGPT (GPT-4o), Google Gemini, Microsoft Copilot, and Deepseek — asks whether they can take a user-written access-control policy and answer simple yes/no questions about who may use each device or record. The paper's central claim is that they cannot yet do this reliably: across smart-home, smart-car, and electronic-health-record scenarios, the assistants answered most direct questions correctly but systematically failed when a correct answer required inference, most notably applying a policy's 'everything else is denied' catch-all to a resource the policy never names. The authors argue this matters because these assistants are already marketed as natural-language controllers for smart locks, cars, and patient records, so a wrong inference is not a quiz error but a potential security or privacy breach. The study also finds that accuracy depends on how a policy is phrased and improves modestly in longer sessions, and it recommends integrating reasoning capabilities, domain-specific pre-training, and real-time adaptability into future assistants. Read sympathetically, the paper establishes that today's general-purpose assistants grasp the letter of a policy but not its deny-by-default logic.","feed_headline":"Four AI assistants all fail the same access-policy test","feed_subtitle":"None could reliably apply a policy's 'everything else is denied' rule to resources the policy never names.","key_machinery":"The load-bearing objects are the U-MAP encodings and the test protocol built around them. Each U-MAP is a role-based rule set derived from a subset of XACML with a single First-Applicable combining algorithm, and each is presented in three formats — Informal natural language, Semi-Formal enumerated statements, and Semi-Formal-Rule-Based code-like rules — plus 'modified' variants that change rules mid-session to test adaptation to dynamic updates. The decisive test element is the default-deny catch-all, 'everything else is denied,' paired with questions about unmentioned resources such as the meter in the smart-home policy, because answering requires the assistant to infer that an unnamed item falls under the catch-all. The protocol contrasts a Contextual method, which primes the assistant with an explanation of RBAC and domain security considerations, against a Non-Contextual cold-start method, and every answer is scored against researcher-established ground truth. Accuracy on the five questions per domain, aggregated across formats and sessions, is the metric that carries the argument.","core_discovery":"Stated on the paper's own terms, the discovery is that the failure point in AI-managed access control is not retrieval but entailment. With ground-truth answers defined by role-based policies adapted from XACML, the four assistants scored from roughly 60% to 100% accuracy depending on assistant, domain, policy format, and whether the session supplied context; the smart-car domain produced the largest drops. The most consistent failure was the inference question: when a policy said 'everything else is denied' and a request named an unlisted resource (for example, whether a homeowner may access a meter), one or more assistants answered affirmatively, declared themselves unable to determine, or apologized rather than concluding 'denied.' Longer sessions that restated and modified the rules lifted every assistant's average accuracy, and structured semi-formal formats slightly outperformed natural-language prose, while code-like rule formats showed the widest variance. The paper concludes that, at the time of testing, these models lacked the reasoning framework needed to enforce deny-by-default semantics, and that deploying them as policy enforcement points would introduce vulnerabilities that stem from inference failure rather than from user error.","pith_inferences":["My inference: the measured failure rates are likely a lower bound on deployment risk, because the test policies were short, single-combining-rule sets with no corner cases — a limitation the paper itself concedes — and realistic policies with multi-rule ordering and overlapping conditions would multiply the number of implicit-denial decisions.","My inference: the paper's design can be compressed into a lasting regression test for model releases, since the handful of 'unstated resource under default-deny' questions costs almost nothing to re-run each time an assistant updates and would give vendors a concrete signal on whether deny-by-default reasoning has improved.","My inference: the apology phrase that titles the paper suggests the models partly detect their own uncertainty; a testable extension is to check whether hedging or apologizing is actually predictive of a wrong answer, which would let a wrapper system escalate uncertain responses to a deterministic policy evaluator.","My inference: the same inference machinery will be stressed harder by attribute-based policies, which the paper names as future work, since conditions involving time, location, or relationship multiply the cases in which a resource is governed by negation or default; ABAC-style policies are a natural place to look for even larger failure gaps."],"forward_implications":["Deploying a general-purpose assistant as the enforcement point for a user-managed policy is unsafe for deny-by-default cases: a request about any resource or role the policy does not name has a measurable chance of being granted rather than denied, so security-sensitive devices such as smart locks should not rely on an assistant's interpretation alone.","Policy phrasing changes outcomes: semi-formal enumerated statements yielded slightly higher accuracy than informal natural language, and code-like rule formats produced the widest accuracy spread, so presenting rules as explicit enumerated statements is the safer way to hand a policy to an assistant.","Restating and updating rules during a session improves adherence across all four assistants, implying that interactive re-affirmation of a policy helps, though the improvements did not bring any assistant to a level suitable for security-critical access decisions.","Assistant behavior tracks how well documented a domain is in training data: the electronic-health-records domain, where access control is extensively documented, produced the highest scores, while smart cars, with scarce security documentation, produced the lowest, so domain-specific pre-training material is a direct lever on policy-handling accuracy.","Adding an explicit reasoning step improves answer quality but costs usability, as Deepseek's roughly 14-second average response time demonstrates, so the paper treats reasoning integration as a trade-off against response time rather than a free win."],"supporting_citations":[{"why":"Supplies the XACML subset and First-Applicable rule semantics from which the formal U-MAP encodings are derived.","marker":"[26]"},{"why":"Provides the role-based access control model that shapes the roles and conditions used in every U-MAP.","marker":"[22]"},{"why":"Grounds the U-MAP construct itself as policies whose lifecycle is handled by untrained end-users.","marker":"[23]"},{"why":"Motivates the smart-home domain and establishes why access-control errors there carry physical security stakes.","marker":"[10]"},{"why":"Frames the 'as an AI language model, I cannot' denial behavior that the paper observes and that titles the study.","marker":"[28]"},{"why":"Is the source for Deepseek's Mixture-of-Experts reasoning architecture, invoked to explain that model's stronger structured reasoning and slower responses.","marker":"[25]"},{"why":"Is cited for ChatGPT's RLHF-based training, used to explain the structure and limitations of its responses.","marker":"[4]"}],"fun_headline_variants":["AI assistants flunk deny-by-default policy tests","Access-policy inference fails in all four AI assistants","Why AI assistants can't say 'denied' for unlisted resources","Entailment gap: AI assistants miss 'everything else is denied'","Smart-car policies trip up AI virtual assistants"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The accuracy numbers assume that five researcher-written questions per domain, applied to the specific policy wordings in the paper's tables, fairly represent how real users phrase policies and requests; without evidence that this tiny test set spans the variation found in actual deployments, the measured failure rates may not carry over to differently phrased policies.","fun_headline_variants_meta":{"raw":{"variants":["AI assistants flunk deny-by-default policy tests","Access-policy inference fails in all four AI assistants","Why AI assistants can't say 'denied' for unlisted resources","Entailment gap: AI assistants miss 'everything else is denied'","Smart-car policies trip up AI virtual assistants"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000198,"raw_usage":{"total_tokens":1390,"prompt_tokens":991,"completion_tokens":399,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":607,"completion_tokens_details":{"reasoning_tokens":318}},"tokens_in":607,"tokens_out":399,"duration_ms":3260,"temperature":1.0,"reasoning_tokens":318,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:08:07.667698+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the inference battery at scale on the current versions of the same assistants: build 50 to 100 access-request questions per domain whose correct answer requires applying a deny-by-default catch-all to a resource or role the policy never names, and score them against ground truth. If accuracy on those implicit-denial questions exceeds roughly 95%, or if asking the assistant to cite the rule that justified a denial consistently yields correct rule references, the paper's claim of a systematic inference failure would be refuted. A cheaper spot check: re-ask the smart-home 'homeowner and the meter' question in the Informal format on all four assistants today and compare the answers with the failures recorded in the paper.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the role-based access control model that shapes the roles and conditions used in every U-MAP."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the XACML subset and First-Applicable rule semantics from which the formal U-MAP encodings are derived."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Motivates the smart-home domain and establishes why access-control errors there carry physical security stakes."},{"cited_title":"As an AI language model, I cannot","cited_arxiv_id":null,"evidence_quote":"Frames the 'as an AI language model, I cannot' denial behavior that the paper observes and that titles the study."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Is cited for ChatGPT's RLHF-based training, used to explain the structure and limitations of its responses."}],"review_version":1}