{"id":"947f47a7-256a-4e8d-9904-8c247cf4ebe7","arxiv_id":"2607.05277","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":5,"one_line_summary":"UCM masks untrusted webpage content at the DOM level before it reaches a web agent and routes content access through a type-constrained quarantined model, achieving 0% attack success rate against prompt injection while preserving task utility.","lead":"This paper presents a defense called Untrusted Content Masking (UCM) that hides untrusted webpage content from AI web agents using DOM-level redaction, replacing it with labeled placeholders. A smart generalist might read it because it offers a practical, deployable architecture for giving web agents security guarantees against prompt injection without crippling their ability to complete tasks.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"The security guarantee is binary in labeling correctness: one missed untrusted element fully exposes the Agent. Automated boundary inference (F1 as low as 0.84) does not close this gap, and the paper never measures ASR under automatically inferred labels.","rationale":"The reader correctly identified the most load-bearing concern: the security guarantee is conditional on perfect DOM labeling, which the automated boundary inference does not achieve (F1 0.84–0.997). This is the single point where the central claim is most fragile — one missed untrusted element fully exposes the Agent to injection. The paper is transparent about this requirement and tests single-element mislabeling sensitivity (Appendix E), but it does not measure end-to-end ASR under automatically inferred labels, which is the key missing experiment. The data-flow attack surface (Section 6.2, Appendix F) is real but is a known, acknowledged limitation of control-flow-protecting defenses, not a hidden flaw. The type-constraint mechanism itself is sound by construction: bool, int, float, enum, and date outputs cannot carry free-form instructions, and the Q-Model's reasoning is correctly withheld from the Agent (Appendix B.5). The string-output fallback (Appendix B.7) is gated on user approval, which is a weaker but clearly scoped mechanism. The CONDITIONAL verdict is appropriate: the guarantee holds under stated assumptions that may not always be met in practice, and the paper is honest about this boundary. No verdict adjustment is needed.","tokens_in":23288,"tokens_out":4111,"duration_ms":125921,"concrete_test":"Run the strengthened WASP attacks from Appendix E against UCM using the automatically inferred labels from Section 7 (specifically the GitLab configuration with F1=0.84, FN=8.5%) instead of hand-labeled ones. If the attack payload lands in any element group that the automated labeling missed, the ASR should rise above 0%. If ASR remains at 0% across all 12 attack goals, the automated labeling is sufficient in practice despite imperfect F1. If ASR rises measurably, the practical security guarantee for passive honest owners is weaker than claimed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central security claim — that the Agent never processes untrusted content — holds if and only if every untrusted DOM element is correctly labeled (Section 3: 'To provide strict security guarantees, UCM requires that each DOM element is correctly labeled'). This is a binary property: a single false negative in labeling places attacker-controlled text directly in the Agent's context window, collapsing the defense to the undefended regime or worse. The paper tests single-element mislabeling (Appendix E, ASR rises to 6±5%) and reports automated boundary inference F1 scores of 0.84–0.997 (Table 2, Section 7.2), with GitLab showing 8.5% false-negative rate at the element-group level. However, the paper never combines these two results: it does not measure ASR when UCM uses automatically inferred labels rather than hand-labeled ones. This is the deployment scenario that matters for passive honest owners. The gap between '0% ASR with perfect labels' and 'unknown ASR with automated labels' is where the practical security guarantee lives or dies. The paper is honest about this being a requirement, but the reader is correct that the guarantee is structurally conditional on a property the system's own automation does not achieve. The data-flow attacks (Section 6.2, Appendix F) are a real but secondary concern — they are acknowledged limitations of control-flow-protecting defenses, not a hole in the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"glm-5.2","summary":"The paper presents Untrusted Content Masking (UCM), a defense for web agents against prompt injection attacks. The core idea is to mask untrusted DOM elements before rendering the page to the agent, and to route any necessary access to untrusted content through a type-constrained Quarantined Model (Q-Model) that returns only structured responses (bool, int, float, enum, date). The approach is evaluated on 10 custom websites and the WebArena GitLab suite across three frontier agents, showing preserved task utility with 1.05x-1.84x cost overhead and 0% attack success rate against strengthened WASP attacks. The paper also presents an automated boundary inference method using an LLM on content-sanitized DOM to identify untrusted regions without manual labeling.","tokens_in":23489,"tokens_out":1287,"duration_ms":176723,"significance":"The paper addresses an important problem: extending provable prompt-injection defenses to web agents, where the structural entanglement of trusted and untrusted content on rendered pages has been a fundamental obstacle. The architectural insight—that DOM structure can separate trust boundaries without reading content—is clean and well-motivated. The type-constrained Q-Model design is sound by construction: well-typed outputs cannot carry free-form injected instructions. The 0% ASR result is a validation of a design property rather than an empirical prediction, which is appropriate. The automated boundary inference on sanitized DOM is a thoughtful addition that addresses deployment concerns. The reproducible code and falsifiable attack taxonomy (control-flow vs. data-flow) are strengths. The paper builds transparently on prior work (Jacob et al. 2025, Debenedetti et al. 2025, Costa et al. 2025) and clearly positions its contribution relative to these.","major_comments":[{"comment":"Section 3 (Threat Model) and Section 7.2 (Table 2): The paper's central security guarantee is explicitly conditional on correct labeling of every untrusted DOM element (Section 3: 'To provide strict security guarantees, UCM requires that each DOM element is correctly labeled'). The automated boundary inference results (Table 2) show F1 scores of 0.84-0.997, with GitLab having an 8.5% false-negative rate at the element-group level. The paper never measures ASR under automatically inferred labels—it only reports 0% ASR with hand-labeled boundaries (Appendix E, Table 7) and single-element mislabeling sensitivity (6±5% ASR). The gap between '0% ASR with perfect labels' and 'unknown ASR with automated labels' is the practically relevant deployment scenario for passive honest owners, and it is unmeasured. The paper should either (a) run the WASP attack evaluation with automatically inferred (非","section":null},{"comment":"Section 6.2 and Appendix F: The data-flow attack discussion is valuable and honest, but the threat model in Section 3 states the attacker's goal is to 'manipulate the Agent into malicious actions.' The selection-hijacking attacks (Appendix F, Figures 8a-8b) demonstrate that well-typed Q-Model outputs can be manipulated to steer the agent toward attacker-controlled repositories or users, which constitutes a malicious action under the paper's own threat model. The paper frames these as 'known limitations of control-flow-protecting defenses,' but the reader should not have to consult Appendix F to understand that the security guarantee covers control-flow integrity only, not data-flow integrity. The abstract and conclusion should state this scope explicitly. As written, the abstract's claim of 'security guarantees' may be read more broadly than the paper delivers.","section":null}],"minor_comments":[{"comment":"Section 5.1, Figure 2: The cost overhead ratios (1.05x-1.84x) are computed from medians, but the text does not mention this until later. Clarify in the figure caption or the results paragraph that medians are used for cost.","section":null},{"comment":"Section 7.2, Table 2: The F1 metric is computed at the element-group level (Section G.4), but this is not stated in the table caption. A reader may interpret 0.997 F1 as near-perfect element-level labeling. Clarify the granularity.","section":null},{"comment":"Appendix E, Table 7: The ASR for the undefended baseline (17±8%) is described as a 'lower bound on true vulnerability.' This is correct, but the large confidence interval (±8%) suggests a small sample size. Report the number of attack instances evaluated.","section":null},{"comment":"Section 3: The honest-owner assumption is justified by citing that 50% of internet traffic is concentrated in 3,000 domains. The citation (Xavier 2024) is an arXiv preprint; verify whether peer-reviewed alternatives exist.","section":null},{"comment":"Appendix B.3, Listing 1: The system prompt contains a note that enum options are capped at 10 and each element can be queried at most 5 times per task. These implementation limits are relevant to the security argument (they bound the Q-Model interaction surface) and should be mentioned in the main text, not only in the appendix.","section":null},{"comment":"Figure 1: The figure caption references 'product-description-2' but the figure shows 'product-d' (truncated). Fix the truncation or adjust the caption.","section":null},{"comment":"Section 5.2: The 'Q-Model string output allowed' condition requires user approval of each free-text answer. The paper does not report how often this fallback was invoked or the user burden it creates. A brief quantitative summary would strengthen the utility claim.","section":null}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about the labeling gap is well-founded and is the primary reason for the major revision recommendation. The paper is honest about the requirement for correct labels, but the disconnect between the 0% ASR claim (with perfect labels) and the automated inference results (with non-trivial false-negative rates) is a gap that needs to be closed empirically or at minimum discussed with an explicit ASR measurement under inferred labels. The data-flow scope issue is secondary but should be addressed in the abstract/conclusion for honesty about what 'security guarantees' covers."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for a careful and constructive review. Both major comments identify genuine gaps between what the paper claims and what it measures, and we agree that revisions are needed. Below we address each point in detail.","responses":[{"response":"The referee is correct that the practically relevant deployment scenario for passive honest owners—ASR under automatically inferred labels—is not measured in the current manuscript. We acknowledge this gap and will address it in revision. Specifically, we plan to run the strengthened WASP attack evaluation on the GitLab WebArena environment using automatically inferred trust boundaries (from the boundary inference pipeline of Section 7) rather than hand-labeled boundaries. This will directly measure ASR under the imperfect-labeling regime where the 8.5% false-negative rate applies. We expect the results to be informative regardless of outcome: if ASR remains low, this validates the automated pipeline for security-critical use; if ASR increases, this quantifies the practical security cost of automation and motivates further work on boundary inference accuracy. In either case, we agree the paper is incomplete without this measurement. We will also add a discussion of the relationship between false-negative rates and attack surface: undermasking exposes specific untrusted elements to the Agent, and the WASP attacks target issue descriptions, so the effective ASR depends on whether the specific attacked elements fall within the false-negative set. This is an empirical question that the proposed experiment will answer. We note that the single-element mislabeling sensitivity experiment (Appendix E, 6±5% ASR) provides a partial data point, but we agree it does not substitute for measuring ASR under the full automated-labeling pipeline.","revision_made":"yes","referee_comment":"Section 3 and Table 2: The gap between 0% ASR with perfect labels and unknown ASR with automated labels is unmeasured. The paper should run WASP attacks with automatically inferred labels or otherwise address this gap."},{"response":"We agree with this comment. The selection-hijacking attacks described in Appendix F do constitute malicious actions under the threat model as stated in Section 3 ('manipulate the Agent into malicious actions'), and the current framing—where the scope limitation to control-flow integrity is discussed in Section 6.2 and Appendix F but not in the abstract or conclusion—could lead readers to overestimate the breadth of the security guarantee. We will revise the abstract and conclusion to explicitly state that UCM provides control-flow integrity guarantees: it prevents prompt injection from hijacking the Agent's instructions or plan, but does not prevent data-flow attacks where an attacker corrupts the values returned by the Q-Model to mislead the Agent's decisions within an otherwise legitimate workflow. We will also add a brief forward reference to the data-flow discussion in Section 6.2 earlier in the paper (at the first mention of 'security guarantees' in the introduction), so that readers encounter the scope limitation before reaching the evaluation. We believe this is an honest and necessary clarification that strengthens the paper. We note that the data-flow limitation is shared by all control-flow-protecting defenses in this literature (CaMeL, FIDES, and the type-directed approach of Jacob et al.), and UCM's contribution—establishing trust boundaries in web environments—is a prerequisite for applying downstream data-flow defenses. We will make this positioning clearer in the revision.","revision_made":"yes","referee_comment":"Section 6.2 and Appendix F: The abstract and conclusion should state explicitly that the security guarantee covers control-flow integrity only, not data-flow integrity. Selection-hijacking attacks constitute malicious actions under the paper's own threat model, and the scope limitation should not be buried in Appendix F."}],"tokens_in":23095,"tokens_out":761,"duration_ms":78875,"standing_objections":[]},"desk_editor":{"model":"glm-5.2","letter":"Bottom line: UCM masks untrusted DOM elements before rendering them to the agent, routing content access through a type-constrained Q-Model. The control-flow security argument is sound by construction — if untrusted text never enters the agent's context, prompt injection can't hijack control flow. The 0% ASR on strengthened WASP attacks validates this. Utility is preserved across three frontier agents and two evaluation suites with 1.05x–1.84x cost overhead. Code is public. This is a solid contribution to an active area, and the paper is honest about its limitations throughout — the data-flow attack discussion (Section 6.2, Appendix F) is particularly well-done, identifying selection hijacking as a real but bounded threat with concrete mitigations proposed rather than hand-waved away. The Q-Model design, letting the agent select return types at query time rather than pre-defining them per field, is a genuine improvement over Jacob et al. [2025]. The DOM sanitization approach for automated boundary inference — running the labeling LLM on content-stripped HTML so the labeling itself can't be injected — is a nice detail that shows real engineering care. The soft spot is real but narrow. The security guarantee is binary in labeling correctness: one missed untrusted element exposes the agent. The paper tests single-element mislabeling (ASR rises to 6±5%, Appendix E) and reports automated boundary inference F1 of 0.84–0.997 (Table 2). But it never combines these — no ASR measurement under automatically inferred labels. That's the deployment scenario for passive honest owners, and it's where the practical guarantee lives or dies. The stress-test note is correct on this point. I'd push the authors to run the WASP evaluation with automated labels rather than hand-labeled ones, even if only on GitLab where F1 is 0.84. The honest-owner assumption is reasonable and well-argued, but the gap between 'mostly correct labels' and 'every element correct' is structural, not incremental. The data-flow attacks are acknowledged limitations of control-flow-protecting defenses, not a hole in the central claim. This paper is for researchers and practitioners working on web agent security. It deserves a serious referee who can push on the automated-labeling evaluation gap and assess whether the boundary inference F1 numbers are robust enough for the conditional guarantee to be meaningful in practice. Recommend accept with revisions requiring the combined automated-labeling + ASR experiment.","headline":"UCM is a clean, well-engineered defense that achieves 0% ASR against strengthened WASP attacks by masking untrusted DOM elements before they reach the agent. The control-flow guarantee is real and architectural. The main gap: the paper never measures ASR under automatically inferred labels, which is the deployment scenario that matters.","tokens_in":24121,"tokens_out":618,"would_cite":true,"duration_ms":99970,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"Masking untrusted web content before agents see it blocks all prompt injections","keywords":[],"falsifier":"If an attacker can place prompt-injection content in a DOM element that the labeling system (manual or automated) marks as trusted, the agent will process that content directly and the security guarantee collapses. If the Q-Model can be induced to return a well-typed but malicious value that causes the agent to take a harmful action (a data-flow attack), the guarantee covers control-flow but not data integrity.","tokens_in":23311,"feed_emoji":"🔒","tokens_out":1069,"duration_ms":45291,"temperature":0.7,"pith_summary":"Web agents face a structural problem: to do their job they must observe rendered web pages, but those pages intermingle trusted site content with untrusted user-generated content — reviews, comments, ads — that may carry prompt injection attacks. Prior defenses that provide formal security guarantees work only in text-based tool-use settings where trusted instructions and untrusted data are naturally separated; on the web, that separation does not exist. This paper proposes Untrusted Content Masking (UCM), which restores the missing trust boundary by exploiting a structural fact about web pages: the Document Object Model (DOM) encodes enough information to identify which regions are untrusted without reading their content. UCM replaces every untrusted DOM element with a labeled placeholder before the page reaches the agent, so the agent never processes adversarial text. When a task genuinely requires reading untrusted content — for example, checking whether a product review is positive — the agent queries a Quarantined Model (Q-Model), an isolated model that reads the hidden content and returns only a structured response constrained to one of five types: boolean, integer, float, enum, or date. Because a type-constrained output cannot carry free-form instructions such as 'navigate to evil.com,' injected commands cannot propagate back into the agent's reasoning. The paper evaluates UCM across ten custom websites and 41 GitLab task templates from WebArena, using three frontier agents, and reports that task utility is preserved while cost overhead ranges from 1.05× to 1.84×. Against strengthened attacks from the WASP benchmark, UCM achieves a 0% attack success rate. The paper also shows that trust boundaries can be inferred automatically from DOM structure alone — without manual labeling — using an LLM operating on content-sanitized HTML, achieving F1 scores of 0.84 to 0.997 on Booking, Reddit, and GitLab.","feed_headline":"Masking untrusted web content before agents see it blocks all prompt injections","feed_subtitle":"A DOM-level defense restores the trust boundary that provable security requires, preserving task utility at 1-2x cost.","key_machinery":"Untrusted Content Masking (UCM): a client-side framework that intercepts page rendering and replaces untrusted DOM elements with labeled placeholders. Quarantined Model (Q-Model): an isolated model that reads hidden untrusted content and returns only type-constrained structured responses (bool, int, float, enum, date). Automated boundary inference: an LLM analyzes content-sanitized HTML (all text replaced with length placeholders) to produce CSS selectors identifying untrusted regions, enabling deployment without manual labels.","core_discovery":"The central mechanism is the combination of two ideas: (1) the DOM carries sufficient structural signal to separate trusted from untrusted regions without reading their content, and (2) type-constrained outputs from an isolated model cannot carry injected instructions. Together, these restore the trust boundary that provable defenses require, but in the web setting where it was previously thought impossible. The agent operates in its normal observe-act loop, seeing page structure with untrusted regions masked; when it needs information from a masked region, it asks a Q-Model a natural-language question and declares the return type, receiving back only a parsed boolean, integer, float, enum,,","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["DOM-level masking isolates web agents from injected prompts","Redacting untrusted DOM regions before rendering blocks prompt injection","Structural DOM signals let agents skip untrusted content entirely","Sandboxed Q-model returns only typed data, defeating web prompt injection","Masking untrusted page regions restores provable web-agent security"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The security guarantee requires that every untrusted DOM element is correctly labeled as such. A single missed element containing attacker-controlled content exposes the agent to injection. The paper's automated boundary inference achieves F1 scores of 0.84–0.997, meaning 0.3–16% of untrusted element groups are missed on real websites, and even a single-element mislabeling raises the attack success rate to 6±5%. The gap between 'mostly correct labels' and 'every element' is a","fun_headline_variants_meta":{"raw":{"variants":["DOM-level masking isolates web agents from injected prompts","Redacting untrusted DOM regions before rendering blocks prompt injection","Structural DOM signals let agents skip untrusted content entirely","Sandboxed Q-model returns only typed data, defeating web prompt injection","Masking untrusted page regions restores provable web-agent security","Web agents stay secure by never reading untrusted content directly","Type-constrained isolated model strips injection from web agent inputs","DOM trust boundaries enable provable defenses for web agents"]},"model":"glm-5.2","effort":"high","cost_usd":0.0,"raw_usage":{"total_tokens":682,"prompt_tokens":517,"completion_tokens":165,"prompt_tokens_details":null},"tokens_in":517,"tokens_out":165,"duration_ms":2928,"temperature":1.0,"reasoning_tokens":54,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-07T19:54:58.967873+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"If an attacker can place prompt-injection content in a DOM element that the labeling system (manual or automated) marks as trusted, the agent will process that content directly and the security guarantee collapses. If the Q-Model can be induced to return a well-typed but malicious value that causes the agent to take a harmful action (a data-flow attack), the guarantee covers control-flow but not data integrity.","supporting_citations":[],"review_version":1}