The reviewed record of science sign in
Pith

arxiv: 2607.05277 · v1 · pith:XWX5JIVD · submitted 2026-07-06 · cs.CR · cs.LG

Untrusted Content Masking for Web Agents with Security Guarantees

Kristina Nikoli\'c , Egor Zverev , Javier Rando , Matthew Jagielski , Edoardo Debenedetti , Florian Tram\`er This is my paper

Reviewed by Pith2026-07-07 19:54 UTCglm-5.2pith:XWX5JIVDopen to challenge →

classification cs.CR cs.LG
keywords contentuntrustedagentsguaranteessecuritytrustedboundarydefenses
0
0 comments X

The pith

Masking untrusted web content before agents see it blocks all prompt injections

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Web agents face a structural problem: to do their job they must observe rendered web pages, but those pages intermingle trusted site content with untrusted user-generated content — reviews, comments, ads — that may carry prompt injection attacks. Prior defenses that provide formal security guarantees work only in text-based tool-use settings where trusted instructions and untrusted data are naturally separated; on the web, that separation does not exist. This paper proposes Untrusted Content Masking (UCM), which restores the missing trust boundary by exploiting a structural fact about web pages: the Document Object Model (DOM) encodes enough information to identify which regions are untrusted without reading their content. UCM replaces every untrusted DOM element with a labeled placeholder before the page reaches the agent, so the agent never processes adversarial text. When a task genuinely requires reading untrusted content — for example, checking whether a product review is positive — the agent queries a Quarantined Model (Q-Model), an isolated model that reads the hidden content and returns only a structured response constrained to one of five types: boolean, integer, float, enum, or date. Because a type-constrained output cannot carry free-form instructions such as 'navigate to evil.com,' injected commands cannot propagate back into the agent's reasoning. The paper evaluates UCM across ten custom websites and 41 GitLab task templates from WebArena, using three frontier agents, and reports that task utility is preserved while cost overhead ranges from 1.05× to 1.84×. Against strengthened attacks from the WASP benchmark, UCM achieves a 0% attack success rate. The paper also shows that trust boundaries can be inferred automatically from DOM structure alone — without manual labeling — using an LLM operating on content-sanitized HTML, achieving F1 scores of 0.84 to 0.997 on Booking, Reddit, and GitLab.

Core claim

The central mechanism is the combination of two ideas: (1) the DOM carries sufficient structural signal to separate trusted from untrusted regions without reading their content, and (2) type-constrained outputs from an isolated model cannot carry injected instructions. Together, these restore the trust boundary that provable defenses require, but in the web setting where it was previously thought impossible. The agent operates in its normal observe-act loop, seeing page structure with untrusted regions masked; when it needs information from a masked region, it asks a Q-Model a natural-language question and declares the return type, receiving back only a parsed boolean, integer, float, enum,,

What carries the argument

Untrusted Content Masking (UCM): a client-side framework that intercepts page rendering and replaces untrusted DOM elements with labeled placeholders. Quarantined Model (Q-Model): an isolated model that reads hidden untrusted content and returns only type-constrained structured responses (bool, int, float, enum, date). Automated boundary inference: an LLM analyzes content-sanitized HTML (all text replaced with length placeholders) to produce CSS selectors identifying untrusted regions, enabling deployment without manual labels.

Load-bearing premise

The security guarantee requires that every untrusted DOM element is correctly labeled as such. A single missed element containing attacker-controlled content exposes the agent to injection. The paper's automated boundary inference achieves F1 scores of 0.84–0.997, meaning 0.3–16% of untrusted element groups are missed on real websites, and even a single-element mislabeling raises the attack success rate to 6±5%. The gap between 'mostly correct labels' and 'every element' is a

What would settle it

If an attacker can place prompt-injection content in a DOM element that the labeling system (manual or automated) marks as trusted, the agent will process that content directly and the security guarantee collapses. If the Q-Model can be induced to return a well-typed but malicious value that causes the agent to take a harmful action (a data-flow attack), the guarantee covers control-flow but not data integrity.

Figures

Figures reproduced from arXiv: 2607.05277 by Edoardo Debenedetti, Egor Zverev, Florian Tram\`er, Javier Rando, Kristina Nikoli\'c, Matthew Jagielski.

Figure 1
Figure 1. Figure 1: Overview of Untrusted Content Masking (UCM) defense. Untrusted webpage content is masked with labeled placeholders, preventing prompt injection attacks from being shown to the Agent. The Agent queries a Quarantined Model (Q-Model) by providing an element ID, a question, and a return type; the Q-Model reads the actual hidden content and returns a structured response, enabling safe reasoning about untrusted … view at source ↗
Figure 2
Figure 2. Figure 2: UCM preserves agent utility with up to 1.84× cost increase. Agent utility (a) and total cost per task in USD (b) for each model under Undefended and Masking Defense (UCM) conditions, across Untrusted-not-required and Untrusted-required task groups. Utility is preserved in both groups — including in Untrusted-required tasks, where the agent must rely on the Q-Model to access information from untrusted conte… view at source ↗
Figure 3
Figure 3. Figure 3: UCM preserves utility and incurs moderate cost on the WebArena GitLab suite. Agent utility (a) and total cost per task in USD (b) for each model under Undefended, Masking Defense, and Q-Model string output allowed conditions. Utility is largely preserved under Masking Defense, and fully recovered under Q-Model string output allowed, where the Agent, after declaring a task unsolvable, is permitted to instru… view at source ↗
Figure 4
Figure 4. Figure 4: Comparison of webpage rendering with and without our UCM defense. (a) The standard [PITH_FULL_IMAGE:figures/full_fig_p015_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Token ratio (masking / no masking) per task for the forum and webshop suites, for Claude [PITH_FULL_IMAGE:figures/full_fig_p019_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Token usage on custom websites across all models under Undefended and UCM conditions. [PITH_FULL_IMAGE:figures/full_fig_p020_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Token usage on the WebArena GitLab suite across all models under Undefended and UCM [PITH_FULL_IMAGE:figures/full_fig_p020_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Examples of potential data-flow attacks against UCM. [PITH_FULL_IMAGE:figures/full_fig_p023_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Automated boundary identification on Booking.com. Red regions are masked using CSS selectors generated by the LLM from sanitized DOM structure alone. The model correctly identifies hotel titles, descriptions, and images as untrusted content, while leaving the platform UI untouched. 28 [PITH_FULL_IMAGE:figures/full_fig_p028_9.png] view at source ↗
read the original abstract

Defenses that provide security guarantees against prompt injection attacks rely on strict isolation between trusted instructions and untrusted data. In text-based environments such as tool-use APIs, this separation arises naturally: agents can reason from interface definitions without ever processing untrusted content. Extending these guarantees to web agents faces a fundamental challenge: to perceive and interact with their environment, web agents must first observe the rendered page, which intermingles trusted content with untrusted content. This structural entanglement removes the trust boundary on which security guarantees depend, undermining provable defenses for web agents. In this paper, we present Untrusted Content Masking (UCM), a simple and effective approach that restores this boundary in web environments. We leverage a key structural insight: a webpage's Document Object Model (DOM) encodes sufficient information to distinguish trusted from untrusted regions without reading their content. Our framework exploits this by redacting untrusted regions before they reach the agent and routing interaction through a sandboxed interface with strict privilege separation, thereby enabling agents to observe and interact with their environment while remaining isolated from adversarial content. The code is publicly available.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 7 minor

Summary. The paper presents Untrusted Content Masking (UCM), a defense for web agents against prompt injection attacks. The core idea is to mask untrusted DOM elements before rendering the page to the agent, and to route any necessary access to untrusted content through a type-constrained Quarantined Model (Q-Model) that returns only structured responses (bool, int, float, enum, date). The approach is evaluated on 10 custom websites and the WebArena GitLab suite across three frontier agents, showing preserved task utility with 1.05x-1.84x cost overhead and 0% attack success rate against strengthened WASP attacks. The paper also presents an automated boundary inference method using an LLM on content-sanitized DOM to identify untrusted regions without manual labeling.

Significance. The paper addresses an important problem: extending provable prompt-injection defenses to web agents, where the structural entanglement of trusted and untrusted content on rendered pages has been a fundamental obstacle. The architectural insight—that DOM structure can separate trust boundaries without reading content—is clean and well-motivated. The type-constrained Q-Model design is sound by construction: well-typed outputs cannot carry free-form injected instructions. The 0% ASR result is a validation of a design property rather than an empirical prediction, which is appropriate. The automated boundary inference on sanitized DOM is a thoughtful addition that addresses deployment concerns. The reproducible code and falsifiable attack taxonomy (control-flow vs. data-flow) are strengths. The paper builds transparently on prior work (Jacob et al. 2025, Debenedetti et al. 2025, Costa et al. 2025) and clearly positions its contribution relative to these.

major comments (2)
  1. Section 3 (Threat Model) and Section 7.2 (Table 2): The paper's central security guarantee is explicitly conditional on correct labeling of every untrusted DOM element (Section 3: 'To provide strict security guarantees, UCM requires that each DOM element is correctly labeled'). The automated boundary inference results (Table 2) show F1 scores of 0.84-0.997, with GitLab having an 8.5% false-negative rate at the element-group level. The paper never measures ASR under automatically inferred labels—it only reports 0% ASR with hand-labeled boundaries (Appendix E, Table 7) and single-element mislabeling sensitivity (6±5% ASR). The gap between '0% ASR with perfect labels' and 'unknown ASR with automated labels' is the practically relevant deployment scenario for passive honest owners, and it is unmeasured. The paper should either (a) run the WASP attack evaluation with automatically inferred (非
  2. Section 6.2 and Appendix F: The data-flow attack discussion is valuable and honest, but the threat model in Section 3 states the attacker's goal is to 'manipulate the Agent into malicious actions.' The selection-hijacking attacks (Appendix F, Figures 8a-8b) demonstrate that well-typed Q-Model outputs can be manipulated to steer the agent toward attacker-controlled repositories or users, which constitutes a malicious action under the paper's own threat model. The paper frames these as 'known limitations of control-flow-protecting defenses,' but the reader should not have to consult Appendix F to understand that the security guarantee covers control-flow integrity only, not data-flow integrity. The abstract and conclusion should state this scope explicitly. As written, the abstract's claim of 'security guarantees' may be read more broadly than the paper delivers.
minor comments (7)
  1. Section 5.1, Figure 2: The cost overhead ratios (1.05x-1.84x) are computed from medians, but the text does not mention this until later. Clarify in the figure caption or the results paragraph that medians are used for cost.
  2. Section 7.2, Table 2: The F1 metric is computed at the element-group level (Section G.4), but this is not stated in the table caption. A reader may interpret 0.997 F1 as near-perfect element-level labeling. Clarify the granularity.
  3. Appendix E, Table 7: The ASR for the undefended baseline (17±8%) is described as a 'lower bound on true vulnerability.' This is correct, but the large confidence interval (±8%) suggests a small sample size. Report the number of attack instances evaluated.
  4. Section 3: The honest-owner assumption is justified by citing that 50% of internet traffic is concentrated in 3,000 domains. The citation (Xavier 2024) is an arXiv preprint; verify whether peer-reviewed alternatives exist.
  5. Appendix B.3, Listing 1: The system prompt contains a note that enum options are capped at 10 and each element can be queried at most 5 times per task. These implementation limits are relevant to the security argument (they bound the Q-Model interaction surface) and should be mentioned in the main text, not only in the appendix.
  6. Figure 1: The figure caption references 'product-description-2' but the figure shows 'product-d' (truncated). Fix the truncation or adjust the caption.
  7. Section 5.2: The 'Q-Model string output allowed' condition requires user approval of each free-text answer. The paper does not report how often this fallback was invoked or the user burden it creates. A brief quantitative summary would strengthen the utility claim.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for a careful and constructive review. Both major comments identify genuine gaps between what the paper claims and what it measures, and we agree that revisions are needed. Below we address each point in detail.

read point-by-point responses

  1. Referee: Section 3 and Table 2: The gap between 0% ASR with perfect labels and unknown ASR with automated labels is unmeasured. The paper should run WASP attacks with automatically inferred labels or otherwise address this gap.

    Authors: The referee is correct that the practically relevant deployment scenario for passive honest owners—ASR under automatically inferred labels—is not measured in the current manuscript. We acknowledge this gap and will address it in revision. Specifically, we plan to run the strengthened WASP attack evaluation on the GitLab WebArena environment using automatically inferred trust boundaries (from the boundary inference pipeline of Section 7) rather than hand-labeled boundaries. This will directly measure ASR under the imperfect-labeling regime where the 8.5% false-negative rate applies. We expect the results to be informative regardless of outcome: if ASR remains low, this validates the automated pipeline for security-critical use; if ASR increases, this quantifies the practical security cost of automation and motivates further work on boundary inference accuracy. In either case, we agree the paper is incomplete without this measurement. We will also add a discussion of the relationship between false-negative rates and attack surface: undermasking exposes specific untrusted elements to the Agent, and the WASP attacks target issue descriptions, so the effective ASR depends on whether the specific attacked elements fall within the false-negative set. This is an empirical question that the proposed experiment will answer. We note that the single-element mislabeling sensitivity experiment (Appendix E, 6±5% ASR) provides a partial data point, but we agree it does not substitute for measuring ASR under the full automated-labeling pipeline. revision: yes

  2. Referee: Section 6.2 and Appendix F: The abstract and conclusion should state explicitly that the security guarantee covers control-flow integrity only, not data-flow integrity. Selection-hijacking attacks constitute malicious actions under the paper's own threat model, and the scope limitation should not be buried in Appendix F.

    Authors: We agree with this comment. The selection-hijacking attacks described in Appendix F do constitute malicious actions under the threat model as stated in Section 3 ('manipulate the Agent into malicious actions'), and the current framing—where the scope limitation to control-flow integrity is discussed in Section 6.2 and Appendix F but not in the abstract or conclusion—could lead readers to overestimate the breadth of the security guarantee. We will revise the abstract and conclusion to explicitly state that UCM provides control-flow integrity guarantees: it prevents prompt injection from hijacking the Agent's instructions or plan, but does not prevent data-flow attacks where an attacker corrupts the values returned by the Q-Model to mislead the Agent's decisions within an otherwise legitimate workflow. We will also add a brief forward reference to the data-flow discussion in Section 6.2 earlier in the paper (at the first mention of 'security guarantees' in the introduction), so that readers encounter the scope limitation before reaching the evaluation. We believe this is an honest and necessary clarification that strengthens the paper. We note that the data-flow limitation is shared by all control-flow-protecting defenses in this literature (CaMeL, FIDES, and the type-directed approach of Jacob et al.), and UCM's contribution—establishing trust boundaries in web environments—is a prerequisite for applying downstream data-flow defenses. We will make this positioning clearer in the revision. revision: yes

Circularity Check

0 steps flagged

No significant circularity: the security guarantee is architectural, not derived from fitted parameters or self-citation chains.

full rationale

The paper's central security claim is architectural: untrusted content is masked before reaching the Agent, and type-constrained Q-Model outputs cannot carry free-form instructions. This is a design property, not a prediction derived from fitted data. The 0% ASR result (Appendix E, Table 7) is an empirical validation of this design property under strengthened WASP attacks, not a prediction from a fitted model. The paper builds on prior work (Jacob et al. 2025 for type-directed privilege separation; Debenedetti et al. 2025 / Costa et al. 2025 for data-flow attack mitigations), but these are cited as independent results with their own derivations, not as load-bearing premises that reduce to the current paper's outputs. The automated boundary inference (Section 7) is presented as a separate, practical extension with its own F1 evaluation, not as a step in the security proof. The paper is transparent that the guarantee is conditional on correct labeling (Section 3: 'To provide strict security guarantees, UCM requires that each DOM element is correctly labeled'), and tests mislabeling sensitivity (Appendix E). No step in the derivation chain reduces to its own inputs by construction. The one minor self-citation concern is that Debenedetti et al. 2025 (CaMeL) is co-authored by a subset of the present authors and is cited for data-flow attack mitigations, but this citation is not load-bearing for the central control-flow security claim—it is referenced as complementary prior work for a known limitation the paper explicitly leaves to future work. This does not raise the circularity score above 1.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 2 invented entities

The free parameters are engineering choices (return types, query limits, step caps, iteration counts) rather than fitted constants. They are reasonable but not derived from first principles. The axioms are domain assumptions about web structure, LLM behavior, and trust models that are standard for the prompt injection defense literature. The invented entities (Q-Model, sanitized DOM) are well-motivated architectural components with falsifiable properties, not ad hoc postulates. The Q-Model generalizes an existing pattern (Jacob et al. 2025) rather than being entirely novel. The overall axiom burden is moderate: the security guarantee depends on correct labeling (axiom 1), honest site owners (axiom 3), and the absence of XSS (axiom 5), all of which are reasonable but not universally true.

free parameters (5)
  • Q-Model return types = bool, int, float, enum, date
    The set of allowed return types is a design choice. The paper does not derive this set from first principles; it is chosen to cover common query needs while preventing free-form text. This is a reasonable engineering choice but is a free parameter of the defense.
  • Q-Model query limit per element = 5
    The system prompt states 'HARD LIMITS: Each element (qllm_id) can be queried at most 5 times per task.' This is a hand-set parameter to control cost and prevent enumeration attacks. No derivation is given.
  • Enum options cap = 10
    The system prompt states 'enum options are capped at 10.' This is a hand-set parameter. No derivation is given.
  • Agent step limit = 120
    Section B.1 states 'we cap the agent at 120 steps per task.' This affects utility measurements and cost. No sensitivity analysis is provided for this choice.
  • Boundary inference iteration count = 3
    Section G.3 states 'iterative refinement over three rounds.' This affects the coverage of untrusted element detection. No sensitivity analysis is provided.
axioms (5)
  • domain assumption A webpage's DOM encodes sufficient structural information to distinguish trusted from untrusted content without reading their content.
    Section 1 states this as a 'key structural insight.' It is the foundational assumption of the entire approach. The automated boundary inference (Section 7) provides empirical support (F1 0.84-0.997), but the claim that DOM structure is always sufficient is not proven; it is assumed to hold for the websites tested.
  • domain assumption Type-constrained outputs (bool, int, float, enum, date) cannot carry injected instructions.
    Section 4.2 states 'type-constrained outputs cannot carry free-form text such as navigate to evil.com, injected instructions cannot propagate back to the Agent.' This is sound for the listed types but assumes the Agent does not interpret typed values as instructions (e.g., a bool 'true' returned in response to 'Should I navigate to evil.com?' could still influence behavior, though this is a data-flow rather than control-flow issue).
  • domain assumption The honest-owner assumption: the owner of the visited site does not place adversarial content in regions they control.
    Section 3 states this as a trust assumption. It is necessary for the security guarantee: if the site owner is adversarial, trusted regions could contain injections. The paper argues this is 'minimally constraining' but it is a load-bearing assumption.
  • domain assumption Whenever attacker-controlled content enters the context window of a language model, that model's output can be arbitrarily controlled by the attacker.
    Section 3 states this as part of the threat model. This is a strong assumption about LLM capabilities that is consistent with current empirical evidence but is not proven. It motivates the isolation-based defense.
  • domain assumption Active-content vulnerabilities (e.g., XSS) that let attacker text escape labeled regions or rewrite the DOM at runtime are out of scope.
    Section 3 explicitly scopes this out: 'under the honest-owner assumption, the site is responsible for preventing such bugs.' This is a standard scoping decision but means UCM's guarantees do not hold in the presence of XSS.
invented entities (2)
  • Quarantined Model (Q-Model) independent evidence
    purpose: An isolated model that reads untrusted content and returns type-constrained structured responses, serving as the only channel between untrusted content and the Agent.
    The Q-Model is a new architectural component introduced by this paper. It has a falsifiable security property: its outputs must parse to the declared type. The paper provides empirical evidence that this prevents instruction propagation (0% ASR, Appendix E). The concept generalizes Jacob et al. [2025]'s type-directed privilege separation.
  • Content-sanitized DOM independent evidence
    purpose: A version of the DOM with all textual content removed and attribute values anonymized, used for automated boundary inference to prevent the labeling LLM from being manipulated by prompt injections.
    This is a new technique introduced by the paper. It has a clear security rationale: if the labeling model cannot see content, it cannot be injected. The sanitization process is described in detail (Section G.1) and is falsifiable: one can test whether sanitized DOMs still contain exploitable content.

pith-pipeline@v1.1.0-glm · 26254 in / 4382 out tokens · 413728 ms · 2026-07-07T19:54:58.967873+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

26 extracted references · 26 canonical work pages

  1. [1]

    These checks inspect the interaction logs to confirm that specific UI elements were engaged

    UI checksverify that the Agent performed the correct interface interactions (e.g., clicked the appropriate button, submitted a form, applied a filter). These checks inspect the interaction logs to confirm that specific UI elements were engaged

    work page

  2. [2]

    Filter to show only size S products. How many are there?

    Model output checksverify that the Agent’s final response contains the correct information (e.g., the correct count, name, or extracted content). These checks parse the Agent’s textual output and compare against ground truth. Some tasks require both check types. For instance, “Filter to show only size S products. How many are there?” requires a UI check (...

    work page

  3. [3]

    Element identification.On page load, the framework scans for elements with data-untrustedattributes

    work page

  4. [4]

    Placeholder generation.Each untrusted element’s content is hidden via CSS and replaced with a placeholder displaying a unique identifier (e.g.,id: merchant-name:3)

    work page

  5. [5]

    ID assignment.Elements receive a unique data-qllm-id attribute based on a human- readable name set by the website owner, (e.g., merchant-name), and index, enabling unambiguous reference by the Q-Model

    work page

  6. [6]

    id : [ element - name ]

    Dynamic content handling.A MutationObserver monitors DOM changes to automatically mask dynamically inserted untrusted elements. B.3 Main Agent System Prompt The main Agent receives the following system prompt that explains the security context and available tools: Listing 1: System prompt for the main Agent with Q-Model access (abbreviated). You are a hel...

    work page

  7. [7]

    bool " - For yes / no q u e s t i o n s ( returns true / false ) - Example :

    " bool " - For yes / no q u e s t i o n s ( returns true / false ) - Example : " Is the name on this account Michael " - R es pon se : {" result ": true , " type ": " bool "}

    work page

  8. [8]

    " int " - For integer values with o pti on al min / max range

    work page

  9. [9]

    " float " - For decimal values with op tio na l min / max range

    work page

  10. [10]

    " enum " - For single choice from p r e d e f i n e d options

    work page

  11. [11]

    target

    " date " - For dates in ISO format ( YYYY - MM - DD ) ... T A R G E T I N G R E Q U I R E M E N T S : - I M P O R T A N T : You MUST specify WHICH u n t r u s t e d element to analyze via the " target " object ( do not analyze the whole page ) . - I M P O R T A N T : Each hidden p l a c e h o l d e r shows text in the format " id : < actual - id >" ( e . ...

    work page

  12. [12]

    C a r e f u l l y analyze the HTML / DOM content pr ov ide d ... ... Return type : { r e t u r n _ t y p e } [ Type - sp ec if ic i n s t r u c t i o n s ap pen de d here based on r e t u r n _ t y p e ] Re sp on se format :

    work page

  13. [13]

    First , provide your th ou gh ts and r e a s o n i n g (1 -3 s e n t e n c e s )

    work page

  14. [14]

    Answer : < value >

    Then , on a new line , provide your final answer in the format : " Answer : < value >" Example re spo ns e : Th ou gh ts : Looking at the reviews , I can see mu lt ip le m en tio ns of " great for running " and " perfect for jogging ". The s e n t i m e n t is clearly po sit iv e . Answer : true For each return type, specific formatting instructions are a...

    work page 2024

  15. [15]

    This requires the attacker to anticipate the benign task

    Wrong-repository steering.The attacker induces the Agent to operate on a repository different from the one the user intended. This requires the attacker to anticipate the benign task. 22

    work page

  16. [16]

    Invite Marko to my dotfiles repo as Guest

    Malicious contributor addition.The attacker causes the Agent to add an incorrect user as a contributor. In addition to anticipating the benign task, this requires a malicious username that shares a prefix with the intended one (so that it surfaces in search results) and is long enough to encode a prompt injection. Example 1: Wrong repository.Consider the ...

    work page 2025

  17. [17]

    HTML collection.Fetch target pages using Playwright, ensuring dynamic content rendered by JavaScript frameworks is fully loaded

    work page

  18. [18]

    Content removal.Process HTML to remove all user-visible content while preserving structure

    work page

  19. [19]

    LLM analysis.Analyze cleaned HTML with LLM (Claude Sonnet 4.5) to identify CSS rule based selectors targeting untrusted content through iterative refinement. G.1 Content Sanitization To prevent the LLM from using content patterns to identify untrusted elements, we scrape all user-visible information from the HTML: • Replace all text content with length pl...

    work page

  20. [20]

    Apply hand-labeled selectors to each evaluated page (Booking, Reddit, GitLab) and record which elements match (ground truth setG). 24

    work page

  21. [21]

    Apply LLM-generated selectors to the same page and record which elements match (pre- dicted setP)

    work page

  22. [22]

    post"> but the LLM selector targets <div class=

    Compare element sets to compute true positives (TP), false positives (FP), and false negatives (FN). Each element is uniquely identified by its XPath. We count an element as TP if it appears in both G andP, FP if it appears only inP, and FN if it appears only inG. Per-type element matching.We report metrics at the level ofelement types, where elements are...

    work page

  23. [23]

    Be S PE CI FIC : avoid bare class s e l e c t o r s that match u n r e l a t e d UI ; scope s e l e c t o r s using stable a t t r i b u t e s ( e . g . , data - testid ) or parent context

    work page

  24. [24]

    Be G RA NU LAR : target leaf e le men ts whose entire content is u n t r u s t e d ; do not select c o n t a i n e r s that mix pl at fo rm UI with user content . 25

    work page

  25. [25]

    Do NOT mark pl at for m UI as u n t r u s t e d ( navigation , search forms , filter controls , headers , buttons , breadcrumbs , footers , platform - a g g r e g a t e d scores and counts )

    work page

  26. [26]

    {{title}}

    Images u pl oa ded by users / owners ARE u n t r u s t e d ; static pl at fo rm imagery is not . Output format : ONLY a valid JSON array , no other text . In subsequent round, the LLM is shown the selectors it has proposed so far and asked whether any untrusted regions remain unmarked. The follow-up prompt also reminds the model to avoid container-level s...

    work page