{"id":"3b129e22-a7c2-4abd-ab74-dfed05790116","arxiv_id":"2504.14348","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A coordinated attack that embeds malicious cues in both visual and textual inputs can hijack black-box multimodal agents, outperforming single-modality prompt injection attacks.","lead":"The paper shows how an attacker can hide malicious instructions in both the image and the text a multimodal AI agent sees, getting the agent to carry out tasks it was not built for. It points to a security gap in agents used for driving and assistance, where combining inputs from several channels can override safety rules.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"All ASR numbers depend on an unvalidated Qwen-Max judge; without human agreement or judge-prompt details, the +30.1% headline is not established.","rationale":"The reader's weakest assumption matches this concern. I considered alternative load-bearing risks: the custom agents are not externally validated, the +30.1% vs +26.4% discrepancy between abstract and conclusion, the absence of error bars, and the weak baseline set. These are real but secondary. The judge validation is upstream of every quantitative claim: if Qwen-Max labels are unreliable, no amount of ablations or baseline comparisons fixes the headline. The paper's own Sec 5.1 gives no judge prompt, no human study, and no error analysis; the limitations section does not mention metric validation. This is an omission, not misconduct. I still do not think the central qualitative conclusion -- that coordinated visual and textual injection can hijack these agents -- collapses; the physical-world stop-sign demonstration and the ablation studies provide partial independent support. But the paper's flagship quantitative claim is conditional on the judge. Since the reader already issued a CONDITIONAL verdict and identified this exact assumption, my stress-test does not move the verdict; it sharpens the condition: validate the judge before trusting the ASR numbers. Thus UNCHANGED.","tokens_in":17674,"tokens_out":4952,"duration_ms":46177,"concrete_test":"Obtain the exact judge prompt used with Qwen-Max. Sample 100 agent responses from Table 1, spread across agents, surfaces, and tasks, plus 50 known refusal responses and 50 known compliant responses. Have two independent human annotators label each response as 'executed injected task' or 'not executed', with disagreements adjudicated by a third annotator. Compute Cohen's kappa between Qwen-Max labels and human labels, then recompute ASR and the headline +30.1% gain using human labels. If kappa is below 0.8 or the recomputed gain drops by more than 5 percentage points, the headline ASR claim is not supported. If agreement is high and the gain is stable, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim (+30.1% ASR) rests entirely on an LLM-as-a-Judge protocol (Sec 5.1: 'Both metrics are calculated using the LLM-as-a-Judge approach'), but the paper gives no judge prompt, no human-agreement study, and no calibration of Qwen-Max against ground truth. Because the attack is designed to make LLM-based systems comply with injected instructions, using a powerful LLM as the arbiter of success creates a direct risk that the metric measures the judge's susceptibility to manipulation or leniency rather than actual execution of the injected task by the victim agent. If the judge labels any superficially compliant utterance (e.g., 'Sure, let me help...') as success, ASR is inflated across all conditions and the +30.1% headline is unsupported. This is the single most load-bearing concern: every digital ASR entry in Table 1 inherits this validation gap, and the paper's contribution is defined by these numbers.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper identifies a vulnerability class called cross-modal prompt injection and proposes CrossInject, a black-box attack against VLM-driven multimodal agents. The attack combines Visual Latent Alignment, which uses a text-to-image model to generate a target image of the malicious instruction and then optimizes an l-infinity-bounded perturbation to align the benign input image with that target in an ensemble of surrogate vision encoders, with Textual Guidance Enhancement, which uses adversarial meta-prompting to construct a defensive system prompt and GCG to optimize a malicious textual command on a surrogate LLM. The paper evaluates on two custom-built digital agents (RecipeMaster and PoetryGenius) powered by Qwen2-VL or Phi-3.5-vision, reports ASR gains of at least +30.1% over baselines, presents ablations and defense evaluations, and includes a physical driving case study on a LIMO vehicle where the agent failed to bypass a stop sign in 9 of 10 trials.","tokens_in":17827,"tokens_out":10163,"duration_ms":77409,"significance":"The contribution is potentially significant: if the measured ASR gains are reliable, CrossInject demonstrates that coordinated visual-textual injection is a realistic threat to multimodal agents, and the use of transferable surrogate models is a practical black-box recipe. The paper provides a clear threat model, includes ablations that isolate the visual and textual components, and attempts a physical-world validation, which is valuable and relatively rare in this literature. The main reason the significance is conditional is that every digital metric is produced by an unvalidated LLM judge, so the headline numbers are not yet established.","major_comments":[{"comment":"All ASR and PNA values in Table 1 and the abstract headline are computed by the Qwen-Max LLM-as-a-judge, but the paper provides no judge prompt, no human-agreement study, no error analysis, and no calibration against ground-truth task completion. Since CrossInject is specifically designed to make LLM-based systems comply with injected instructions, the judge may itself be manipulated or over-lenient; the reported ASR could then reflect judge compliance rather than victim-agent execution. Please provide the exact judge prompt and scoring criteria, validate the judge on a human-annotated subset with per-condition agreement, and, where possible, replace judge-based success with objective checks such as verifying that the output actually edits the target text or outputs the required sentiment label.","section":"§5.1, Evaluation Metrics"},{"comment":"The headline improvement is internally inconsistent and undefined. The abstract reports +30.1%, the conclusion reports +26.4%, and Section 5.2 reports +32.7% for local documents and +27.5% for online webpages, with no statement of the formula used to average over baselines and conditions. Recomputing simple averages over Table 1 does not reproduce any of these numbers. Define the exact metric, including whether the gain is absolute or relative, which baselines are included, and how the sixteen settings are aggregated; report the computed value; and make the abstract, body, and conclusion agree.","section":"Abstract; §5.2; §8"},{"comment":"JIP obtains 0% ASR in all sixteen tested conditions. This is surprising for a published visual jailbreak attack and suggests either a fundamental incompatibility or an implementation issue. The paper does not say how JIP was adapted to agent-style inference, which hyperparameters were used, or whether its original goal of jailbreaking a VLM with an embedded instruction is equivalent to the prompt-injection setting here. Please document the baseline configuration and, if JIP is not directly applicable, say so explicitly; otherwise the comparison with CrossInject is not a controlled one.","section":"§5.2, Table 1"},{"comment":"The physical-world evidence is a single scenario repeated ten times, and the success criterion is described inconsistently: the text says the naive textual attack \"prevented the vehicle from bypassing the stop sign in 4 instances\" and CrossInject \"succeeded\" 9 times, while the caption says the attacked case \"collided with a traffic sign.\" It is unclear whether success means ignoring the stop sign, failing to detour, or colliding, and no trajectories, metrics, or confidence intervals are reported. Please define a precise behavioral outcome, report per-trial results, and provide at least basic statistical support such as a binomial confidence interval before drawing safety-critical conclusions.","section":"§6"},{"comment":"Several ablation claims do not match the tables. For Table 3, comparing \"w/o Enhance\" with \"Ours\" gives reductions of 47, 9, 6, and 14 percentage points, averaging 19.0 points, not the stated 24.8%; for Table 4, the claimed 26.9% average improvement of Llama-3.1-8B over the other surrogates is not reproducible from the numbers shown. Please recompute all average-difference claims and ensure the text and tables agree, since these ablations are used to argue that both attack components are necessary.","section":"§5.3, Tables 2–4"}],"minor_comments":[{"comment":"The word \"unrerlated\" should be \"unrelated.\"","section":"§1, Fig. 1 caption"},{"comment":"The phrase \"an malicious textual command\" should be \"a malicious textual command.\"","section":"§4.2"},{"comment":"The text says \"our evaluation leverages three public natural language processing datasets,\" but only CoEDIT and SST2 are listed; either add the missing dataset or correct the count.","section":"§5.1"},{"comment":"The threat model asserts that external data E is processed before the user command C, but the physical case study in §6 removes the external-data interface and delivers the malicious task directly as a user command, so that experiment does not exercise the external-data attack surface described in the threat model.","section":"§3.1 and §6"},{"comment":"Reference [94] is cited as \"Sandwitch Defense\"; the standard name is \"Sandwich Defense.\"","section":"References"},{"comment":"The paper does not state whether code or artifacts will be released; for reproducibility of the GCG and SSA-CWA hyperparameters, please provide the exact optimization recipes or commit to releasing code.","section":"§5.1, Implementation Details"}],"recommendation":"major_revision","confidential_remarks":"To the editor: I see no evidence of misconduct, but the internal numeric inconsistencies (abstract vs conclusion vs §5.2, and the ablation averages) and the reliance on an unvalidated LLM judge should be treated as blocking issues for the quantitative claims. The paper is otherwise a reasonable fit for a security or computer vision venue, and the authors should be encouraged to resubmit after adding human validation of the judge and clarifying the evaluation protocol."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: the attack surface is real and the two-channel method is well designed, but I would not trust the ASR numbers as reported. The central idea—jointly attacking vision and text in a black-box VLM agent—is genuinely new relative to the cited work, and the authors ablate both components cleanly. The qualitative finding that coordinated cross-modal injection beats unimodal baselines is likely correct. The physical driving case (9/10 vs 4/10) is a nice reality check. But the headline +30.1% is not established.\n\nThe load-bearing problem is the evaluation metric. All ASR and PNA numbers come from Qwen-Max as LLM-as-a-Judge, with no judge prompt, no human agreement study, no error analysis. Since the attack is designed to make LLM-based systems comply with injected instructions, using another LLM to decide success risks measuring the judge's own susceptibility rather than actual task execution. That alone makes every digital ASR entry in Table 1 provisional.\n\nSecond, the +30.1% gain appears to be an average over all three baselines, including JIP which scores 0% in every scenario. Averaging against a baseline that always fails inflates the headline. A gain over the strongest baseline would be the honest number. Also, the abstract says '+30.1%' but the conclusion says '+26.4%'—these should reconcile.\n\nThird, the experimental base is narrow. Two custom-built chatbots and one physical vehicle. No standard agent benchmarks, missing baselines like the white-box attack in [63] and typographic visual injections. No error bars, despite claiming three runs. Code and data are not released, so the numbers can't be checked independently.\n\nNone of this kills the qualitative claim. The ablations in Tables 2 and 3 show that removing visual alignment or textual enhancement drops ASR by up to 28 and 47 points respectively, so both components earn their keep. The method section is clear and the threat model is reasonable. I'd take this seriously as a contribution to prompt-injection research, but only after the judge is validated against human labels, the headline gain is recomputed against the best baseline, and the artifacts are released.\n\nRecommendation: send to peer review with a request for major revision. A good referee could turn this into a solid paper. I would not cite the numbers as-is, but I'd cite the method once the evaluation is fixed.","headline":"Cross-modal prompt injection is a real and underexplored attack surface, and the two-channel method is well designed, but the headline numbers rest on an unvalidated LLM judge and an inflated baseline average.","tokens_in":18398,"tokens_out":2721,"would_cite":false,"duration_ms":23014,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Coordinated image-text injection hijacks multimodal agents, beating single-modality attacks by 30.1 percent.","keywords":["cross-modal prompt injection","multimodal agents","vision-language models","adversarial perturbations","black-box attacks","LLM-as-a-Judge","autonomous driving safety","prompt injection defense"],"falsifier":"Re-run the attack suite with the same agents and commands but have human annotators or direct task execution decide success; if the human-verified attack success rate does not exceed the naive baseline by the claimed margin, the LLM-as-a-Judge protocol is carrying the result.","tokens_in":17453,"feed_emoji":"🚗","tokens_out":8300,"duration_ms":69501,"temperature":0.7,"pith_summary":"This paper claims that multimodal agents, systems that fuse images, text, and retrieved external data to act on user requests, are vulnerable to a new attack class called cross-modal prompt injection. The proposed attack, CrossInject, adds imperceptible noise to an image so its latent features align with a malicious instruction while appending an optimized deceptive text command that steers the agent's response. In black-box settings, the paper reports attack success rates at least 30.1 percentage points higher than existing single-modality injection methods across two purpose-built visual-question-answering agents, and it demonstrates the attack on a physical autonomous driving platform where the vehicle failed to detour around a stop sign in 9 of 10 trials. If correct, the paper establishes that securing multimodal agents requires defending the fusion of modalities, not just each input channel separately.","feed_headline":"One perturbed image plus one crafted command hijacks AI agents","feed_subtitle":"Optimized image noise plus a deceptive command beats single-modality attacks by 30.1 percent.","key_machinery":"The load-bearing machinery is a pair of coordinated injection functions with a shared malicious target. Visual Latent Alignment defines a loss that minimizes the normalized $\\ell^2$ distance between a perturbed image's features and a text-to-image-generated target image in an ensemble of vision encoders, subject to an $\\ell^\\infty$ perturbation budget of 16. Textual Guidance Enhancement defines a token-level loss that minimizes the negative log-likelihood of the malicious action under a surrogate LLM prompted with a defense-aware system prompt, optimized by greedy coordinate gradient (GCG) token search. The two components are complementary: visual injection plants task-relevant cues at the input end, while textual injection biases the output distribution toward compliance, so that filtering or refusing one modality is not enough to stop the hijack.","core_discovery":"The central claim is that coordinated adversarial manipulation of both visual and textual inputs to a black-box VLM-driven agent can override the agent's role instructions and force it to execute attacker-specified tasks. The visual component generates a target image that semantically encodes the injected task using a text-to-image model, then optimizes a bounded perturbation so the features of the benign image move toward that target in an ensemble of vision encoders; the textual component constructs a defense-like system prompt through adversarial meta-prompting and runs token-level optimization to find a short deceptive command that maximizes the probability of the desired malicious response. The paper reports that this combination outperforms visual-only and text-only baselines by at least 30.1 percent average attack success rate across text-editing and sentiment-analysis injections on two agent roles and two vision-language planners, and shows transfer to the physical world where the attack made a driving agent ignore a stop sign in 9 of 10 trials.","pith_inferences":["The same alignment-plus-guidance recipe appears portable to other modalities, such as audio, whenever a surrogate encoder and a generative model for target content exist, though the paper only tests vision and text.","If success rates were re-measured by human annotators or by actual task completion rather than an LLM judge, the reported gain could change; a public human-verified benchmark would clarify how much of the effect is real.","The attack's reliance on knowing an approximate role description could be relaxed by inferring the system prompt from the agent's public behavior, widening its black-box applicability."],"forward_implications":["Any deployed VLM-based agent that accepts an uploaded image and a text command should treat input images as potential instruction channels, not just as data to be analyzed.","Single-channel defenses, such as role-prompt reinforcement or image blurring, will likely remain insufficient because the attack coordinates both modalities.","Because the attack transfers across surrogate encoders and surrogate LLMs to unseen victim models, a patch and command optimized once may work against multiple black-box agents.","Physical deployments are in scope: in the paper's driving experiment, the attack turned a stop-sign encounter into a failure to detour in 9 of 10 runs."],"supporting_citations":[{"why":"This visual-only baseline achieves 0 percent attack success across the tested scenarios, which motivates the paper's claim that cross-modal coordination is necessary.","marker":"[21]"},{"why":"This text-only baseline yields only limited attack success, establishing the insufficiency of unimodal textual injection.","marker":"[58]"},{"why":"This supplies the greedy coordinate gradient token optimization used to craft the deceptive textual command.","marker":"[59]"},{"why":"This supplies the text-to-image latent diffusion model that generates target images encoding the malicious task.","marker":"[72]"},{"why":"This supplies the SSA-CWA optimizer used to solve the visual latent alignment loss over an ensemble of encoders.","marker":"[75]"},{"why":"This supplies adversarial meta prompting, used to construct defensive system prompts for the surrogate LLM.","marker":"[78]"},{"why":"This supplies the LLM-as-a-Judge protocol that defines every reported attack success rate.","marker":"[86]"},{"why":"This supplies the physical driving platform used in the real-world case study where the vehicle failed to detour around a stop sign.","marker":"[92]"}],"fun_headline_variants":["Coordinated image-text attack hijacks AI agents","Image plus text attack beats single-modal by 30%","Visual-textual injection overrides agent decisions","Combined adversarial input seizes agent control"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline success rates all come from an LLM judge labeling responses as successful, so if that judge is overly lenient the reported attack success would shrink; the results also assume that two custom-built chatbots and one driving platform adequately represent real multimodal agents.","fun_headline_variants_meta":{"raw":{"variants":["Coordinated image-text attack hijacks AI agents","Image plus text attack beats single-modal by 30%","Visual-textual injection overrides agent decisions","Combined adversarial input seizes agent control"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000225,"raw_usage":{"total_tokens":1476,"prompt_tokens":966,"completion_tokens":510,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":582,"completion_tokens_details":{"reasoning_tokens":451}},"tokens_in":582,"tokens_out":510,"duration_ms":4906,"temperature":1.0,"reasoning_tokens":451,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:51:26.646161+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the attack suite with the same agents and commands but have human annotators or direct task execution decide success; if the human-verified attack success rate does not exceed the naive baseline by the claimed margin, the LLM-as-a-Judge protocol is carrying the result.","supporting_citations":[{"cited_title":"In: USENIX Security Symposium (2024)","cited_arxiv_id":null,"evidence_quote":"This text-only baseline yields only limited attack success, establishing the insufficiency of unimodal textual injection."},{"cited_title":"In: Proceedings of the IEEE/CVF International Conference on Computer Vision (2022)","cited_arxiv_id":null,"evidence_quote":"This supplies the text-to-image latent diffusion model that generates target images encoding the malicious task."},{"cited_title":"In: International Con- ference on Learning Representations (2024)","cited_arxiv_id":null,"evidence_quote":"This supplies the SSA-CWA optimizer used to solve the visual latent alignment loss over an ensemble of encoders."},{"cited_title":"https://global","cited_arxiv_id":null,"evidence_quote":"This supplies the physical driving platform used in the real-world case study where the vehicle failed to detour around a stop sign."}],"review_version":1}