{"id":"837fddaf-914d-4eba-b9f1-7fb7c1e32e74","arxiv_id":"2607.02819","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"low","formal_verification":"none","parameter_count":4,"one_line_summary":"Black-box man-in-the-middle attacks that alter just 10% of transmitted vision tokens reduce LVLM accuracy by up to 88.31% across six models and four benchmarks.","lead":"Cloud-edge LVLMs send intermediate vision tokens over the network, creating an unprotected attack surface. Changing only 10% of those tokens with simple black-box tricks can drop accuracy by as much as 88%, exposing a practical vulnerability in split multimodal inference.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Self-attention disruption computed solely on the intercepted vision-token matrix is only a weak black-box proxy for true token influence on the full LVLM, so the optimization-based selector’s claimed advantage rests on an unvalidated heuristic.","rationale":"The reader correctly isolates the vision-only self-attention proxy (Eqs. 2–4 + continuous relaxation in Appendix B) as the weakest assumption supporting the optimization-based half of the strongest claim. The naïve sign-flip results already demonstrate large accuracy collapses under a 10 % budget, so the existence of the vulnerability is empirically robust; the proxy only needs to be faithful for the additional claim that the selector “significantly improves attack effectiveness.” Because that improvement is modest on the headline numbers and rests on an architectural heuristic that has never been validated against true token influence, the paper remains CONDITIONAL pending stronger evidence (error bars, released code, and a white-box sanity check of the proxy). No deeper internal inconsistency or experimental artifact undermines the core empirical finding, therefore the reader’s verdict is left unchanged.","tokens_in":18076,"tokens_out":620,"duration_ms":39539,"concrete_test":"On a held-out subset of OCRBench/MMBench images for Qwen2.5-VL-3B (white-box access permitted only for this diagnostic), rank vision tokens by the magnitude of the gradient of the model’s next-token cross-entropy w.r.t. each token embedding; compute Jaccard overlap of the Top-k gradient set with the Top-k set returned by Algorithm 1. If mean overlap is statistically indistinguishable from random (≈ ρ), the proxy does not identify the most damaging tokens and the optimization claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim’s strongest quantitative result (88.31 % drop) and the novelty of the optimization method both rely on maximizing L_SA(w) − λ L_norm(w) (Eqs. 2–4) over continuous selection weights w, then taking Top-k. L_SA is defined exclusively on the vision-token Gram matrix Attn(V) = softmax(V V^⊤ / √d) V; no text tokens, no cross-attention, no later LLM layers, and no output loss enter the objective. Appendix C acknowledges a high-norm bias and the mitigating effect of LayerNorm, yet the paper never verifies that the resulting index set I actually coincides with the tokens that most alter P(y_t | · ; θ). Because the naïve sign-flip baseline already collapses several Qwen models, the vulnerability itself is not at stake; what is at stake is the assertion that the black-box procedure “identifies these vulnerable vision tokens” and thereby amplifies the attack. If the proxy is poorly correlated with true influence, the optimization contribution collapses to a modest, possibly non-significant improvement over random selection.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper identifies the transmission of intermediate vision tokens from edge to cloud in partitioned LVLM inference as a new man-in-the-middle attack surface. Under a black-box budget constraint (token manipulation ratio ρ), it defines four naïve VTM-Attacks (permutation, masking, Gaussian perturbation, sign flip) that act on a random subset of tokens, plus an optimization-based selector that maximizes a self-attention disruption loss L_SA regularized by a norm-aware term L_norm (Eqs. 2–4) via projected gradient ascent on continuous weights w, followed by Top-k. Experiments on six Qwen2.5-VL and InternVL3.5 models (3B–72B) across OCRBench, MMBench, HallusionBench and MathVista show that ρ=0.1 already produces large accuracy collapses (up to 88.31 percentage points under optimized sign flip), with ablations on ρ, objective components and wall-clock cost.","tokens_in":18423,"tokens_out":1137,"duration_ms":19465,"significance":"If the reported effect sizes hold, the work establishes a practically relevant vulnerability for an increasingly common deployment pattern (edge vision encoder + cloud LLM). The evaluation is unusually broad for an attack paper—six models spanning two families, four diverse benchmarks, systematic ρ sweeps, objective ablations and latency comparisons—and the quantitative drops are large enough to be actionable for system designers. The optimization procedure, while heuristic, is a concrete black-box contribution that measurably improves over random selection on several model–benchmark pairs. The manuscript therefore supplies both a clear threat model and reproducible empirical evidence that defenses at the vision-token interface are needed.","major_comments":[{"comment":"Section III-C and Eqs. (2)–(4) (with continuous relaxation in Appendix B): L_SA is defined solely on the vision-token Gram matrix Attn(V)=softmax(VV⊤/√d)V; no text tokens, cross-attention or downstream LLM layers enter the objective. While Table I and Fig. 3 show that the resulting Top-k sets improve attack success over random selection, the paper never directly checks whether those indices coincide with tokens of highest influence on P(y_t|V,P,Y_<t;θ). A simple white-box or leave-one-out correlation study (even on a single model) would confirm that the proxy is not merely selecting high-norm or high-variance tokens that happen to work under sign flip.","section":"Section III-C, Eqs. (2)–(4)"},{"comment":"Table I (and the corresponding ρ-sweep tables in Appendix A): All naïve-attack numbers appear to be single-run point estimates with no standard deviation or multi-seed statistics. Because the optimization gains are sometimes modest (a few points) and the random baseline itself is stochastic, it is impossible to judge whether the reported improvements are statistically reliable. Reporting mean±std over at least 3–5 independent index draws for the random columns would make the claim that the selector “significantly improves attack effectiveness” quantitatively defensible.","section":"Table I"}],"minor_comments":[{"comment":"Fig. 2 and the Appendix A tables: the y-axis scales and line styles make it hard to compare absolute accuracy across model families at a glance; a common y-range or an additional relative-drop panel would improve readability.","section":"Fig. 2"},{"comment":"Notation: the composite signal is introduced as T:={V,S} but later the cloud receives T and tokenizes S; a short clarifying sentence that S already contains the visual placeholders would avoid momentary confusion.","section":"Section II"},{"comment":"Several OCR/encoding artifacts remain (e.g., “naïve” rendered with diaeresis, “TMA-Attacks” vs. “VTM-Attacks” in the efficiency subsection). A final proof-reading pass is needed.","section":"Section IV-D"},{"comment":"Appendix D sets α=0.5 without sensitivity analysis; a one-sentence note on how results change for α∈{0.25,1.0} would reassure readers that the adaptive-λ scheme is not brittle.","section":"Appendix D"}],"recommendation":"minor_revision","confidential_remarks":"The central vulnerability claim is already supported by the naïve sign-flip results alone; the optimization contribution is secondary and the two major comments above are fixable with modest additional experiments or discussion. The paper is a solid fit for a security or systems venue that values empirical threat demonstrations. I see no citation or novelty-disclosure issues."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The punchline is simple: once you split an LVLM so that vision tokens leave the edge, a black-box MITM that flips the sign of only 10 % of them can drive accuracy from the high 80s to near zero on several Qwen models. That is a concrete, high-impact finding for a deployment pattern that is already being productized.\n\nWhat is new is the attack surface itself. Prior work hits raw images or intermediate text; here the adversary never sees pixels or model weights and still collapses performance under a strict token budget. The four naïve operators (permutation, masking, Gaussian, sign-flip) are cleanly defined, and the multi-model, multi-benchmark tables (six LVLMs, four tasks, full ρ sweeps in the appendix) make the vulnerability hard to dismiss. Sign-flip is consistently the most destructive, which matches the geometry they describe. The wall-clock comparison showing that even the optimized selector finishes in tens of milliseconds is a nice practical touch.\n\nThe soft spots are real but secondary. The optimization objective maximizes self-attention disruption computed only on the vision Gram matrix; no text tokens, cross-attention, or output loss ever enter. Appendix C notes the high-norm bias and LayerNorm, yet they never check whether the Top-k set actually coincides with the tokens that most change P(y|·). Because naïve sign-flip already destroys several models, the vulnerability claim stands without the selector. The selector’s reported gains are therefore modest improvements over random, not the load-bearing novelty. Missing multi-seed error bars on the random baselines and unreleased attack code are ordinary reproducibility gaps, not fatal ones. Mitigations are mentioned only in the conclusion.\n\nThis is for people building or securing split multimodal systems. The math is elementary, the data are broad, and the citation pattern is fair. I would send it to peer review; a referee can demand the correlation check and code release without questioning the core empirical result. Worth reading and, once the artifacts appear, worth citing when you discuss edge-AI threat models.","headline":"Budgeted MITM on vision tokens is a real, well-demonstrated attack surface for cloud-edge LVLMs; the attention selector is a useful heuristic whose advantage is real but modest and under-validated.","tokens_in":18989,"tokens_out":513,"would_cite":true,"duration_ms":6155,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Manipulating only 10% of vision tokens in cloud-edge LVLM inference can cut accuracy by up to 88%.","keywords":["cloud-edge inference","large vision-language models","vision token manipulation","man-in-the-middle attack","self-attention disruption","secure inference","edge intelligence"],"falsifier":"Re-run the same sign-flip and optimisation-based attacks on an LVLM whose intermediate vision tokens are protected by authenticated encryption or whose vision encoder is fused with the language backbone so that no free-standing token stream exists; if accuracy remains near the benign baseline, the claimed vulnerability disappears.","tokens_in":18969,"feed_emoji":"🔓","tokens_out":854,"duration_ms":9912,"temperature":0.7,"pith_summary":"Cloud-edge systems for large vision-language models send compact vision tokens from a light edge encoder to a heavy cloud language model. The paper shows that a black-box man-in-the-middle adversary who intercepts that stream can flip, mask, noise, or reorder a small budget of those tokens and drive accuracy near zero. Four simple manipulations already cause large drops; an optimization that picks tokens to maximize self-attention disruption while avoiding high-norm bias makes the attack still stronger. Across six models (3B–72B) and four standard benchmarks the worst-case drop reaches 88 percentage points when only one-tenth of the tokens are touched, and the attack itself costs only milliseconds. The work therefore establishes that the vision-token link is a practical, high-leverage attack surface that current cloud-edge designs leave unprotected.","feed_headline":"Flip 10% of vision tokens, accuracy falls up to 88%","feed_subtitle":"Black-box attacks on the edge-to-cloud link expose a critical hole in split LVLM systems","key_machinery":"VTM-Attack together with the continuous relaxation of token selection that maximises the Frobenius gap between original and manipulated self-attention matrices while subtracting a norm-aware penalty (Eqs. 2–4 and Algorithm 1).","core_discovery":"Under a black-box man-in-the-middle model that may alter at most a fraction ρ of the vision tokens while preserving their shape, simple value-based manipulations—especially sign flip—already collapse accuracy on multiple LVLMs; an optimization that selects the subset maximizing self-attention change under a norm-aware regularizer further amplifies the damage, showing that 10 % token manipulation is sufficient to reduce accuracy by as much as 88.31 %.","pith_inferences":["The same self-attention proxy could be inverted to produce a cheap token-level integrity check that flags anomalous attention patterns before decoding begins.","Because the attack works without model weights, similar token-stream vulnerabilities may appear in any split-inference system that ships intermediate embeddings, not only vision-language models.","Sign-flip’s extreme effectiveness suggests that directional (rather than magnitude) information is the dominant carrier of visual semantics inside current LVLMs."],"forward_implications":["Any cloud-edge LVLM pipeline that transmits unprotected vision tokens is immediately open to high-impact, low-latency sabotage.","Defences must either encrypt or authenticate the token stream, or redesign the split so that no semantically rich intermediate representation leaves the edge.","Model families differ sharply in robustness (Qwen collapses far more readily than InternVL), so architecture choice itself becomes a security decision.","Attack cost is negligible compared with 4G/5G transmission latency, so the adversary can act without introducing detectable delay."],"fun_headline_variants":["10% vision token flips drop LVLM accuracy up to 88%","Black-box 10% token changes cut cloud-edge accuracy 88%","Sign-flip 10% vision tokens, accuracy falls 88%","Select and alter 10% tokens, LVLM accuracy drops 88%","Edge-to-cloud 10% token edits reduce accuracy by 88%"],"cache_read_input_tokens":11648,"weakest_assumption_plain":"Self-attention computed only on the intercepted vision-token matrix is a faithful enough black-box proxy for how those tokens later affect the full model’s output distribution.","fun_headline_variants_meta":{"raw":{"variants":["10% vision token flips drop LVLM accuracy up to 88%","Black-box 10% token changes cut cloud-edge accuracy 88%","Sign-flip 10% vision tokens, accuracy falls 88%","Select and alter 10% tokens, LVLM accuracy drops 88%","Edge-to-cloud 10% token edits reduce accuracy by 88%"]},"model":"grok-4.5","effort":"low","cost_usd":0.007102,"raw_usage":{"total_tokens":1731,"prompt_tokens":725,"num_sources_used":0,"completion_tokens":99,"cost_in_usd_ticks":71020000,"prompt_tokens_details":{"text_tokens":725,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":907,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":725,"tokens_out":99,"duration_ms":7070,"temperature":1.0,"reasoning_tokens":907,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-12T06:50:15.268524+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Re-run the same sign-flip and optimisation-based attacks on an LVLM whose intermediate vision tokens are protected by authenticated encryption or whose vision encoder is fused with the language backbone so that no free-standing token stream exists; if accuracy remains near the benign baseline, the claimed vulnerability disappears.","supporting_citations":[],"review_version":1}