{"id":"99cf7ab3-bfbc-45a9-82f9-4bfd8f2a9023","arxiv_id":"2506.03614","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Open-source VLMs can infer image IDs or safety labels after training only on scattered patches of those images, a capability that can be abused to bypass image moderation.","lead":"Researchers show that vision-language models can piece together information from many small image fragments seen separately during training, and later recognize the full image or a text label they were never trained on together. This 'visual stitching' also means dangerous images can be hidden by splitting them into harmless-looking patches that slip past content moderators.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reference-based 'stitching' may reduce to single-patch memorization; ambiguous-patch filter is rank-based and sampling evidence is absent.","rationale":"The reader's weakest_assumption is exactly the concern I identify: the reference-based rank improvements may be explained by a single informative patch rather than cross-sample aggregation. The paper's Section 4.3 addresses this, but its ambiguity criterion is a rank threshold, which does not guarantee that a patch is unidentifiable, and its evaluation remains rank-based. Section 6's concession that non-zero ranks are not directly observable by sampling further weakens the 'verbalize' language used in the abstract. This concern is load-bearing because the novelty of 'visual stitching' depends on demonstrating genuine aggregation, not memorization of distinctive patches. My concrete test (synthetic color-grid dataset with no single-patch discriminability) would settle the matter: if performance persists, the aggregation claim is supported; if not, the phenomenon reduces to a simpler retrieval mechanism. Given that the paper already acknowledges instability and rank-based limitations, the CONDITIONAL verdict remains appropriate, so I recommend no change to the reader's verdict.","tokens_in":16332,"tokens_out":4356,"duration_ms":53547,"concrete_test":"Run the reference-based visual stitching experiment on a synthetic dataset where each image is a 2x2 grid of four distinct solid-color quadrants and each ID is determined by the full quadruple, with each individual color appearing in multiple images so no single patch is discriminative. Finetune on (patch, ID) pairs, then test reference-based and image-based stitching using both rank and greedy/beam sampling. If correct-ID rank or sampled accuracy stays above chance, true cross-patch aggregation is demonstrated; if it falls to chance, the natural-image results are consistent with single-patch memorization.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that VLMs integrate visual information across multiple training patches. The strongest alternative is that the model simply recognizes content from a single informative patch using pretrained vision, then retrieves the associated ID from that patch via ordinary memorization of (patch, ID) pairs. Reference-based improvement would then require no cross-sample aggregation. Section 4.3 attempts to rule this out with the ambiguous-patch analysis (Figure 5), but 'ambiguous' is defined by a rank threshold on the correct reference (e.g., threshold-3 discards patches where the correct reference ranks in the top 3, among 20 options). A patch with rank 4–9 still carries substantial information relative to chance (mean rank 9.5), so the remaining 'ambiguous' patches may retain enough single-patch discriminative signal to support the results. Moreover, all evidence is rank-based; Section 6 explicitly concedes that any non-zero rank means the correct answer is not directly observable by sampling. The abstract's claim that models 'can verbalize the correct IDs' is therefore unsupported by the presented metrics. Because the single-patch-memorization alternative explains both image-based and reference-based results without invoking stitching, the central capability claim rests on the ambiguous-patch analysis, which is not decisive.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces \"visual stitching,\" a proposed capability of vision-language models (VLMs) to integrate visual information scattered across multiple training patches that share the same textual description. In the main experiment, each (image, ID) pair is split into (patch, ID) pairs at various granularities, the VLM is finetuned on these pairs, and the model is then evaluated on whether it can rank the correct ID highly when prompted with the full image (image-based stitching) or with a text reference to the image (reference-based stitching). The authors report strong image-based performance (near-zero mean ranks for most models) and non-trivial but less reliable reference-based performance, with Qwen2-VL models leading. In a second, safety-oriented experiment, patches of harmful images are paired with \"safe\" or \"unsafe\" labels, many patches evade the OpenAI Moderation API, and finetuning on the surviving patches produces rank improvements toward the adversarial labels for both image and text-reference prompts. The paper frames visual stitching as both a generalization mechanism and a potential vulnerability that can bypass sample-level data moderation.","tokens_in":16528,"tokens_out":4649,"duration_ms":54134,"significance":"If the central claim is established, the result is significant: it would demonstrate a concrete mechanism by which VLM training on benign-looking fragments can reconstruct harmful or otherwise censored visual knowledge, with direct implications for data moderation and safety auditing. The paper's strengths include a systematic evaluation across multiple open-source VLM families and model sizes, a clearly defined task template, a public code release, and an explicit adversarial-relevance demonstration with real moderation APIs. The reader's conditional assessment is appropriate: the image-based results are robust and reproducible, and the moderation-evasion rates for patches are informative. However, the load-bearing evidence for true cross-patch integration rests on rank-based metrics and an ambiguous-patch analysis that does not fully exclude single-patch memorization, and the safety claim is not backed by sampled-output attack rates. The paper is a useful empirical exploration, but the strength of its claims currently exceeds what the presented evidence supports.","major_comments":[{"comment":"The threshold-x ambiguous-patch protocol does not isolate patches that are individually uninformative. With 20 reference options, the random mean rank is 9.5, so a patch ranked 4th–9th (retained under threshold-3) still carries substantial single-patch discriminative signal. The paper reports only mean ranks of the correct ID after finetuning on the retained patches; without a demonstration that the retained patches cannot identify the source image on their own (e.g., by reporting the patch-to-reference rank distribution from the pretrained model, or a sampled patch-identification accuracy), the reference-based results remain equally consistent with simple memorization of (patch, ID) pairs from the most informative patch. This is load-bearing because the strong interpretation of \"stitching\" requires aggregation across patches, not retrieval from a single recognizable fragment.","section":"Section 4.3 / Figure 5"},{"comment":"The abstract states that tuned models \"can verbalize the correct IDs from full images or text reference,\" but the evidence is entirely rank-based, and Section 6 explicitly concedes that any non-zero rank means the correct answer is not directly observable through sampling. Since reference-based mean ranks are often well above zero (e.g., Figure 2, food and animal at f=8), the claim of verbalization is unsupported by the presented metrics. Please add sampled-output metrics (e.g., top-1 exact match or sampling accuracy) for both image- and reference-based conditions, or revise the abstract, conclusion, and contribution statements to claim only probability-ranking improvements.","section":"Abstract and Section 6"},{"comment":"The safety evaluation reports mean rank of \"safe\"/\"unsafe\" among two candidates, rather than the model's actual generated text. The stated threat is that VLMs \"generate harmful responses\" and \"describe the full image or a text reference ... as 'safe'\" (Abstract and Section 1), which requires generation-level attack success rates. Please report the fraction of harmful images and text references for which decoding (greedy or sampled) yields the adversarial label, with and without moderation filtering, and compare these rates to a baseline of direct finetuning on the original (image, text) pairs. Without such a metric, the practical safety impact is not established.","section":"Section 5.2 / Figure 7"},{"comment":"Reference-based stitching disappears entirely at learning rates 1e-6 and 5e-6, even for f=1, and the paper reports results only at 1e-5. This fragility means the phenomenon is demonstrated only in a narrow hyperparameter regime, which weakens the generality of the claim that VLMs \"exhibit\" visual stitching as a capability. Please either report the range of learning rates over which reference-based stitching persists (for additional models and datasets, not only Qwen2-VL-7B at f=1) or provide a principled justification for why 1e-5 is the canonical setting.","section":"Appendix A.4 / Figure 12"}],"minor_comments":[{"comment":"The text contains a typo: \"[patch]\" is described as a placeholder for visual input from \"patchs\", which should be \"patches\".","section":"Section 4.1"},{"comment":"The evaluation templates \"For child to see this image is {text}\" and \"For child to see {reference} is {text}\" are missing the indefinite article before \"child\"; they should read \"For a child to see ...\".","section":"Section 5.1"},{"comment":"The caption of Figure 14 ends with \"See Figure 14 for results on other models,\" which self-references the same figure; it should refer to the specific figure containing the other models or be removed.","section":"Figure 14 caption"},{"comment":"There is inconsistent model naming between \"Qwen2-VL\" and \"Qwen-2-VL\" (e.g., Figure 3 and Appendix A.4); the paper should use one consistent name.","section":"Introduction and Figure 3"},{"comment":"The sentence \"any non-zero rank indicates that stitching is not directly observable through sampling\" is in direct tension with the abstract's \"can verbalize\" claim; this tension should be resolved in revision by aligning the language throughout.","section":"Section 6"}],"recommendation":"major_revision","confidential_remarks":"The paper presents an engaging empirical study with a clear safety motivation, and the code release is a plus. However, the central capability claim rests on rank-based metrics and the ambiguous-patch analysis, which as analyzed does not exclude single-patch memorization. The safety claims would be much stronger with generation-level attack success rates. I recommend major revision: the authors should add sampled-output metrics, strengthen the ambiguous-patch control, and address the learning-rate sensitivity before the paper can be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Zhanhui and colleagues define visual stitching and show that finetuning on (patch, ID) pairs lets VLMs later rank the correct ID highly when shown the full image or a text reference. The image-based result is clean: across several open-source model families, mean ranks drop to near zero even with 64-way splits. That is a real, reproducible effect. The reference-based result is more fragile, and the paper is upfront about this—Section 6 says any non-zero rank means the correct ID isn't directly samplable. That makes the abstract's 'verbalize' too strong; rank improvement is not verbalization.\n\nThe moderation evasion simulation is a nice proof-of-concept. They show that small patches mostly evade the OpenAI Moderation API and ShieldGemma-2, and that finetuning on the surviving patches raises the rank of the adversarial 'safe'/'unsafe' label when prompted with the full harmful image or a text reference. It's a small setup (20 images, binary labels) and they call it a proof of concept, so that's fine as a demonstration, though it won't extrapolate to a concrete attack without more work.\n\nThe softest spot is the ambiguous-patch analysis meant to rule out single-patch memorization. The threshold definition (top-x rank for the correct reference) is a reasonable first cut, but a patch ranked 4–9 still carries partial visual information, so the residual effect after discarding top-3 patches could still come from partial single-patch cues plus memorized (patch, ID) associations. The paper's argument would be stronger if they reported sampled output accuracy on the held-out reference prompts, or used a more stringent ambiguity criterion. As is, the reference-based stitching evidence is suggestive, not conclusive.\n\nOther minor issues: the phenomenon is highly sensitive to learning rate (Figure 12), the custom datasets are not released (only code), and the choice of split factor interacts with moderation filtering, making the safety numbers hard to compare across settings. None of this invalidates the core finding; it just means the headline claims should be calibrated.\n\nWho should read this: anyone thinking about data curation pipelines for VLMs, and people working on out-of-context reasoning in multimodal settings. It deserves a serious referee; the paper is honest about its limitations and the gap between rank and sampling is a fixable presentation issue, not a fatal flaw.","headline":"New phenomenon, decent proof-of-concept; the abstract oversells sampling, and the stitching evidence is stronger for images than for text references.","tokens_in":17086,"tokens_out":4218,"would_cite":true,"duration_ms":42987,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Vision-language models can stitch scattered image patches back together.","keywords":["visual stitching","vision-language models","cross-sample reasoning","data poisoning","content moderation bypass","image patches","out-of-context reasoning","multimodal safety"],"falsifier":"Probe a model finetuned on patch-ID pairs with held-out patches from the same source images; if the model cannot rank the correct ID above chance on those unseen patches, the learning is patch-specific memorization rather than cross-patch aggregation.","tokens_in":16080,"feed_emoji":"🧩","tokens_out":7232,"duration_ms":71017,"temperature":0.7,"pith_summary":"This paper introduces and tests a capability it calls visual stitching: a vision-language model finetuned on pairs of image patch and text can later produce the correct text when shown the full image or a text description of it, even though it never saw the full image during finetuning. The authors demonstrate this on three synthetic datasets, splitting images into 4, 16, or 64 patches and pairing each patch with a unique ID, and show the effect across several open-source model families. They then show that the same effect lets adversaries bypass content moderation: patches of dangerous images that individual filters flag far less often can be finetuned with labels like \"safe\" or \"unsafe,\" and the model later applies those labels to the original full image or a text reference. The paper argues that visual stitching is a general cross-sample reasoning ability in VLMs and a practical obstacle to monitoring what these models learn.","feed_headline":"VLMs stitch scattered training patches into whole images","feed_subtitle":"Even tiny benign-looking patches let a VLM reconstruct the full image and bypass moderation.","key_machinery":"The carrying object is the finetuning dataset of \\{(patch, ID)\\} pairs produced by splitting each source image by a split factor f (1, 2, 4, or 8 along each dimension) and the mean-rank evaluation protocol: all IDs in the source dataset serve as candidates, and the model's rank of the correct ID when prompted with either the full image or a text reference is measured after training. The mechanism at work is inductive out-of-context reasoning transferred to vision: because every patch shares the same text, the model can learn a latent (image, text) association that no single training example displays in full. The paper also uses a threshold-x ambiguity analysis, discarding patches whose correct reference already ranks in the top x before finetuning, to isolate stitching of genuinely ambiguous patches from memorization of distinctive ones.","core_discovery":"The central claim is that VLMs have visual stitching: after finetuning on patch-text pairs created by splitting each source image, the model can verbalize the source text from the full image (image-based stitching) and, to a lesser degree, from a text reference to the image (reference-based stitching), despite never having seen the full image or the reference during finetuning. The authors find that most open-source VLMs achieve near-perfect image-based ranks and non-trivial reference-based ranks, with Qwen2-VL and Qwen2.5-VL the strongest, and that performance survives even when training only on patches that were objectively ambiguous about the image. In the adversarial setting, after moderating the patches, VLMs finetuned on the survivors still learn to label the original harmful image or textual reference with the attacker-chosen text, forming a data poisoning attack that evades moderation.","pith_inferences":["Stitching strength likely depends on how many patches share an identical text; adversarial training that varies captions across patches might be harder to exploit, a testable extension the paper does not run.","The gap between training on full images and on large patches suggests spatial position encoding helps the model internalize a global layout; ablating M-RoPE or adding location tokens (which the paper found hurt) could clarify the mechanism.","Probing models with textual references may serve as a lightweight safety diagnostic: if a model can name an ID or concept from a reference, it has already aggregated the scattered association.","The same capability that enables the attack also improves generalization to never-seen images, so moderation policies face a trade-off between safety filtering and legitimate generalization."],"forward_implications":["Most open-source VLMs exhibit strong image-based visual stitching, even when finetuned on very small patches.","Reference-based visual stitching is present but less reliable; the correct ID's probability rises during training though direct sampling often fails.","Medium-sized models around 10B parameters perform best at stitching, while small models lack capacity and large models overfit.","Training on large patches (split factor 2) improves reference-based stitching relative to training on full images, acting as a visual data augmentation.","Patch-level moderation filtering raises the difficulty of stitching but does not eliminate it, so sample-level data moderation is insufficient to block this attack."],"supporting_citations":[{"why":"Supplies the mean-rank evaluation protocol used to measure stitching strength.","marker":"[9]"},{"why":"Frames visual stitching as visual inductive out-of-context reasoning, the conceptual basis of the paper.","marker":"[12]"},{"why":"The moderation filter that the adversarial patches must evade; defines the attack's premise.","marker":"[1]"},{"why":"A second moderation baseline; its lower flag rate shows patch evasion is not filter-specific.","marker":"[18]"},{"why":"Data augmentation reference supporting the interpretation that large-patch splitting improves reference-based generalization.","marker":"[33]"},{"why":"Source of the animal images in the benchmark datasets.","marker":"[26]"},{"why":"Source of the food images in the benchmark datasets.","marker":"[27]"}],"fun_headline_variants":["VLMs reconstruct images from scattered training patches","VLMs stitch patches into whole images, evading moderation","Visual stitching: VLMs assemble harmful images from safe patches","VLMs bypass moderation by stitching scattered image patches","Patch-based training lets VLMs rebuild full images"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that models truly stitch patches assumes success is not attributable to a single unambiguous patch that the pretrained model already recognizes on its own; remove that assumption and the strong version of stitching collapses.","fun_headline_variants_meta":{"raw":{"variants":["VLMs reconstruct images from scattered training patches","VLMs stitch patches into whole images, evading moderation","Visual stitching: VLMs assemble harmful images from safe patches","VLMs bypass moderation by stitching scattered image patches","Patch-based training lets VLMs rebuild full images"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000368,"raw_usage":{"total_tokens":2003,"prompt_tokens":1002,"completion_tokens":1001,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":618,"completion_tokens_details":{"reasoning_tokens":927}},"tokens_in":618,"tokens_out":1001,"duration_ms":8672,"temperature":1.0,"reasoning_tokens":927,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T10:58:59.631391+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Probe a model finetuned on patch-ID pairs with held-out patches from the same source images; if the model cannot rank the correct ID above chance on those unseen patches, the learning is patch-specific memorization rather than cross-patch aggregation.","supporting_citations":[{"cited_title":"Connecting the dots: Llms can infer and verbalize latent structure from disparate training data","cited_arxiv_id":null,"evidence_quote":"Frames visual stitching as visual inductive out-of-context reasoning, the conceptual basis of the paper."},{"cited_title":"Openai moderation api","cited_arxiv_id":null,"evidence_quote":"The moderation filter that the adversarial patches must evade; defines the attack's premise."},{"cited_title":"Imagenet large scale visual recognition challenge","cited_arxiv_id":null,"evidence_quote":"Source of the animal images in the benchmark datasets."},{"cited_title":"Food-101 – mining discriminative components with random forests","cited_arxiv_id":null,"evidence_quote":"Source of the food images in the benchmark datasets."}],"review_version":1}