{"id":"e5439d54-176a-4717-82ff-a21c0893f559","arxiv_id":"2608.02109","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"SPIRAL aligns a model's rendered-image input behavior to its native-text input behavior using self-generated text-path supervision, nearly closing the vision-text compression gap on VTCBench.","lead":"This paper presents SPIRAL, a training method that makes vision-language models understand text that is rendered into images instead of fed as normal text. It lifts the model's score from 35 to 54 on a standard benchmark, close to the 55.6 score the same model gets with ordinary text input.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SPIRAL's gains are consistent with an image-blind model that answers from a text-path prior; style-perturbation robustness does not prove the rendered image is read, and the 32k DPO result reinforces the concern.","rationale":"The reader's weakest assumption is essentially the one I identify: the paper assumes the cross-path gap is due to visual-style sensitivity and that rendering preserves the needed information, so aligning pS to pT is safe. I agree with that reading, and I sharpen it: even granting that rendering is lossless, the training objective as written does not force the image path to actually use the rendered image. The perturbation robustness result is the key evidence cited for semantic grounding, but it is equally consistent with an image-blind model, because an image-blind model is by definition invariant to rendering style. The 32k DPO result is the strongest internal signal that something beyond pS-to-pT alignment is happening; a distillation process should not systematically exceed its teacher on the same content. That observation could have a benign explanation (preference optimization can improve over the teacher), but it makes the faithfulness question urgent rather than merely hypothetical. The proposed content-substitution audit is a single experiment that separates 'reads the image' from 'ignores the image and uses the query/text prior.' I am not claiming the paper is wrong; the framework and results are coherent enough to merit conditional acceptance with this check required. The lack of released code/data and the same-family training/evaluation are additional concerns, but the faithfulness question is more load-bearing because it targets the stated mechanism. I therefore keep the reader's CONDITIONAL verdict: accept pending the audit, not reject.","tokens_in":15593,"tokens_out":5500,"duration_ms":54214,"concrete_test":"Content-substitution audit: fix 100 retrieval/reasoning queries and the rendering parameters; for each query, render two long contexts that differ only in the target answer (e.g., different needle values) while keeping style, length, and surrounding text as similar as possible. Run the trained SPIRAL-OPD and SPIRAL-DPO models on both rendered contexts with the same query, and measure how often the output matches the answer actually present in the rendered context. Compare against an image-blind control that receives the query without the context image. If SPIRAL's tracking accuracy is near the control or chance while style-perturbation robustness stays high, the gains are not evidence of reading the image; if tracking is high, the concern is resolved. Repeat at 32k context length to check whether the DPO-vs-text-mode anomaly remains.","verdict_should_be":"UNCHANGED","load_bearing_attack":"SPIRAL's central claim is that cross-path inconsistency is the bottleneck of VTC and that aligning the image-path distribution pS to the text-path distribution pT recovers performance because the ViT is style-sensitive rather than information-lossy. The evidence for the 'style-sensitive, not lossy' premise (Section 3.2, Figure 5b) consists of perturbation studies that change rendering style while keeping text content fixed. A model that has learned to ignore the rendered image and answer from a query-conditioned textual prior would also be invariant to such perturbations, so the perturbation result cannot distinguish genuine semantic grounding from image blindness. The paper never varies the semantic content of the rendered image with query and style held fixed, leaving faithfulness unmeasured. The surprising 32k reasoning result (Section 4.4: DPO 22.26 vs. native text 7.18) makes the concern concrete: a pure pS-to-pT alignment should not systematically beat the teacher on identical content; DPO's objective (Eq. 5) rewards the preferred text-path response without any term enforcing that the response is entailed by the rendered image, so it can improve benchmark scores by teaching the model to rely on the query rather than the image. Section 5.2 lists limitations but omits a faithfulness/attribution audit. Without a content-substitution test, the headline claim that alignment teaches the ViT linguistic semantics is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper addresses Vision-Text Compression (VTC), where long text is rendered into an image and encoded through the vision encoder, yielding visual tokens that are much fewer than the native text tokens. The authors identify \"cross-path inconsistency\": the model's behavior on the rendered-image input diverges from its behavior on the same content given as native text. They propose SPIRAL, a self-supervised framework that aligns the image-path output distribution toward the text-path distribution using token-level on-policy distillation (OPD) and sequence-level direct preference optimization (DPO), with all supervision derived from the model's own text-path outputs. On VTCBench, SPIRAL improves Qwen3-VL-8B from 35.10 to 54.02, approaching the native text-input score of 55.60 and outperforming existing VTC methods; the authors also report gains on out-of-domain benchmarks and on a second backbone, InternVL3.5-8B. The paper concludes that cross-path alignment, rather than compression ratio alone, is the central challenge for effective VTC.","tokens_in":15844,"tokens_out":4865,"duration_ms":45143,"significance":"If the central claim holds, the paper reframes the VTC research agenda from compression efficiency to cross-path representation alignment and offers a practical, fully self-supervised recipe that preserves inference-time efficiency. The strengths are the self-supervised formulation (no external teacher or annotations), the systematic comparison of token-level versus sequence-level alignment, the perturbation-robustness analysis, the reported NLL-gap reduction, and the preliminary cross-backbone evidence. However, the significance is contingent on two issues: the in-domain gains may be inflated by overlap between the training data and the evaluation benchmark families, and the paper does not demonstrate that the aligned model actually reads the rendered image rather than relying on a query-conditioned text prior. Both issues are testable and fixable, but they are load-bearing for the interpretation of the results.","major_comments":[{"comment":"The rendering-perturbation experiment does not establish the premise that the cross-path gap is due to style sensitivity rather than information loss. A model that ignores the rendered image and answers from the query and a text prior would also be invariant to rendering perturbations, because it never reads the image. The conclusion stated at the end of Section 3.2, that \"the cross-path gap arises because the ViT encoder is sensitive to rendering style rather than to the linguistic content,\" is therefore not supported by the presented evidence. Please add a content-substitution experiment: hold the query and rendering style fixed while replacing the rendered document with a semantically different document, and measure whether the model's outputs and accuracy track the substituted content. Such an experiment is necessary to distinguish genuine semantic grounding from image blindness.","section":"Section 3.2, Table 1"},{"comment":"The training data overlaps the evaluation benchmark families. Retrieval training uses RULER needles in a PG19 haystack, reasoning training uses NoLiMa needles, and memory training uses LoCoMo-style conversations, while evaluation uses RULER, NoLiMa, and LoCoMo. The headline improvement from 35.10 to 54.02 may therefore be substantially inflated by in-distribution task familiarity. The out-of-domain results in Table 3 are encouraging, but they do not quantify how much of the VTCBench gain is attributable to the overlap. Please report results on an evaluation split whose task families and data sources are disjoint from the training data, or retrain on non-overlapping task families, and explicitly discuss the degree of overlap and its possible effect on the main comparison.","section":"Section 4.1, Table 2"},{"comment":"The DPO reasoning result at 32k (22.26) exceeds the native text-mode score at the same length (7.18). Because the DPO objective uses text-path responses as the preferred samples, a pure imitation mechanism should not systematically outperform the teacher. This result raises the concern that DPO is learning a shortcut: satisfying the preference by increasing dependence on the query and a text prior rather than by reading the rendered image. Please provide diagnostics that test actual use of the rendered content, for example: (a) content substitution (same query, different rendered document), (b) occlusion or removal of the image tokens, and (c) analysis of attention attributed to image tokens. These tests are directly feasible and would resolve whether the observed gains reflect genuine cross-path alignment or an image-blind policy.","section":"Section 4.4, Figure 3, Eq. (5)"},{"comment":"The manuscript reports no error bars, no number of seeds, and no significance tests for any of the headline numbers. The central comparison, OPD at 54.02 versus text mode at 55.60, is a 1.58-point difference that may fall within the run-to-run variance of fine-tuning an 8B model. Please report mean and standard deviation over at least three seeds for all main configurations (baseline, SFT, OPD, DPO, and text mode) on the primary VTCBench results, and provide a statistical test for the claimed gap-closing effect.","section":"Section 4.2, Tables 2, 4, 5"}],"minor_comments":[{"comment":"The phrase \"disjoint needle vocabularies\" is not explained; please define how disjointness is enforced and clarify why it does not prevent task-format leakage between training and evaluation.","section":"Section 4.1"},{"comment":"Table 4 reports an OPD total of 55.57 at 10k training examples, while Table 2 reports OPD at 54.02 for what appears to be the same method; the relationship between these numbers is unclear. Please state the exact evaluation configuration used in Table 4 (context lengths, task subsets) and explain why the 10k model appears to outperform the 30k model.","section":"Tables 2 and 4"},{"comment":"The notation for the divergence is confusing: the text says D(pT, pS) denotes KL(pS||pT). Please use a consistent argument order, e.g., D(pT, pS) with a convention that the first argument is the target, or define D explicitly as D_target(source).","section":"Section 3.3, Eq. (2)"},{"comment":"The native text-mode reasoning score in Figure 3 at 32k (7.18) differs dramatically from the overall text-mode reasoning score in Table 2 (37.26). Please clarify the context-length composition of the Table 2 number and explicitly state that Figure 3 reports a length-conditioned breakdown.","section":"Figure 3 and Table 2"},{"comment":"The limitations section does not discuss the possibility that the model may learn to rely on the query or a text prior instead of the rendered image. Please add an explicit discussion of this failure mode and how it relates to the reported results.","section":"Section 5.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is promising and the core idea is plausible, but two load-bearing concerns need to be addressed before acceptance: the training/evaluation overlap and the lack of a faithfulness test that rules out an image-blind shortcut. Both are fixable within the scope of a revision. The authors should also be asked to reconcile the apparent inconsistency between the 10k model in Table 4 and the 30k model in Table 2, and to provide a clearer statement of the evaluation protocol for each table."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth reading: it identifies a genuine problem — rendered text goes through the ViT and the representations drift from the native-text path — and it proposes a sensible self-supervised remedy, aligning the image path to the text path with token-level OPD and sequence-level DPO. The perturbation study is a nice diagnostic, and the reported gains are large: 35.10 to 54.02 on VTCBench, close to the 55.60 text-mode ceiling. The retrieval numbers in particular (OPD 89.34) suggest the model is doing something right, because you cannot retrieve a needle from a haystack by guessing from the query alone. The scaling analysis (OPD sample-efficient, DPO scales) and the OOD transfer are also genuinely informative.\n\nThat said, the support has real gaps. First, there are no error bars or seeds, and no code or data. For a paper whose main claim is a 19-point jump, that is a problem. Second, the training data is built from the same benchmark families as VTCBench (RULER, NoLiMa, LoCoMo), even with disjoint vocabularies, so the in-domain numbers are partly a distribution-match story. Third — and this is the load-bearing issue — the paper never tests whether the aligned model is actually reading the rendered image. The perturbation invariance is consistent with the model learning to ignore the image and answer from a query-conditioned prior. The 32k DPO reasoning result (22.26 vs. 7.18 for native text) is exactly what that failure mode would look like, and the DPO objective in Eq. 5 rewards text-path agreement without any entailment term.\n\nI don't think the image-blind interpretation is proven — the retrieval gains push against it — but the paper needs to settle it with a content-substitution test: keep the query fixed, swap the rendered text, and check whether answers track the actual content. That is a routine faithfulness audit, and it is missing from Section 5.2's limitations. The NLL-gap measurement in Figure 5(a) measures agreement with the text path, not correctness, so it does not resolve the question.\n\nMy take: this deserves a serious referee, but it should come back with the faithfulness test, error bars, and ideally the code. If the faithfulness audit shows the model is genuinely grounded in the rendered image, the core claim stands. If it shows the model is an image-blind text prior with good haystack retrieval, then the framing collapses. As written, the evidence is not enough to trust the headline, but the idea is strong enough to invest referee time in.","headline":"A real diagnosis and a plausible fix, but the paper never proves the aligned model is actually reading the rendered image, so the headline claim is under-evidenced.","tokens_in":16406,"tokens_out":2775,"would_cite":false,"duration_ms":28446,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Vision-text compression's real bottleneck is cross-path inconsistency, and a self-supervised method closes most of the gap.","keywords":["Vision-Text Compression","Cross-path alignment","Cross-path inconsistency","On-policy distillation","Direct preference optimization","Long-context modeling","Multimodal large language models","Self-improving alignment"],"falsifier":"Render a long context at a resolution low enough that a competent OCR system cannot recover the text, keep the query in native text, and run SPIRAL: if the aligned model still approaches native-text scores, the recovery cannot come from reading the rendered content and the premise that no information is lost must be false.","tokens_in":15347,"feed_emoji":"🖼️","tokens_out":5878,"duration_ms":48176,"temperature":0.7,"pith_summary":"Vision-Text Compression (VTC) promises cheap long-context processing by rendering long text into images and encoding it with the vision transformer, but it consistently underperforms native text input. This paper argues that the cause is not compression itself but cross-path inconsistency: the vision encoder reads glyphs, fonts, and layout, while the language model expects linguistic semantics. The paper claims this gap can be closed with only the model's own text-input behavior as supervision, through SPIRAL, which combines token-level on-policy distillation and sequence-level preference optimization. On VTCBench, SPIRAL raises Qwen3-VL-8B from 35.10 to 54.02, close to the native-text score of 55.60, while adding no cost at inference time. If right, representational fidelity rather than compression ratio is the key design target for VTC.","feed_headline":"Rendered-text alignment lifts VTC score from 35.10 to 54.02","feed_subtitle":"Self-supervised alignment makes vision-text compression nearly match native text, at zero extra inference cost.","key_machinery":"The load-bearing object is the pair of conditional distributions $p_T$ and $p_S$ produced by the same multimodal backbone from native text and from rendered-image input. The mechanism that carries the argument is self-generated cross-path supervision: because the same model generates both, no external teacher is needed. OPD applies a mode-seeking divergence (reverse KL) at each decoding step on the image path's own rollouts, sharpening $p_S$ toward $p_T$; DPO steers the image-path policy at the sequence level by maximizing the margin between text-derived and image-derived responses. Both operate only during training, leaving the compressed inference path unchanged.","core_discovery":"The paper's central discovery is that VTC's performance gap is dominated by a mismatch between the two paths the same backbone can take for identical content: native text tokenization and rendered-image encoding. They introduce SPIRAL, which supervises the image path with the text path's own outputs. Token-level OPD distills the image path's next-token distribution toward the text path's distribution along self-generated image-path trajectories, using reverse KL; sequence-level DPO constructs preference pairs where the native-text answer is preferred and the rendered-image answer is dispreferred, both conditioned on the rendered input. Together these close most of the 20.5-point gap to native text, improve robustness to rendering perturbations, and generalize to context lengths and benchmarks not seen in training.","pith_inferences":["Beyond the paper: if DPO reasoning at 32k surpasses native text, the rendered-image path plus alignment may act as a beneficial regularizer for very long contexts, not merely a lossy approximation.","Beyond the paper: the same dual-view alignment recipe could apply to other rendered modalities such as tables, charts, or code listings, where a text-path teacher exists and the vision encoder may again prefer layout over content.","Beyond the paper: a hybrid objective that interleaves or schedules OPD and DPO may outperform either alone, since the paper reports complementary strengths but does not combine them.","Beyond the paper: the surprising ability to match or beat text-mode reasoning could also mean the model is learning to exploit question priors rather than reading the rendered context; a content-masking control would separate these."],"forward_implications":["VTC benchmarks should report or control for cross-path alignment, since a large share of the performance gap can be removed without changing the compression pipeline.","Models aligned this way preserve the full token-compression and inference-speedup benefits of VTC, because alignment adds nothing at inference time.","Alignment trained on contexts up to 8k tokens transfers to 32k contexts, implying the method learns a general semantic grounding rather than length-specific patterns.","OPD and DPO offer complementary scaling behavior: OPD is sample-efficient and best for retrieval, while DPO improves reasoning and memory and benefits from more preference data.","The gains generalize out of domain (LongBench, 2WikiMultihopQA, GSM8K, TriviaQA) and to a second backbone, InternVL3.5-8B."],"supporting_citations":[{"why":"Provides VTCBench, the benchmark and task suite against which the cross-path gap and the alignment gains are measured.","marker":"[54]"},{"why":"Supplies the on-policy distillation method of aligning on self-generated trajectories, which SPIRAL adapts for token-level cross-path alignment.","marker":"[1]"},{"why":"Provides the DPO preference-optimization objective used for sequence-level alignment in SPIRAL-DPO.","marker":"[44]"},{"why":"A representative VTC system that establishes rendering text into images as the compression paradigm SPIRAL builds on.","marker":"[8]"},{"why":"Earlier work encoding rendered text as visual tokens, grounding the image-path setup the paper analyzes.","marker":"[45]"},{"why":"Another VTC baseline using vision-centric token compression, providing a comparison system in the benchmark evaluation.","marker":"[51]"},{"why":"Defines the RULER retrieval task and needle-in-a-haystack protocol used in the VTCBench retrieval group.","marker":"[20]"},{"why":"Provides LongBench, an out-of-domain benchmark used to test whether cross-path alignment generalizes beyond VTCBench.","marker":"[3]"}],"fun_headline_variants":["SPIRAL self-alignment hikes VTC score 35 to 54","Self-taught SPIRAL realigns VTC to near native text","VTC gap shrinks 35 to 54 with self-supervised SPIRAL","No external teacher: SPIRAL boosts VTC to near-native level","SPIRAL: vision-text compression reaches 97% of native performance"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes the problem is how the model reads the rendered image, not whether the rendered image still contains the information; if rendering destroys content, then pushing image-path behavior toward text-path behavior could make the model guess from the question instead of reading.","fun_headline_variants_meta":{"raw":{"variants":["SPIRAL self-alignment hikes VTC score 35 to 54","Self-taught SPIRAL realigns VTC to near native text","VTC gap shrinks 35 to 54 with self-supervised SPIRAL","No external teacher: SPIRAL boosts VTC to near-native level","SPIRAL: vision-text compression reaches 97% of native performance"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000644,"raw_usage":{"total_tokens":2976,"prompt_tokens":979,"completion_tokens":1997,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":595,"completion_tokens_details":{"reasoning_tokens":1896}},"tokens_in":595,"tokens_out":1997,"duration_ms":14291,"temperature":1.0,"reasoning_tokens":1896,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:01:39.207925+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Render a long context at a resolution low enough that a competent OCR system cannot recover the text, keep the query in native text, and run SPIRAL: if the aligned model still approaches native-text scores, the recovery cannot come from reading the rendered content and the premise that no information is lost must be false.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Earlier work encoding rendered text as visual tokens, grounding the image-path setup the paper analyzes."}],"review_version":2}