{"id":"26d4615b-53ee-4d6f-b6d1-dda148eab973","arxiv_id":"2505.08971","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"PRIOR reweights the next-token prediction loss in vision-language pretraining by 1 minus the probability assigned by a text-only reference LLM, and reports consistent benchmark improvements over standard NTP.","lead":"This paper introduces PRIOR, a training loss for vision-language models that upweights caption tokens a text-only language model finds hard to predict, aiming to focus learning on words that genuinely depend on the image. The authors report benchmark gains of roughly 19% and 8% on two model families, plus better training stability and scaling, though the experiments lack error bars and the stated mechanism is only loosely validated.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 3's surprisal proxy is not validated as image-relatedness and is internally contradicted by the paper's own motivating example: address/price tokens are text-surprising yet image-unrelated, so the reported gains may stem from incidental length/position reweighting rather than the stated…","rationale":"I agree with the reader's weakest_assumption: Eq. 3 is the hinge of the paper. My pass through the full text found additional evidence for that concern. The motivating example in Fig. 1 is a counterexample to the heuristic, since the location and price tokens are maximally surprising to a text-only model yet explicitly image-unrelated; Appendix C's formal argument uses a log-probability difference, not 1-p_r; and Section 4.1 (Fig. 7) reveals a strong length/position confound in the importance scores. Because the empirical comparison is controlled and consistent across two architectures, the correct response is not rejection but conditional acceptance pending a direct validation of the score-to-image-relatedness mapping and/or a position/length control baseline. The reader's verdict already captures this, so I do not change it. I see no need to question the integrity of the experiments or the existence of the performance gain; the concern is about attribution of the effect to the stated mechanism, not about whether the effect occurs.","tokens_in":20445,"tokens_out":7871,"duration_ms":84234,"concrete_test":"On the 100 CapsFusion examples already annotated in Section 1, compute token-level importance scores w_i with the same reference LLM used in the paper, map the human word-level image-related labels to tokens as in Fig. 1, and measure the AUROC (or rank correlation) of w_i against the human labels. If AUROC is not significantly above 0.5, Eq. 3 does not operationalize image-relatedness and the central mechanism is unsupported; if AUROC is clearly above 0.5, this concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing claim is that w_i=(1-p_r(t_i|t_<i))^alpha in Eq. 3 identifies image-related tokens, so that reweighting the NTP loss by w_i prioritizes visual content. This mapping is never tested, and the paper's own example cuts against it. In Fig. 1, the tokens '7338 24th Ave NE, Seattle' and '$365,500' are described as image-unrelated, yet a text-only LLM assigns them near-zero probability; Eq. 3 therefore upweights exactly those tokens, not the highlighted visual words. The mutual-information argument in Appendix C does not repair this: the token-level MI contribution is log p(t_i|v,t_<i) - log p(t_i|t_<i), and 1-p_r is not a proxy for that log-ratio (for a fixed absolute probability gain, the log-ratio is largest when p_r is large, not when p_r is small). Moreover, Section 4.1 (Fig. 7) shows that average importance scores decrease with caption length and with token position, so the reweighting is confounded with length and position. The NTP-vs-PRIOR comparison is a fair controlled experiment, and the 18.6% / 7.9% gains may be real, but they have not been shown to come from prioritizing image-related tokens. The paper already possesses 100 human-annotated examples from CapsFusion (Section 1) that could validate Eq. 3 directly, yet no such validation is reported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PRIOR, a token-level reweighting of the next-token prediction (NTP) loss in vision-language pre-training. Each caption token receives a weight w_i = (1 - p_r(t_i | t_<i))^α, where p_r is the probability assigned by a text-only reference LLM trained on captions without images. The authors argue that tokens hard to predict from text alone are image-related, so upweighting them focuses pre-training on visual content. PRIOR is evaluated against standard NTP on two architectures—H-LVLMs (ViT-MLP-LLM) and U-LVLMs (unified transformer)—across six benchmarks, reporting 18.61% and 7.93% average relative improvements, improved training stability, and better scaling-law exponents. The paper also includes ablations on α and the loss-scale factor k, a comparison with ITM/ITC/ReC baselines, and an importance-sampling and mutual-information motivation.","tokens_in":20790,"tokens_out":7278,"duration_ms":66693,"significance":"If the empirical gains are robust, PRIOR is a simple, architecture-agnostic modification to the standard LVLM pre-training objective that requires only offline-computed token-level scores from a text-only LLM. The controlled comparison against NTP at multiple intermediate checkpoints on two architectures is a genuine strength: the gains are consistent in direction across most checkpoints, and the method is easy to integrate into existing pipelines. The paper does not yet ship code or release the reference-model scores, but the experimental setup is described in sufficient detail for replication. The central weakness is that the paper's stated mechanism—that hard-to-predict tokens are image-related—is not validated, and the headline improvement is computed in a way that overstates per-benchmark gains. The work is best viewed as an empirical contribution on surprisal-based reweighting, provided the authors either validate the mechanism or reframe the claims accordingly.","major_comments":[{"comment":"The reported 'average relative improvement' of 18.61% for H-LVLMs is an aggregate relative gain (sum of PRIOR scores versus sum of NTP scores) rather than a mean of per-benchmark relative gains. From the final checkpoints in Fig. 2, the per-benchmark relative gains are approximately 19.7% (MME-P), 22.5% (MME-R), 12.4% (MMStar), 8.0% (POPE), 4.7% (MMBench), and 4.1% (SEEDBench), whose simple mean is about 11.9%. The aggregate number is dominated by the large-magnitude MME scores. Since the abstract and Fig. 1 headline the 19% figure, this is misleading; the authors should report per-benchmark gains or a clearly defined aggregated average.","section":"§3.3, Figs. 2–3"},{"comment":"The load-bearing assumption that (1 - p_r(t_i | t_<i)) identifies image-related tokens is never validated. The paper's own motivating example in Fig. 1 describes tokens such as '7338 24th Ave NE, Seattle' and '$365,500' as image-unrelated, yet a text-only LLM assigns these tokens very low probability, so Eq. (3) upweights exactly those tokens rather than the highlighted visual words. Section 1 states that 100 human-annotated examples from CapsFusion are available, but no correlation between surprisal and human image-relatedness judgments is reported. The mutual-information argument in Appendix C does not repair this: the token-level MI contribution is log[p(t_i | v, t_<i) / p(t_i | t_<i)] (Eq. 14), and 1 - p_r is not a proxy for that log-ratio (for a fixed absolute probability gain, the log-ratio is largest when p_r is large, not small). The method may still improve downstream performance, but the paper does not demonstrate that it works by prioritizing image-related tokens. Please either provide the human-annotation validation or explicitly reframe the method as surprisal-based reweighting without the image-relatedness claim.","section":"§2.2, Eq. (3), Fig. 1"},{"comment":"The importance scores are confounded with token position and caption length: Fig. 7 shows that the average importance score decreases with caption length (r = -0.243) and also decreases with token position within a caption. This means PRIOR changes the effective length and position distribution of the training loss relative to NTP. The observed gains could stem from these incidental properties rather than from the stated image-relatedness mechanism. A control experiment that reweights by position or length, or by a synthetic score with the same marginal distribution as the importance scores, would be needed to isolate the mechanism. As written, the comparison against NTP does not support the causal claim in the paper's title.","section":"§4.1, Fig. 7"},{"comment":"The importance-sampling derivation is post-hoc: the target distribution in Eq. (6) is defined to be proportional to p_data(t_i | v, t_<i) * (1 - p_r(t_i | t_<i)), which makes the importance weight in Eq. (8) equal to (1 - p_r) by construction. This provides no independent support for the choice of weighting. Similarly, the statement that self-normalization (Eq. 9) 'substantially reduces variance' is asserted without any measurement or reference specific to this setting. The section should be presented as a framing or intuition rather than a derivation of the weight from first principles.","section":"§2.3, Eq. (6)"},{"comment":"The scaling-law analysis lacks the statistical detail needed to support the claim of 'significantly higher scaling coefficients.' The number of data points per fit, the number of independent runs, and the definition of '8 sampling models' are not specified, and no confidence intervals or standard errors are reported for α_D. The extrapolation to 100M tokens from fits on data up to 70M tokens may be brittle, and the paper does not state whether the 100M-token point was actually evaluated. Please provide the fitted parameter values, uncertainty estimates, and a precise description of the sampling procedure.","section":"§3.5, Figs. 5–6"}],"minor_comments":[{"comment":"Equation (7) contains a typo: the subscript 'pdata(t_i|i,t_<i)' should read 'pdata(t_i|v,t_<i)'.","section":"§2.3, Eq. (7)"},{"comment":"The abstract reports '19% and 8%' average relative improvement, while §3.3 reports 18.61% and 7.93%; these numbers should be made consistent.","section":"Abstract and §3.3"},{"comment":"The experimental curves appear to be from a single run per condition; error bars or a statement of the number of seeds would help assess the stability claim.","section":"§3.3, Figs. 2 and 9"},{"comment":"The phrase 'including 8 sampling models' is undefined; please specify whether this refers to data subsets, checkpoints, or independent model runs.","section":"§3.5"},{"comment":"The limitation section acknowledges the tokenizer-sharing constraint but does not mention the unvalidated assumption behind Eq. (3); this missing validation should be listed as a limitation or addressed in the main text.","section":"§D (Limitations)"},{"comment":"The caption states that highlighting is 'word-level for better visualization'; it would be clearer to specify how word-level annotations were mapped to the BPE tokens used for the probability scores.","section":"Fig. 1 caption"},{"comment":"The claim that 'PRIOR also optimizes learning on image-unrelated tokens' is based on average loss differences; lower loss on unweighted tokens may be an indirect effect of shared representations, so a causal interpretation requires more evidence.","section":"§4.2, Fig. 8"}],"recommendation":"major_revision","confidential_remarks":"The empirical comparison is controlled and the gains are directionally consistent, which makes me inclined to keep this paper in the revision track rather than reject it. The main block is the unsupported mechanism claim: the title and framing rest on the identification of 'image-related tokens' via Eq. (3), and the paper's own example cuts against that identification. The authors can plausibly fix this by adding the human-annotation validation they already claim to have, or by reframing the contribution as surprisal-based reweighting. The second block is the misleading headline average, which should be corrected. I see no citation or misconduct concerns beyond the heavy self-citation of the authors' prior work on the U-LVLM architecture, which is relevant here."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the method is simple and the empirical comparison is fair; the headline gains are probably real. But the paper's central story—that tokens the text-only model can't predict are image-related—is contradicted by its own example and never verified. The empirical result and the story are separable, and the paper should be judged on whether the result holds up under cleaner controls.\n\nWhat's new is the specific package: offline-computed soft weights (1−p_r)^α, per-caption normalization, applied to two quite different architectures, with training-curve and scaling-law comparisons against NTP. That is a clean, useful study design. The consistent improvement across intermediate checkpoints, rather than only at the end, is the most convincing evidence that something real is happening.\n\nThe soft spots are proportionate. First, the motivating example. '7338 24th Ave NE, Seattle' and '$365,500' are text-surprising but image-unrelated, so Eq. 3 upweights precisely the tokens the paper wants to de-emphasize. The paper's own 31.3% figure means the majority of non-image tokens are also unpredictable; the surrogate is not validated against those 100 human annotations, which are sitting there. Second, Fig. 7 shows importance weights fall with caption length and token position, so PRIOR is also a position/length reweighting; a control baseline with length-only or position-only weights is needed to attribute the gain to image-relatedness. Third, Rho-1's selective token weighting with a reference model is not cited; the overlap deserves at least a paragraph. Fourth, the 'average relative improvement' is an aggregate, not a mean of per-benchmark relative gains; minor but should be stated. Fifth, there are no error bars; single seeds. Minor given the consistency.\n\nOne correction to the stress-test note: for a fixed absolute probability gain, the log-ratio is larger when p_r is small, not large, so the MI appendix's direction is not mathematically backwards. The real problem is the weight ignores whether the LVLM actually exploits the image for that token, so it can amplify tokens that stay unpredictable even with vision.\n\nAudience: people working on LVLM pretraining objectives, data selection, and scaling. It deserves a serious referee, but the referee should insist on the validation and the control. Without those, it's a heuristic that works for unclear reasons.","headline":"A simple, controlled loss-reweighting method that likely helps in practice, but its stated image-relatedness mechanism is contradicted by the paper's own example and remains unvalidated.","tokens_in":21306,"tokens_out":5336,"would_cite":true,"duration_ms":52207,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a token-level reweighting of the captioning loss by text-only LLM surprisal consistently improves vision-language pre-training over naive next-token prediction, and that the gain grows with compute.","keywords":["vision-language pre-training","next-token prediction","token reweighting","importance sampling","text-only reference model","image-related tokens","scaling laws","hallucination"],"falsifier":"Annotate a sample of captions for human-rated image-relatedness and compute the correlation with $w_i = (1-p_r(t_i|t_{<i}))^\\alpha$; if the correlation is not positive, or if an equal-strength reweighting with shuffled weights reproduces the gains, PRIOR's stated mechanism is not what drives the improvement.","tokens_in":20222,"feed_emoji":"🖼️","tokens_out":5125,"duration_ms":46150,"temperature":0.7,"pith_summary":"The paper argues that standard next-token prediction is the wrong objective for vision-language pre-training because it spreads loss evenly over caption tokens, most of which have nothing to do with the picture. It proposes PRIOR, which reweights each token's loss by how surprising that token is to a text-only language model: tokens the text model cannot predict are treated as image-related and get upweighted. Across two architectures, encoder-based and encoder-free, this reweighted objective consistently outperforms the unweighted baseline by 19% and 8% average relative improvement, with better training stability and steeper scaling curves. If the effect holds, a purely offline, model-free weight can replace auxiliary objectives and data filtering in large-scale vision-language pre-training.","feed_headline":"Surprisal-weighted captions beat plain next-token pre-training by 19%","feed_subtitle":"A text-only LLM's surprise at each caption word tells the LVLM which tokens to learn from the image.","key_machinery":"The load-bearing object is the importance score $w_i = (1 - p_r(t_i|t_{<i}))^\\alpha$, computed offline by a text-only reference model trained on captions without images. The paper inserts the normalized weight $k w_i / \\sum_j w_j$ into the NTP loss so that tokens the text model finds hard to predict receive more gradient signal. This implements the paper's assumption that surprisal under a text-only model tracks visual information content; the same score drives the importance-sampling interpretation and the mutual-information justification in the appendix.","core_discovery":"PRIOR's central claim is that the conditional probability assigned by a text-only LLM, $p_r(t_i|t_{<i})$, can identify which caption tokens carry visual information, and that reweighting the next-token-prediction loss by $w_i = (1 - p_r(t_i|t_{<i}))^\\alpha$ makes vision-language pre-training more sample-efficient and less prone to hallucination. The paper frames this as importance sampling with a target distribution that upweights tokens the reference model finds surprising, then normalizes the weights across the caption and rescales by $k$ to preserve loss magnitude. The paper reports an 18.61% average relative gain for encoder-based LVLMs and 7.93% for encoder-free LVLMs at the end of 5,000 training steps, and higher scaling-law exponents $\\alpha_D$ than plain NTP.","pith_inferences":["A reader might infer that the same offline surprisal scores could be reused as a data-filtering or caption-rewriting signal, not just a loss weight, to clean noisy web-scale corpora before training.","The tokenizer-sharing limitation noted in the paper suggests a tokenizer-agnostic scoring scheme would make the weighted dataset portable across LVLM families.","The paper's own motivating example contains tokens such as a house address and price that are hard to predict from text yet are precisely the tokens the method intends to de-emphasize, so part of PRIOR's gain may come from variance reduction rather than from the stated image-relatedness story."],"forward_implications":["Because the importance scores are computed offline and stored with the image-caption tuples, PRIOR can be dropped into existing NTP pipelines with minimal code changes and no extra reference-model inference during LVLM training.","The reweighting works for both encoder-based and encoder-free LVLM architectures, so it is not tied to a particular design choice about visual encoders.","PRIOR lowers loss on both image-related and image-unrelated tokens, which the paper reads as faster overall pre-training rather than a narrow gain on a single token class.","The reported scaling-law exponents imply PRIOR turns additional data and compute into downstream benchmark performance more efficiently than NTP."],"supporting_citations":[{"why":"Supplies the 120M image-caption pairs used for reference-model training, LVLM pre-training, and the human annotation that only 31.3% of words are image-related.","marker":"[85]"},{"why":"Defines the next-token prediction objective over captions conditioned on images that PRIOR reweights.","marker":"[56]"},{"why":"Provides the backbone and tokenizer for the text-only reference LLM whose probabilities define the importance scores.","marker":"[23]"},{"why":"Establishes the encoder-free, single-transformer LVLM setting and the post-training data used for evaluation.","marker":"[15]"},{"why":"Gives the power-law form $L(D) = (D/D_c)^{\\alpha_D}$ used to compare scaling behavior.","marker":"[35]"},{"why":"Provides the CLIP vision encoder used in the encoder-based LVLM experiments.","marker":"[66]"}],"fun_headline_variants":["Surprise-weighted tokens improve vision-language pre-training by 19%","Text-only LLM surprise reweights caption tokens for better LVLMs","PRIOR: Weight caption tokens by text-only LLM surprise, cut hallucination","Reweight image-related tokens via LLM surprise: 19% gain in LVLMs","Vision-language pre-training: prioritize surprising tokens to beat NTP"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method stands or falls on the assumption that a token the text-only model struggles to predict is a token whose content comes from the image; the paper does not validate this mapping against human judgments of image-relatedness.","fun_headline_variants_meta":{"raw":{"variants":["Surprise-weighted tokens improve vision-language pre-training by 19%","Text-only LLM surprise reweights caption tokens for better LVLMs","PRIOR: Weight caption tokens by text-only LLM surprise, cut hallucination","Reweight image-related tokens via LLM surprise: 19% gain in LVLMs","Vision-language pre-training: prioritize surprising tokens to beat NTP"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000814,"raw_usage":{"total_tokens":3591,"prompt_tokens":994,"completion_tokens":2597,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":610,"completion_tokens_details":{"reasoning_tokens":2497}},"tokens_in":610,"tokens_out":2597,"duration_ms":18817,"temperature":1.0,"reasoning_tokens":2497,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:43:58.882922+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Annotate a sample of captions for human-rated image-relatedness and compute the correlation with $w_i = (1-p_r(t_i|t_{<i}))^\\alpha$; if the correlation is not positive, or if an equal-strength reweighting with shuffled weights reproduces the gains, PRIOR's stated mechanism is not what drives the improvement.","supporting_citations":[{"cited_title":"Visual instruction tuning","cited_arxiv_id":null,"evidence_quote":"Defines the next-token prediction objective over captions conditioned on images that PRIOR reweights."}],"review_version":1}