{"id":"3f2aebc9-5e80-4f8f-9abf-5ef27fb77559","arxiv_id":"2506.15649","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A two-stage, value-guided decoding strategy with a margin-based reward adjustment is claimed to yield more faithful, detailed VLM captions at about a quarter of VisVM's inference cost.","lead":"ViMaR is a two-stage decoding method that uses a learned value model to pick the best full image caption and then refines only the weakest sentences, claiming roughly 4x less inference cost than the prior VisVM approach while reducing hallucinated details. It matters because it proposes a practical, model-agnostic way to make vision-language models more faithful and detailed without retraining the captioner itself.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (1) inverts the margin signal: for any δ<τ it assigns a positive reward τ−δ that grows as grounding confidence falls, directly contradicting the stated 'negative penalty' and the TD objective that drives value-guided search. The central mechanism is therefore not correctly specified.","rationale":"The paper's central claim is that margin-based reward adjustment drives hallucination reduction. This requires Eq. (1) to penalize low-confidence candidates; it does the opposite. This is not a stylistic issue: Eq. (2) trains the value model on r_si as immediate reward, and Algorithm 1 selects argmax Vρ in both stages. A value model that assigns higher predicted value to low-CLIP-similarity states will steer search toward weakly grounded captions, opposite to the reported CHAIR/MMHal gains. I therefore cannot treat the abstract's 'calibrated margin-based penalty' as correctly specified. The reader's rationale already flagged this exact sign problem, and their REJECT verdict is appropriate. I did not select the CLIP-proxy concern as primary because Eq. (1) fails even under the most favorable assumption about CLIP; the sign error is an internal inconsistency, not a disagreement with external consensus. The train/eval overlap on COCO Train2017 and the unreconstructable 15.87% average in Table 2 are additional supporting issues, but the Eq. (1) sign is the load-bearing break in the argument. On the credit side, the two-stage 'select-then-refine' framing and the speedup claim are clear and potentially useful; the paper also gives concrete Table 1 numbers and qualitative figures. However, without code or a sign correction, the central mechanism cannot be verified. My concrete check is designed to distinguish a typographical error from an actual implementation flaw.","tokens_in":15288,"tokens_out":5112,"duration_ms":60671,"concrete_test":"Release or inspect the training code and print r_si for training triplets with δ<τ. Then run a controlled comparison of three value models on COCO Val2017: (a) Eq. (1) as written, (b) corrected negative margin r=δ−τ for δ<τ, (c) no margin (r=δ). If (b) reproduces Table 1's CHAIR_S≈23.1/MMHal≈0.35 improvements while (a) does not, the paper's stated mechanism depends on a sign correction; if (a) also improves, the reported gains cannot be explained by the margin reward as defined; if neither improves, the central claim lacks support.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Eq. (1) defines r_si = δ if δ≥τ, and r_si = τ−δ otherwise. Since δ<τ in the second branch, r_si is positive and monotonically increasing as CLIP similarity decreases (e.g., δ=0.10, τ=0.16 gives +0.06; δ=0.14 gives +0.02). Sections 3.1 and Appendix D repeatedly describe a 'negative penalty proportional to the margin gap', i.e., a reward that decreases for weakly grounded candidates. As written, the reward increases for exactly those candidates. Because this r_si enters the TD target in Eq. (2), the value model Vρ is trained to assign higher long-term value to low-confidence, potentially hallucinated continuations; stage 1 and stage 2 both select by argmax Vρ, so low-grounding sentences would be preferred. Thus the margin reward, the paper's stated novelty and claimed driver of hallucination reduction, has the opposite sign of what is claimed. Without released code, one cannot tell whether the implementation uses τ−δ or the intended negative gap (e.g., δ−τ), but as submitted the central mechanism is internally inconsistent. This concern is prior to the proxy-quality question: even if CLIP similarity is a perfect grounding signal, Eq. (1) as printed rewards the wrong states.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ViMaR, a two-stage inference-time value-guided decoding strategy for vision-language model captioning. In the first stage, a temporal-difference value model scores N×K full-caption candidates generated at different temperatures and selects the highest-value caption; in the second stage, only under-grounded or missing segments are resampled and rescored, which the authors claim yields over 4× speedup over the VisVM baseline. The value model is trained on a CLIP-similarity reward with a margin-based adjustment intended to penalize low-confidence continuations. The paper reports reduced hallucination (CHAIR, MMHal) and improved caption quality by human and GPT-4o judgments, plus self-training gains on eight visual-comprehension benchmarks and cross-model generalization to LLaVA-OneVision-Qwen2-7B.","tokens_in":15376,"tokens_out":5092,"duration_ms":63733,"significance":"If the mechanism worked as described, the paper would make a useful contribution: it offers an explicit two-stage inference procedure, reports human and GPT-4o preference studies, evaluates hallucination with standard metrics, demonstrates cross-model transfer of the value model, and shows that ViMaR-generated captions can be used for self-training. These are concrete, falsifiable empirical claims. However, the central novelty of the paper, the margin-based reward adjustment, is mis-specified as printed in Eq. (1), and this error propagates through the TD training objective and both stages of inference. Because the paper does not release code or checkpoints, the reader cannot determine whether the implemented reward matches the intended negative penalty or the printed positive one. The significance of the empirical results is therefore conditional on a corrected and verifiable specification.","major_comments":[{"comment":"The reward defined in Eq. (1) is r_si = δ when δ ≥ τ, and r_si = τ − δ otherwise. For any δ < τ, this second branch is positive and grows as the CLIP similarity δ decreases; for example, with τ = 0.16, δ = 0.10 gives +0.06 while δ = 0.14 gives +0.02. This is the opposite of the 'negative penalty proportional to the margin gap' described in Section 3.1 and Appendix D. Since r_si enters the TD target in Eq. (2), the value model Vρ is trained to assign higher long-term value to low-confidence, weakly grounded states, and both stages of Algorithm 1 select candidates by argmax Vρ. As printed, the proposed margin mechanism would prefer hallucinated or under-grounded continuations, contradicting the paper's central claim. Please correct Eq. (1) (the intended second branch appears to be δ − τ), update all corresponding text in Appendix D, retrain and rerun the experiments, and release code so the implemented sign can be verified.","section":"Section 3.1, Eq. (1); Appendix D"},{"comment":"Stage 2 is not formally specified. The text says to 'identify under-grounded or missing visual regions' and to refine 'segments that lack sufficient visual grounding', but no concrete criterion is given for deciding which segment yi is under-grounded, how many refinement rounds are performed, or when the loop in Algorithm 1 terminates ('while Generation is not Done', 'until all salient content is addressed'). The efficiency claim of 108 s per sample and the comparison with VisVM depend critically on this stopping rule. Please provide a precise, reproducible definition of the under-grounding test and the termination condition, or report the empirical distribution of refinement rounds.","section":"Section 3.2, Algorithm 1"},{"comment":"The LLaVA-OneVision-Qwen row reports ViMaR hallucination results (CHAIR_s 15.3, CHAIR_i 3.0, MMHal 3.96, MMHal rate 0.34) but the corresponding 'Original' row has dashes for all hallucination metrics. As a result, the claim that ViMaR reduces hallucinations in the stronger unseen model is not supported by the table. Please provide the baseline hallucination measurements for LLaVA-OneVision-Qwen under greedy decoding or the same inference scheme, or restrict the cross-model claim to the visual-comprehension benchmarks where baselines are present.","section":"Table 2, cross-model generalization"}],"minor_comments":[{"comment":"The text twice writes 'CLP-PRM' instead of 'CLIP-PRM'.","section":"Section 4.1, Table 1 caption and text"},{"comment":"The sentence 'In all experiments, we kept the total decode calls identical to ensure a fair comparison of quality versus compute' is unclear: ViMaR uses fewer value-model scoring calls by design, so 'total decode calls' cannot be identical across methods. Please clarify which quantity was controlled (e.g., number of sampled sentences, number of CLIP evaluations, or wall-clock budget).","section":"Section 4.1, paragraph on fairness"},{"comment":"The text describes local and global value scores (e.g., 2.285, 2.342, 2.254) but does not state the units or the discount factor γ used to compute them; adding this detail would make the case study reproducible.","section":"Section 5 and Figure 2"},{"comment":"The training details say the model was fine-tuned 'on the LLAVA dataset using the provided train and test splits', but the main text says training samples come from COCO 2017 with LLaVA-150K prompts. Please state which dataset and splits were actually used for the value-model training and for self-training.","section":"Appendix C"}],"recommendation":"major_revision","confidential_remarks":"The Eq. (1) sign issue is the central technical problem and must be resolved before the paper can be considered further. If the implementation uses the intended δ − τ branch, the paper is salvageable, but code or a clear statement of the implemented reward is necessary. I would also ask the authors to report the self-training average improvement in a way that does not mix differently scaled metrics: the 15.87% figure combines normalized CHAIR/MMHal scores with benchmark percentages, which can overstate the gain."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis one deserves a careful look before you decide to desk reject. The two-stage idea—select a full caption by value, then refine only under-grounded segments—is a real simplification over VisVM's per-step scoring, and the cross-model generalization experiment (a value model trained on LLaVA-Mistral applied to Qwen) is genuinely interesting. The 4x speedup claim is plausible on its face, and the paper is clearly written.\n\nThe problem is that the central mechanism is mis-specified as printed. Eq. (1) defines r_si = tau - delta for delta < tau, which is positive and grows as the CLIP similarity falls. The text and Appendix D repeatedly say this is a 'negative penalty proportional to the margin gap.' Those two things cannot both be true. Since r enters the TD target in Eq. (2), and both stages of search maximize the resulting value model, the printed equation would reward the exact low-confidence continuations the paper says it penalizes. This is not a proxy-quality debate; even if CLIP similarity were a perfect grounding signal, the equation contradicts the stated objective. It is likely a sign typo, but without code the reader cannot tell, and the central claim as submitted is unsupported.\n\nThere is also a train/test overlap issue. The human and GPT-4o win-rate evaluations in Section 4.1 use 1,000 images from COCO Train2017, the same split used to train the value model on 23K images. The hallucination numbers in Table 1 are on Val2017, so those are clean, but the headline preference numbers may be inflated. The self-training aggregate gain of 15.87% does not recompute from the per-benchmark numbers with the stated normalization, and two Qwen baseline cells are missing. All of that is fixable.\n\nThe paper does earn credit: the two-stage formulation is novel relative to VisVM, the cross-model transfer is a useful data point, and the self-training pipeline is a natural extension. If the sign error is corrected and the evaluation is moved to non-overlapping data, this could be a solid incremental systems paper. As submitted, the central claim does not hold.\n\nMy recommendation: do not desk reject. Send it to review, but the first-round decision should be major revision. A referee needs to see the corrected reward equation, a check that the implementation matches it, and a re-run of the preference evaluations on a non-overlapping split.","headline":"A genuinely useful two-stage search idea, but the paper's central margin reward equation is written backwards, and the main caption-quality evaluation overlaps the training split.","tokens_in":16132,"tokens_out":3311,"would_cite":false,"duration_ms":35648,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ViMaR claims that a two-stage search guided by a temporal-difference value model with a margin-based CLIP penalty produces more faithful, more detailed VLM captions at over four times lower inference cost than the VisVM baseline.","keywords":["vision-language models","inference-time search","value model","temporal-difference learning","margin-based reward","hallucination mitigation","image captioning","self-training"],"falsifier":"Take a held-out COCO set with ground-truth object annotations, compute CLIP-ViT similarity for each candidate sentence, and check whether sentences with delta below tau are more likely to contain hallucinated objects; if the correlation is absent or reversed, the margin mechanism cannot be the cause of the reported hallucination gains.","tokens_in":14891,"feed_emoji":"🖼️","tokens_out":6207,"duration_ms":65234,"temperature":0.7,"pith_summary":"The paper claims that VLM hallucination during descriptive captioning can be reduced without slowing generation by replacing exhaustive stepwise rescoring with a two-stage search: first pick the best whole caption from a sampled batch, then re-generate only the weakly grounded sentences. The decisive addition is a margin-based reward adjustment in value-model training, where candidates whose CLIP similarity falls below a calibrated threshold are supposed to be penalized proportionally to the gap, steering the value model away from low-confidence continuations. If correct, the approach would make inference-time value guidance practical: roughly four times faster than the existing VisVM search while improving CHAIR and MMHal hallucination scores, and it would transfer to a stronger unseen VLM without retraining. The paper also claims that captions produced this way serve as supervision that improves the base model by 15.87% on average across eight visual benchmarks.","feed_headline":"Two-stage search cuts caption hallucinations and runs 4× faster","feed_subtitle":"A margin-based value model guides VLMs to grounded, detailed captions and transfers to unseen models.","key_machinery":"The central object is the value model $V_\\rho$, a scalar head attached to the penultimate transformer layer of LLaVA-Next-Mistral-7B and trained by temporal-difference learning on (current sentence, next sentence, image) triplets. Its reward signal is the margin-adjusted CLIP similarity $r_{s_i} = \\delta$ if $\\delta \\geq \\tau$ and $\\tau - \\delta$ otherwise, with $\\tau = 0.16$ calibrated to the low end of the CLIP score distribution; the paper's stated intent is that weakly grounded, low-CLIP candidates receive a penalty proportional to how far they fall below the margin. The search machinery is the two-stage procedure: a holistic best-of pass over $N \\times K$ temperature-sampled full captions, followed by targeted resampling and rescoring of only under-grounded segments, which removes the per-step full-candidate rescoring that makes VisVM slow.","core_discovery":"The paper's central claim is that ViMaR, a two-stage inference-time decoding framework built on a temporal-difference value model, generates captions that are more reliable, factually accurate, detailed, and explanatory than prior value-guided search while running substantially faster. Trained on sentence-level transitions from COCO and LLaVA-150K data with CLIP-ViT similarity as the process reward, the value model scores whole captions and candidate sentences; at inference, Stage 1 samples paragraph candidates over multiple temperatures, scores them holistically, and selects the highest-value caption, and Stage 2 resamples only segments with weak grounding. The paper reports hallucination improvements over VisVM-guided search (CHAIR_S 23.1 vs 26.2, MMHal rate 0.35 vs 0.39) at 108 seconds per sample versus 462 seconds, and it demonstrates that a value model trained only on LLaVA-Mistral-7B can guide a stronger unseen model, LLaVA-OneVision-Qwen2-7B, to consistent gains. It further claims that ViMaR-generated captions, when used as supervised fine-tuning data, lift the base LLaVA-Next-7B model by an average of 15.87% across eight visual comprehension and hallucination benchmarks.","pith_inferences":["The paper does not ablate the margin penalty from the two-stage search, so the reported gains should be read as the joint effect of the value model, holistic reranking, and targeted refinement, not as proof that the margin term alone is responsible.","Equation (1) as printed assigns a positive reward tau - delta to low-confidence sentences, which is the opposite of a penalty; if the implementation matches the printed formula, the value model would be trained to prefer low-CLIP candidates and the stated mechanism would need reinterpretation.","The threshold tau = 0.16 is calibrated to the CLIP score distribution of COCO and LLaVA-150K; applying ViMaR to datasets with different CLIP score scales would likely require recalibrating tau, so cross-model generalization should be tested for sensitivity to this choice.","The self-training result suggests a closed loop in which ViMaR-generated captions improve the base VLM, plausibly enabling another round of value-model training and search on the improved model; testing whether multiple rounds compound would clarify the scalability claim."],"forward_implications":["ViMaR-guided decoding cuts hallucination on the COCO Val2017 evaluation: CHAIR_S drops to 23.1 from VisVM's 26.2 and the MMHal rate to 0.35 from 0.39, while average per-sample time falls from 462 seconds to 108 seconds.","A value model trained solely on LLaVA-Mistral-7B data can be attached to a stronger unseen VLM, LLaVA-OneVision-Qwen2-7B, and still improve every reported benchmark.","Captions generated by ViMaR can be used as supervised fine-tuning data, yielding a 15.87% average improvement over the base LLaVA-Next-7B across eight visual comprehension and hallucination benchmarks.","A single holistic best-of pass plus targeted refinement replaces stepwise rescoring of every candidate, so the quality gains do not require the O(N x S) inference overhead of VisVM-style search."],"supporting_citations":[{"why":"Supplies the VisVM baseline, its temporal-difference value-model design, and the comparison numbers and evaluation setup that ViMaR extends.","marker":"[44]"},{"why":"Provides the temporal-difference learning objective used to train ViMaR's value model.","marker":"[35]"},{"why":"Defines the CHAIR hallucination metrics used for the paper's central evaluation.","marker":"[30]"},{"why":"Supplies the MMHal benchmark and scoring used to measure hallucination rates.","marker":"[34]"},{"why":"The base VLM architecture on which the value head is initialized and the policy that both ViMaR and baselines decode from.","marker":"[23]"},{"why":"Establishes CLIP-based scoring as a proxy reward for VLMs, the basis of the process reward model that ViMaR modifies.","marker":"[53]"},{"why":"Represents the process-reward-guided tree search that motivates value-guided inference and is contrasted on computational cost.","marker":"[52]"}],"fun_headline_variants":["ViMaR: 4× faster VLM captions with fewer hallucinations","Two-stage value search improves VLM caption fidelity 4× faster","Margin-based reward guides faithful VLM captions at 4× speed","ViMaR transfers across VLMs: 4× faster, grounded captions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a sentence's CLIP-ViT similarity to the image is a valid and monotonic proxy for visual grounding, so steering the value model away from low-similarity continuations suppresses hallucinations rather than punishing descriptive detail or rewarding generic text.","fun_headline_variants_meta":{"raw":{"variants":["ViMaR: 4× faster VLM captions with fewer hallucinations","Two-stage value search improves VLM caption fidelity 4× faster","Margin-based reward guides faithful VLM captions at 4× speed","ViMaR transfers across VLMs: 4× faster, grounded captions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000649,"raw_usage":{"total_tokens":3072,"prompt_tokens":1131,"completion_tokens":1941,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":747,"completion_tokens_details":{"reasoning_tokens":1860}},"tokens_in":747,"tokens_out":1941,"duration_ms":17098,"temperature":1.0,"reasoning_tokens":1860,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:52:28.940145+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a held-out COCO set with ground-truth object annotations, compute CLIP-ViT similarity for each candidate sentence, and check whether sentences with delta below tau are more likely to contain hallucinated objects; if the correlation is absent or reversed, the margin mechanism cannot be the cause of the reported hallucination gains.","supporting_citations":[{"cited_title":"Learning to predict by the methods of temporal differences.Machine learning, 3:9–44, 1988","cited_arxiv_id":null,"evidence_quote":"Provides the temporal-difference learning objective used to train ViMaR's value model."}],"review_version":1}