{"id":"2a91ce39-71ed-44c6-ae02-f110f09229d9","arxiv_id":"2508.11616","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"MRGD guides MLLM decoding with a learned hallucination reward and a detector-based recall reward, allowing users to trade off object precision, recall, and test-time compute while reducing object hallucinations on CHAIR and AMBER.","lead":"A new decoding method guides multimodal language models with two reward signals, one for object precision and one for object recall, during caption generation. Users can trade precision against recall and search effort against visual grounding, and measured hallucination rates drop substantially.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Potential train/eval overlap: rhal is trained on SugarCrepe, built from COCO images, and evaluated on COCO-based CHAIR/AMBER with no reported disjoint split; this could inflate the headline hallucination reductions.","rationale":"The reader identified partial-response reward noise as the weakest assumption, but the more fundamental threat is potential train/eval overlap. The paper reports training rhal on SugarCrepe, a benchmark built from COCO images, and evaluating on COCO CHAIR and AMBER—both COCO-derived—without any explicit disjoint-split statement. If overlap exists, the headline ~70% CHAIR_i reduction could be an artifact of the reward model having seen the test images. This is load-bearing because the 'consistently outperforming' claim is the paper's central contribution. The reader's concerns about test-set hyperparameter selection and missing error bars are valid but secondary: they affect the strength of the comparison, whereas data leakage would invalidate the comparison entirely. I therefore recommend that the verdict be 'unverified' pending the overlap check, rather than 'conditional' on the originally stated issues. The proposed concrete test is straightforward and decisive: if the image sets are disjoint, the concern is resolved and the paper's empirical case stands on its original merits; if they overlap, the main tables must be recomputed.","tokens_in":18348,"tokens_out":7490,"duration_ms":87631,"concrete_test":"Compute the intersection of image IDs in rhal's training data (especially the SugarCrepe 80% split, and also LLaVA-RLHF/RLHF-V/POVID images) with the image IDs used in the COCO CHAIR and AMBER evaluation sets. If the intersection is non-empty, retrain rhal on a split that excludes all overlapping images (or evaluate only on disjoint images) and recompute Tables 1 and 2. If the gap vs. greedy decoding and vs. CGD shrinks materially (e.g., CHAIR_i reduction falls below 50%), the 'consistently outperforming' claim is not supported.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"The core empirical claim—that MRGD 'consistently outperforms' existing hallucination mitigation methods—rests on rhal transferring to unseen images. Section 3.1.1 trains rhal on 30.6k preference examples, including 7.5k from SugarCrepe. SugarCrepe is constructed from COCO images, and the evaluation benchmarks COCO CHAIR (5k) and AMBER (1k) are also COCO-based. The paper does not report any exclusion of overlapping images between rhal's training data and the evaluation sets. If the SugarCrepe images (or those from LLaVA-RLHF, RLHF-V, POVID) overlap with the CHAIR/AMBER test images, rhal can exploit memorized object-caption associations, inflating MRGD's precision and recall gains exactly on the evaluation benchmarks. The ablations in Table 3 do not address this, and the cross-model results in Table 2 use the same rhal. Thus the central comparative claim is not yet established; it requires an overlap analysis before the reported improvements can be trusted.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces MRGD, a reward-guided decoding method for multimodal LLMs. Two reward models are built: rhal, a PaliGemma-based reward model trained with Bradley-Terry and MSE losses on preference data from LLaVA-RLHF, RLHF-V, POVID, and a repurposed SugarCrepe split, and rrec, composed from OWLv2, Sentence-BERT, and NLTK to estimate object recall. At inference, candidate sentence-level completions are sampled and selected by maximizing w·rhal + (1−w)·rrec, giving a precision/recall control knob (w) and a compute/grounding knob (k, T). Experiments on CHAIR and AMBER for LLaVA-1.5, Llama-3.2-Vision, and SmolVLM-2 report substantial hallucination reductions, e.g., CHAIR_i on COCO dropping from 15.05 to 4.53 at w=1, with controllability across w and improved performance over CGD and several fine-tuning baselines.","tokens_in":18655,"tokens_out":4236,"duration_ms":50283,"significance":"If the empirical claims hold, the method is a practically valuable, training-free-at-inference mechanism for adapting existing MLLMs: it provides two user-controllable trade-offs (precision vs. recall, and compute vs. grounding) and transfers reward models across three architecturally different base MLLMs without retraining. The paper is also careful in several respects: it evaluates rhal on held-out VLFeedback data, ablates the preference-data mix and reward-model components, reports latency, and includes qualitative examples. These strengths make the central idea worth pursuing. However, the headline comparative claims are currently supported by an insufficiently specified hyperparameter-selection procedure and by a potential train/evaluation image overlap, so the quantitative results need additional validation before they can be accepted as stated.","major_comments":[{"comment":"The MRGD rows are reported as 'the best performing variant w.r.t. the number of samples k, the reward evaluation period T, and the temperature t,' but no validation procedure is described. The central comparison—e.g., the ~70% CHAIR_i reduction from 15.05 to 4.53—therefore conflates method performance with test-set hyperparameter selection. Since k and T are core controllability parameters, the authors should either report a single fixed configuration used for all rows (e.g., k=30, T=1, t=1.0, as in Table 2 and Figure 2) or provide a validation-based selection protocol and report the chosen values for every w and benchmark.","section":"§4.3, Table 1"},{"comment":"rhal is trained on 7.5k SugarCrepe examples, and SugarCrepe is constructed from COCO images; the evaluation benchmarks CHAIR and AMBER are also COCO-based. The paper does not report any overlap analysis or disjoint split between rhal's training images and the evaluation images. If overlapping images appear, rhal can memorize object–caption associations and inflate exactly the hallucination reductions used to support the main claim. This is a load-bearing concern, not a minor one: please quantify the overlap and rerun the main comparisons with any overlapping examples removed, or evaluate on a held-out COCO split not used in any rhal training set.","section":"§3.1.1, §4.1, Tables 1–2"},{"comment":"rhal is trained on complete chosen/rejected responses, but MRGD with T=1 scores partial sentence-level completions. The paper acknowledges this issue in §3.2 but provides no empirical check that the reward model's ranking quality transfers to incomplete prefixes; the reported VLFeedback accuracy (67.68%) is measured on complete responses. Since the compute-efficiency claim (T=1 being more sample-efficient than rejection sampling) depends directly on this transfer, please add a partial-prefix validation — e.g., rank accuracy on truncated held-out responses as a function of sentence index — or compare T=1 against T=∞ at matched total sample counts.","section":"§3.2, Algorithm 1, Figure 2"}],"minor_comments":[{"comment":"The naming of the preference datasets is inconsistent: the paper alternately uses 'RLAIF-V' and 'RLHF-V' (Refs. [50] and [49]). Please unify the names and ensure Table 3's 'RLAIF-V' rows are unambiguous.","section":"§4.1, §4.7, Table 3"},{"comment":"The notation in Eq. (2) drops the dependence on x_v and x_q that is explicit in Eq. (1). Also, the parenthetical in the MSE terms is slightly awkward; consider rewriting for clarity.","section":"§3.1.1, Eq. (2)"},{"comment":"The claim that 'CGD can be viewed as a particular instance of MRGD' is instructive, but CGD's SigLIP scores are normalized across candidates while rhal scores are trained to a [0,1] scale; the comparison would benefit from a one-sentence discussion of whether normalization affects the combined score in Eq. (4).","section":"Appendix 6.4"},{"comment":"The left and right panels use different y-axis scales and the marker for T=∞ (rejection sampling) is not shown; adding explicit markers and a legend for T values would improve readability.","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The overlap concern is the one I would most want the editor to track: because SugarCrepe, CHAIR, and AMBER share the COCO image source, the burden is on the authors to demonstrate disjointness. The hyperparameter-selection issue is also central and fixable. If a revision provides a clean overlap analysis and a validation protocol, the paper could become a solid contribution; as it stands, the quantitative comparison is not yet trustworthy."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"MRGD is a real contribution to inference-time control for MLLMs, but the headline gains come with two caveats that need fixing before I'd trust the comparative claims: the main table selects the best k, T, and t on the test set without a validation split, and the reward model's training data includes SugarCrepe, which is built from COCO images—the same image pool used by CHAIR and AMBER.\n\nWhat's actually new: previous guided-decoding work for MLLMs (CGD) used an off-the-shelf CLIP score. MRGD trains a multimodal reward model on preference data and combines it with a second, recall-oriented reward assembled from off-the-shelf components (OWLv2, Sentence-BERT, NLTK). The continuous weight w and the search breadth k / evaluation period T give the user two real dials. That's a useful step beyond fine-tuning or prompting, and the cross-model results on Llama-3.2-Vision and SmolVLM-2 show the reward transfers without retraining.\n\nWhat it does well: the evaluation is more thorough than most papers in this subfield—three base models, two benchmarks, ablations on reward backbone, detector, threshold, and data mix, plus latency numbers. The authors also report the prompting baseline and rerun prior methods where the original papers didn't report captioning numbers. The limitations paragraph is honest: the method targets object-level hallucination in captioning, not attributes or relations. Credit where due.\n\nSoft spots, in order of severity. First, Table 1 cherry-picks k, T, and t as the best-performing variant on the evaluation benchmarks, with no described validation procedure. That makes the 70% CHAIR_i reduction look optimistic, and the comparison to baselines is not apples-to-apples: the baselines are run at their default settings while MRGD gets the best of several settings. This must be addressed, either by fixing hyperparameters on a validation set or by reporting the whole grid. Second, no error bars or seed variance anywhere. Given that the method samples k candidate sentences and picks the best, stochasticity is nontrivial; the reported differences between w values may be within noise. Third, the SugarCrepe overlap: 7.5k of the 30.6k preference examples come from SugarCrepe, which is built from COCO, and CHAIR/AMBER use COCO images. The paper doesn't report whether any of those images appear in both. If they do, rhal could memorize object-caption associations for exactly the test images, inflating the gains. The cross-model consistency makes me think the effect is real, but the authors need to check and report disjointness, or the central claim stays unproven. Fourth, the reward model is trained on complete responses and applied to partial sentence completions; the paper acknowledges this, and the VLFeedback accuracy of 67.68% is modest. Minor but worth a caveat.\n\nBottom line: the controllability story holds up on the trends, and the precision/recall and compute/grounding dials are valuable. The comparative claim needs stronger support. The paper deserves a serious referee; I'd send it to review with required revisions: report a validation-based hyperparameter selection, add error bars, rule out image overlap, release code and checkpoints.\n\nFor whom: anyone working on MLLM alignment, hallucination mitigation, or test-time scaling. I'd bring it to a reading group. I'd cite it as a baseline for inference-time control.","headline":"A genuinely useful inference-time control method for MLLMs, but the headline comparisons need stronger evaluation hygiene before the claims are fully trusted.","tokens_in":19139,"tokens_out":4036,"would_cite":true,"duration_ms":42063,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Reward-guided decoding lets users dial an MLLM's object precision against recall at inference time, cutting hallucinations about 70%.","keywords":["reward-guided decoding","multimodal LLM","object hallucination","visual grounding","precision-recall trade-off","test-time compute","image captioning","preference reward model"],"falsifier":"Take a held-out set of images with ground-truth object annotations and a base MLLM not used in the paper; run MRGD with $w=1$ and increasing $k$, and compare the chosen partial responses' $r_{\\mathrm{hal}}$ scores with the hallucination rate of the finished captions. If ordering partial candidates by $r_{\\mathrm{hal}}$ does not predict which final caption has fewer hallucinated objects, the sentence-level guidance is not the cause of the reported reductions.","tokens_in":18264,"feed_emoji":"🎛️","tokens_out":7117,"duration_ms":72301,"temperature":0.7,"pith_summary":"This paper argues that the behavior of a multimodal LLM can be steered at inference time by searching over candidate sentence completions with a weighted reward signal. The authors build two small reward models—one trained from preference data to reward captions that name only objects actually in the image (object precision), one assembled from an object detector and word embeddings to reward captions that name objects present (object recall)—and combine them with a single weight $w$. On the COCO object-hallucination benchmark, the precision-guided setting ($w=1$) cuts the instance-level hallucination rate from 15.05% to 4.53%, while reducing object recall by only about 6.5%; a middle setting trades a little precision for substantially higher recall. The same reward models transfer to newer, architecturally different MLLMs without retraining, and varying the search breadth $k$ and evaluation period $T$ trades visual grounding against test-time compute.","feed_headline":"Reward-guided decoding cuts MLLM hallucinations by 70%","feed_subtitle":"A new inference-time dial trades object precision against recall in multimodal LLMs.","key_machinery":"The load-bearing mechanism is the weighted reward score $s = w\\,r_{\\mathrm{hal}} + (1-w)\\,r_{\\mathrm{rec}}$, used as the selection criterion in an iterative sentence-level search: every $T$ sentences, $k$ candidate continuations are sampled from the base MLLM and the one with the largest $s$ is appended to the context. The weight $w$ is the precision/recall dial; $k$ and $T$ are the compute/grounding dials. The search turns an off-the-shelf MLLM into an optimizer of a user-specified reward without changing its weights.","core_discovery":"The central discovery is that MLLM decoding can be made controllable on the fly through multimodal reward-guided decoding (MRGD), and that this controllability exposes an inherent precision/recall trade-off in the base models. MRGD samples $k$ candidate sentence-level completions from the current partial response and keeps the one maximizing $s(x_v,x_q,y)=w\\,r_{\\mathrm{hal}}(x_v,x_q,y)+(1-w)\\,r_{\\mathrm{rec}}(x_v,x_q,y)$, repeating until the end-of-sentence token. $r_{\\mathrm{hal}}$ is a PaliGemma-based reward model trained with a Bradley-Terry preference loss plus regression targets on a mix of LLaVA-RLHF, RLHF-V, POVID, and repurposed SugarCrepe data; $r_{\\mathrm{rec}}$ counts, via object","pith_inferences":["Editorial inference: if sentence-level reward scoring is reliable, the same search recipe should extend to other hallucination axes—attributes, counts, spatial relations—by training one reward model per axis and weighting them in the same $w$ dial.","Editorial inference: the method's ceiling is set by $r_{\\mathrm{hal}}$'s ability to score partial prefixes; a reward model trained specifically on truncated responses, or a value head trained on prefixes, would likely reduce the gap the paper acknowledges in Section 3.2.","Editorial inference: since $r_{\\mathrm{rec}}$ is assembled from off-the-shelf components, its detector's vocabulary and POS-tagger errors bound recall estimates; a more precise detector or a caption-aware reference set should sharpen the $w=0$ operating point.","Editorial inference: a testable extension is to run MRGD on discriminative hallucination tasks such as visual questioning, where the reward would have to judge answers rather than captions; the paper leaves this open in its limitations and future work."],"forward_implications":["A user can set the operating point per query: precision-first ($w$ near 1), recall-first ($w$ near 0), or balanced, with no retraining.","The same reward models apply to base MLLMs they were not trained with, making MRGD a plug-in inference-time alignment layer on top of instruction-tuned and RLHF-tuned models.","Because evaluating partial responses every $T$ sentences is more sample-efficient than best-of-$k$ over full responses, modest test-time budgets can buy most of the grounding gain.","MRGD combines with prompting and fine-tuning rather than replacing them, so existing hallucination mitigations remain usable underneath.","The observed precision/recall curve indicates an inherent trade-off in current MLLMs; more compute (larger $k$) shifts the whole curve toward the ideal operating point."],"supporting_citations":[{"why":"Supplies the PaliGemma backbone fine-tuned with LoRA to create the object-hallucination reward model $r_{\\mathrm{hal}}$.","marker":"[4]"},{"why":"Provides one of the preference datasets (LLaVA-RLHF) used to train $r_{\\mathrm{hal}}$.","marker":"[42]"},{"why":"Provides RLHF-V preference data used in the mix for $r_{\\mathrm{hal}}$.","marker":"[49]"},{"why":"Provides POVID preference data; the ablation shows removing it degrades performance.","marker":"[56]"},{"why":"Repurposed SugarCrepe examples supply additional preference pairs for $r_{\\mathrm{hal}}$.","marker":"[17]"},{"why":"OWLv2 open-vocabulary detector extracts reference objects for the recall reward $r_{\\mathrm{rec}}$.","marker":"[31]"},{"why":"Sentence-BERT word embeddings compute semantic similarity between detected and caption objects in $r_{\\mathrm{rec}}$.","marker":"[37]"},{"why":"LLaVA-1.5 7B is the main base MLLM whose greedy and guided decoding are compared.","marker":"[27]"},{"why":"Llama-3.2-Vision serves as the RLHF-tuned base model showing MRGD works on top of preference-aligned models.","marker":"[14]"},{"why":"CLIP-guided decoding (CGD) is the closest guided-decoding baseline MRGD must outperform.","marker":"[12]"}],"fun_headline_variants":["Tune object precision vs recall on the fly in MLLMs","Reward-guided decoding controls MLLM hallucinations","New dial for MLLM decoding: trade precision for recall","Steer multimodal LLM outputs with reward-guided decoding","MLLM decoding gets a precision-recall control knob"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The reward model trained on complete preferred/rejected caption pairs can also assign trustworthy scores to partial sentence prefixes, so selecting the best prefix every $T$ sentences actually improves the final caption.","fun_headline_variants_meta":{"raw":{"variants":["Tune object precision vs recall on the fly in MLLMs","Reward-guided decoding controls MLLM hallucinations","New dial for MLLM decoding: trade precision for recall","Steer multimodal LLM outputs with reward-guided decoding","MLLM decoding gets a precision-recall control knob"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000239,"raw_usage":{"total_tokens":1366,"prompt_tokens":771,"completion_tokens":595,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":515,"completion_tokens_details":{"reasoning_tokens":514}},"tokens_in":515,"tokens_out":595,"duration_ms":6779,"temperature":1.0,"reasoning_tokens":514,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T19:48:01.218996+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a held-out set of images with ground-truth object annotations and a base MLLM not used in the paper; run MRGD with $w=1$ and increasing $k$, and compare the chosen partial responses' $r_{\\mathrm{hal}}$ scores with the hallucination rate of the finished captions. If ordering partial candidates by $r_{\\mathrm{hal}}$ does not predict which final caption has fewer hallucinated objects, the sentence-level guidance is not the cause of the reported reductions.","supporting_citations":[{"cited_title":"Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional hu- man feedback","cited_arxiv_id":null,"evidence_quote":"Provides RLHF-V preference data used in the mix for $r_{\\mathrm{hal}}$."},{"cited_title":"Aligning modalities in vision large lan- guage models via preference fine-tuning","cited_arxiv_id":null,"evidence_quote":"Provides POVID preference data; the ablation shows removing it degrades performance."},{"cited_title":"Sugarcrepe: Fixing hackable benchmarks for vision-language compositionality","cited_arxiv_id":null,"evidence_quote":"Repurposed SugarCrepe examples supply additional preference pairs for $r_{\\mathrm{hal}}$."},{"cited_title":"Scaling open-vocabulary object detection","cited_arxiv_id":null,"evidence_quote":"OWLv2 open-vocabulary detector extracts reference objects for the recall reward $r_{\\mathrm{rec}}$."},{"cited_title":"Sentence-bert: Sentence embeddings using siamese bert-networks","cited_arxiv_id":null,"evidence_quote":"Sentence-BERT word embeddings compute semantic similarity between detected and caption objects in $r_{\\mathrm{rec}}$."},{"cited_title":"Improved baselines with visual instruction tuning","cited_arxiv_id":null,"evidence_quote":"LLaVA-1.5 7B is the main base MLLM whose greedy and guided decoding are compared."}],"review_version":1}