{"id":"420b6012-ef78-4fed-821a-b18452e3a9b6","arxiv_id":"2506.07963","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"SUDER uses the likelihood of reconstructing the original input from a sampled output as a self-reward, improving both understanding and generation in unified multimodal models without external supervision.","lead":"A new training method lets a single image-text model score its own outputs by checking how well each half can reconstruct the other, then improves using only those self-scores. The authors report large gains on text-to-image benchmarks and some improvement on visual question answering, without any human labels or external reward models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Dual self-reward is a self-referential training signal with no demonstrated correlation to external alignment; without a validation check, reported gains may reflect reward hacking rather than improved semantic alignment.","rationale":"The central claim is that a model's own dual likelihood is a sufficient self-supervision signal for improving vision-language alignment. The reader's weakest assumption correctly identifies this as the load-bearing premise. Everything in the experimental section depends on DSR selecting genuinely better outputs: if DSR is uncorrelated with true alignment or is gameable, SimPO/GRPO could improve benchmark scores through shortcut behaviors (e.g., increasing reverse-model confidence, reducing output diversity) while leaving true alignment unchanged. I agree with the reader's conditional verdict: the method is plausible and the paper provides some supporting evidence (ablations, CLIP comparison, gains on GenEval and LLaVA-Bench), but the missing validation of the reward itself is the critical gap. I also considered whether the T2I-CompBench numbers are contaminated by training on the benchmark's 5,600 prompts; that is a serious secondary concern worth checking, but the DSR reliability issue is more fundamental because it threatens the interpretation of all reported gains, not just one table. A correlation test between DSR and an external alignment score on held-out samples would settle the matter. If the correlation is high and diversity is preserved, the concern is resolved; if not, the paper's central claim is unsupported.","tokens_in":20809,"tokens_out":9901,"duration_ms":127467,"concrete_test":"On 200 held-out GenEval prompts, sample 8 images per prompt with the final Janus-Pro-7B+SUDER model; score each image with DSR and with CLIPScore (plus human ratings on a 100-pair subset) and compute the Spearman rank correlation between DSR and the external score within each prompt group. If the average correlation is not clearly positive (ρ < 0.3), the dual self-reward is not a reliable proxy for semantic alignment and the claimed improvements are not explained by the stated mechanism.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assumption is stated in §3.2: 'if a textual description accurately reflects the visual input, then the likelihood of generating the original image conditioned on this description should be high.' Both rewards in Eq. (3) are log-likelihoods of the original input under the model's own conditional distribution, R_U(Y_T|X_V) = (1/||X_V||) log π_θ(X_V|Y_T) and R_G(Y_V|X_T) = (1/||X_T||) log π_θ(X_T|Y_V), with π_θ the very model being optimized. This is a self-referential training signal: the model is rewarded for outputs that the current model finds easy to reverse. Optimizing such a signal can inflate reverse-direction confidence without improving true alignment. Concretely, the understanding branch can drift toward reconstructing the prompt from almost any image, which inflates R_G for arbitrary generated images; the generation branch can collapse to a narrow set of images that the understanding branch captions confidently. The paper offers only anecdotal support (Figs. 9-10), no correlation between DSR and external alignment judgments, no diversity/FID measurement, and no random-reward control. Since SimPO (Eq. 4) is reference-free, nothing anchors the reward to external semantics. If DSR is hackable, the reported benchmark gains do not establish the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents SUDER, a self-supervised optimization framework for unified large multimodal models (LMMs). The core idea is a dual self-reward (DSR): for a sampled response in one modality, the model computes the likelihood of reconstructing the original input in the other modality, using the very model being optimized (Eq. 3). DSR scores are then used with SimPO or GRPO to reinforce both visual understanding and text-to-image generation, either jointly in one model or alternately in two models. On Janus-Pro-7B and Janus-Pro-1B, the authors report average T2I-CompBench increases of 11.68% and 10.5%, an overall GenEval gain of 5% for Janus-Pro-7B, and improvements or stability on understanding benchmarks such as LLaVA-Bench and HallusionBench. An appendix extends the method to Show-o. The experiments use non-parallel image and text data and require no external reward model or human annotations.","tokens_in":21115,"tokens_out":10361,"duration_ms":120186,"significance":"If robust, SUDER would be an important result in self-supervised multimodal alignment: it turns the model's own inverse prediction into a label-free reward, is computable in a single forward pass, and shows consistent gains across two scales of Janus-Pro and, in the appendix, on Show-o. The authors are explicit about their central assumption in Section 3.2 and provide qualitative rankings in Figures 9 and 10. However, the paper currently lacks three pieces of evidence needed to support the central claim: a demonstration that DSR correlates with external alignment, a control against reward hacking, and any measure of statistical reliability (error bars, seeds, significance). The reported gains are consistent and plausible, but the self-referential nature of the reward is a real risk that is not yet addressed; the significance is therefore conditional.","major_comments":[{"comment":"The reward in Eq. (3) is the log-likelihood of the original input under the same conditional distribution π_θ that SimPO (Eq. 4) or GRPO (Eq. 7) subsequently optimizes. Because SimPO is reference-free, there is no external anchor preventing the model from increasing DSR by exploiting its own reversible-computation shortcuts: for example, the generation branch can drift toward images that the captioning head can reconstruct regardless of prompt fidelity, and the understanding branch can drift toward captions that make any image likely. The manuscript offers anecdotal examples (Figs. 9–10) and an indirect CLIP comparison (Table 5), but no correlation measure between DSR and human or CLIP alignment on held-out data, no random-reward control, and no diversity or FID metric. Please add at least: (i) a control that optimizes with rewards computed by a frozen or random model, and (ii) a rank correlation of DSR against CLIP or human judgments on a held-out set. Without this, the central claim that the self-supervised loop improves semantic alignment is not established.","section":"§3.2, Eq. (3); §3.3, Eq. (4)"},{"comment":"All benchmark numbers are reported as single point estimates with no standard deviations, number of seeds, or significance tests. Several gains are small enough to be within typical evaluation noise, e.g., Non-Spatial +1% for Janus-Pro-7B, HalluBench +1.9 points, and POPE −0.2 points in Table 3; even the headline +11.68% T2I average needs a variance estimate given the instability visible in the per-category numbers across scales. Please report mean and standard deviation over at least three independent training runs (or provide a justified single-run protocol), and state whether any hyperparameter selection (e.g., β, γ, clipping thresholds) was performed on the evaluation benchmarks.","section":"Tables 1–3 and §4.2–4.3"},{"comment":"The training data uses the 5,600-prompt training set of T2I-CompBench, while the evaluation uses the benchmark's 6,000 'original text prompts.' The paper does not state whether these sets are disjoint. If the evaluation includes training prompts, the T2I-CompBench gains are partially a memorization effect and do not measure generalization. Please state the exact split, and if no disjoint evaluation split exists, re-evaluate on a held-out compositional benchmark (e.g., GenEval or a filtered subset of T2I-CompBench) to certify generalization.","section":"§4.1, T2I-CompBench split"},{"comment":"The ablation shows that optimizing only the generation branch already raises LLaVA-Bench from 74.0 to 77.3 and HalluBench from 37.0 to 37.3, even though no understanding-specific training is performed. This is a substantial transfer effect and is not explained; it complicates the attribution of the understanding gains to the dual reward, since a single-branch control produces most of the LLaVA-Bench improvement seen in the unified model (81.5). Please investigate whether this transfer is due to shared parameters, to the shared training distribution, or to instability, and ideally include a control where generation is optimized with a random reward to distinguish the effect of DSR from generic fine-tuning.","section":"Table 4 and §4.4"}],"minor_comments":[{"comment":"The sentence 'The results can be seen in Figure 3, which shows that SimPO is slightly better than GRPO' should refer to Figure 4; Figure 3 is the separate-vs-unified strategy comparison.","section":"§4.5, Figure 3 vs Figure 4"},{"comment":"Equation (4) contains an editing leftover: the sentence 'where R(·|·) denotes the corresponding dual self-reward from Section 3.2' appears inside the SimPO objective, but R is never used in that objective; the sentence should be removed or integrated into the surrounding text.","section":"Eq. (4)"},{"comment":"The header 'LLaV ABench' contains an unintended space, and 'HallBench' / 'HalluBench' are used inconsistently across tables and text; please unify these names.","section":"Table 3"},{"comment":"Reference [31] is listed as 'Chameleon: Plug-and-play compositional reasoning...' by Lu et al., but the Chameleon model discussed in Tables 1–3 is the unified multimodal model; please cite the correct paper.","section":"Reference [31]"},{"comment":"The Show-o results show the method mostly improves attribute binding (about +10 points) while spatial and non-spatial relationships are essentially flat (−0.2 and −0.1); the discussion should acknowledge this concentration rather than claim a uniform gain.","section":"Appendix A.4, Table 6"},{"comment":"The manuscript does not include a code or model availability statement; please add one, since the reproducibility of the RL training details (e.g., sampling temperature, reward computation for normalized likelihoods) would otherwise be hard to assess.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The self-referential reward is the key technical risk, and the missing error bars and absence of code are compounding. I would be willing to accept after the authors provide a random-reward control, a DSR-to-external-alignment correlation analysis, and seed-averaged results. The paper's template still contains ACM placeholders (conference name, DOI) that should be cleaned before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know about this paper: it proposes using the model's own reverse-direction likelihood as a reward for both understanding and generation, and reports large gains on T2I-CompBench, GenEval, and LLaVA-Bench across Janus-Pro-7B/1B and a Show-o backbone. The dual self-reward idea is genuinely new—SILMM only self-VQAs generation, T2I-R1 uses rule-based rewards, CoMat uses an external image-to-text model. The framework is elegant and the unified training strategy is well ablated: optimizing both capabilities together outperforms optimizing either alone, and a comparison with CLIP shows DSR is at least competitive. The authors also provide solid training details and a bonus Show-o experiment, which helps generalize the claim.\n\nThe soft spot is exactly where the stress-test note lands. Both rewards in Eq. (3) are log-likelihoods of the original input under the same model being optimized. With SimPO being reference-free, nothing anchors the reward to external semantic alignment. The model could drift toward outputs that are easy to reverse—e.g., the generation branch collapsing to a narrow image distribution that the understanding branch captions confidently—without actually improving fidelity. The paper's only evidence against this is anecdotal (Figs. 9-10). There is no correlation between DSR and human or CLIP-based alignment judgments, no diversity or FID measurement, and no random-reward control. The Limitations section does not acknowledge this self-referential issue at all. Missing code and error bars do not help.\n\nStill, the improvements are consistent and the idea is worth testing seriously. The central assumption—that reverse likelihood is a good proxy for alignment—is plausible but unvalidated. A referee should ask for a validation set: compare DSR with external reward models, measure generation diversity, and run a control with shuffled or random rewards. If that holds, this becomes a useful self-supervised alignment recipe.\n\nThis paper is for researchers working on self-improving multimodal models, especially those wanting to avoid human feedback or external reward models. It deserves a serious referee, but the verdict should be conditional on the reward-validation analysis.","headline":"Clever dual self-reward idea with consistent reported gains, but the self-referential reward is unvalidated and could be reward hacking; worth refereeing, not yet believable.","tokens_in":21610,"tokens_out":1914,"would_cite":false,"duration_ms":26481,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SUDER claims unified multimodal models can improve both text-to-image generation and visual understanding using their own reversed input-output likelihoods as self-rewards, without any external supervision or paired data.","keywords":["large multimodal model","text-to-image generation","vision-language alignment","self-supervised learning","dual self-reward","preference optimization","visual understanding"],"falsifier":"Take a held-out set of prompts and images, rank candidate outputs by the dual self-reward, and compare those rankings against independent alignment judgments (human ratings or a strong external VQA/hallucination detector). If the dual self-reward's ranking does no better than chance at picking the output humans judge as more aligned, the training signal is not tracking alignment and the reported gains would not transfer to new data.","tokens_in":20638,"feed_emoji":"🔄","tokens_out":17188,"duration_ms":159323,"temperature":0.7,"pith_summary":"The paper claims that unified multimodal models—models that can both read and generate images—can improve their own performance on both abilities with no external feedback, human labels, or paired image-text data. SUDER treats image captioning and text-to-image generation as inverse tasks: it samples several candidate outputs for an input, then scores each candidate by asking the model how likely it is to regenerate the original input from that candidate. These scores become dual self-rewards that drive a preference-optimization or reinforcement-learning step. If the claim holds, a free self-supervised loop can reduce hallucinations in visual question answering and prompt-following failures in text-to-image generation, without any supervised alignment data.","feed_headline":"Dual self-reward boosts multimodal alignment with no external labels","feed_subtitle":"SUDER uses reverse likelihood as a free reward, improving generation and understanding benchmarks without human feedback.","key_machinery":"The dual self-reward (DSR) mechanism is the load-bearing object. For each sampled output, DSR computes the length-normalized log-likelihood of the model regenerating the original input when the output is used as the condition: $R_U(Y_T|X_V) = \\frac{1}{\\|X_V\\|}\\log\\pi_\\theta(X_V|Y_T)$ for a candidate caption in understanding, and $R_G(Y_V|X_T) = \\frac{1}{\\|X_T\\|}\\log\\pi_\\theta(X_T|Y_V)$ for a candidate image in generation. These reversed likelihoods serve as the rewards that select preferred candidates for SimPO or define group-normalized advantages for GRPO. The mechanism runs in a single forward pass, uses no external reward model, and length-normalization shields it from the length bias that affects reward-model scores.","core_discovery":"The central claim is that the duality between understanding and generation can be converted into a training signal, so the two tasks supervise each other. For a visual-understanding input, SUDER samples multiple captions for an image, reverses each pair, and computes the likelihood of regenerating the image given each caption; the caption with the higher reverse likelihood is treated as the more accurate one. For text-to-image generation, sampled images are scored by the likelihood of regenerating the original prompt given the image. Optimizing with these self-rewards improves Janus-Pro-7B by 11.68% on average on T2I-CompBench and 5% overall on GenEval, while improving LLaVA-Bench by 7.5%; the same mechanism transfers to a second backbone, Show-o, with a more than 5% average gain on T2I-CompBench. These gains are obtained without any external supervision or parallel image-text pairs in training.","pith_inferences":["The same reverse-likelihood loop could apply to any modality pair with usable inverse directions—audio-to-text, video-to-text, or code-and-execution pairs—since the signal is effectively the mutual information between the two representations; the paper names audio as an open direction.","The stated requirement that the backbone already have basic alignment implies a testable prediction: on weaker or less-aligned backbones the dual self-reward is a noisier proxy, so the gains should shrink as initial alignment drops.","The alternating two-model scheme resembles a self-play critic-generator game; if scaled, it may develop instabilities and mode collapse like other self-play systems, an issue the paper does not examine.","A risk the paper leaves implicit is reward hacking: the model could raise its reverse likelihood by producing outputs that are simply easy to reconstruct—generic images or templated captions—which the benchmark gains do not rule out on out-of-distribution prompts."],"forward_implications":["Text-to-image alignment improves measurably on compositional benchmarks, with the largest gains in attribute binding (color, shape, texture) across both tested backbones.","Jointly optimizing understanding and generation beats optimizing either alone, so the dual reward does not trade one capability against the other.","The framework works with two different optimizers (SimPO and GRPO) and two different architectures (Janus-Pro-7B/1B and Show-o), which points to the reward signal rather than a specific training recipe as the source of improvement.","A two-model variant that trains understanding and generation alternately, using each model's rewards to improve the other, outperforms the single-model version on both tasks."],"supporting_citations":[{"why":"Janus-Pro-7B and Janus-Pro-1B are the primary unified backbones that provide the dual understanding and generation capability the mechanism exploits.","marker":"[7]"},{"why":"T2I-CompBench supplies the 5,600 training prompts and is the compositional text-to-image benchmark on which the +11.68% average gain is measured.","marker":"[15]"},{"why":"GenEval is the object-focused text-to-image benchmark on which the tuned model gains 5% overall.","marker":"[14]"},{"why":"Online SimPO is the preference-optimization objective that turns the dual self-rewards into a training loss in the main experiments.","marker":"[32]"},{"why":"GRPO is the reinforcement-learning algorithm used as the alternative optimizer, showing the reward mechanism works across optimization families.","marker":"[41]"},{"why":"CLIP serves as the external reward-model baseline in the comparison showing DSR outperforms CLIP and avoids understanding degradation.","marker":"[37]"},{"why":"The LLaVA-Bench benchmark on which the tuned model gains 7.5% is introduced with this reference, which also supplies the LLaVA baseline in the understanding table.","marker":"[26]"},{"why":"Show-o is the second backbone demonstrating that SUDER transfers to a different unified architecture, with a more than 5% average gain on T2I-CompBench.","marker":"[55]"}],"fun_headline_variants":["Self-reward duality improves multimodal AI without human labels","AI teaches itself via inverse tasks to boost image-text alignment","SUDER's dual self-reward sharpens multimodal understanding and generation","No labels needed: SUDER's dual reward lifts multimodal performance","Dual self-reward turns understanding and generation into mutual teachers"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The model's own probability of regenerating the original input is assumed to reliably measure how well the caption and image actually match, and this link must stay strong through training instead of being exploitable to inflate the score.","fun_headline_variants_meta":{"raw":{"variants":["Self-reward duality improves multimodal AI without human labels","AI teaches itself via inverse tasks to boost image-text alignment","SUDER's dual self-reward sharpens multimodal understanding and generation","No labels needed: SUDER's dual reward lifts multimodal performance","Dual self-reward turns understanding and generation into mutual teachers"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000655,"raw_usage":{"total_tokens":3019,"prompt_tokens":982,"completion_tokens":2037,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":598,"completion_tokens_details":{"reasoning_tokens":1962}},"tokens_in":598,"tokens_out":2037,"duration_ms":16585,"temperature":1.0,"reasoning_tokens":1962,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T05:21:29.120271+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a held-out set of prompts and images, rank candidate outputs by the dual self-reward, and compare those rankings against independent alignment judgments (human ratings or a strong external VQA/hallucination detector). If the dual self-reward's ranking does no better than chance at picking the output humans judge as more aligned, the training signal is not tracking alignment and the reported gains would not transfer to new data.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"T2I-CompBench supplies the 5,600 training prompts and is the compositional text-to-image benchmark on which the +11.68% average gain is measured."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"GenEval is the object-focused text-to-image benchmark on which the tuned model gains 5% overall."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CLIP serves as the external reward-model baseline in the comparison showing DSR outperforms CLIP and avoids understanding degradation."}],"review_version":1}