{"id":"71c2b26a-5c2f-4db0-9db6-6703d4f73073","arxiv_id":"2508.14460","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"DuPO trains LLMs by rewarding them when they can reconstruct hidden parts of their input from their own output, using this self-generated signal to improve translation, math, and reranking.","lead":"A new training method called DuPO uses a task's own reversed version to generate feedback for AI models without needing human labels. It improved translation, math reasoning, and answer reranking in tests, and could make AI self-improvement cheaper and more general.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reconstruction-quality reward may reward fluency, not correctness; no evidence yet that it correlates with task correctness.","rationale":"The reader's weakest assumption identifies the same load-bearing premise: that reconstruction quality serves as a faithful self-supervised reward. My analysis agrees with this concern and makes it more concrete. The reader's verdict is UNVERDICTED, which is appropriate given the abstract-only evidence and the demonstrated absence of support for the core mechanism. My recommended verdict is therefore UNCHANGED: the concern does not shift the verdict, but it reinforces the need for empirical validation of reward–correctness correlation before the central claim can be credited.","tokens_in":723,"tokens_out":1812,"duration_ms":25367,"concrete_test":"On a labeled dataset with known correct/incorrect outputs (e.g., GSM8K test set, or WMT translations with human quality scores), compute the DuPO self-supervised reconstruction-quality reward for each generated output. Measure the Spearman rank correlation between the reward and ground-truth correctness. If the correlation is weak (e.g., <0.2) or if the reward does not significantly separate correct from incorrect outputs that are matched for fluency, the central premise fails; if the correlation is strong, the premise is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends on the premise that the quality of reconstructing the unknown input component from the primal output is a faithful reward signal for task correctness. This premise is plausible but not established. The abstract gives no definition of how reconstruction quality is measured, nor any evidence that it tracks correctness rather than surface-level properties such as fluency, coherence, or self-similarity. For mathematical reasoning, a fluent but incorrect solution may still permit a plausible reconstruction of the problem statement, making the reward insensitive to the correctness of the solution. For translation, a back-translation may be fluent even when the forward translation is wrong. If the reward rewards stylistic consistency or reconstruction ease rather than semantic correctness, the reported gains (2.13 COMET, +6.4 points, +9.3 reranker) could arise from optimizing a proxy that does not generalize as a true measure of task quality. No full text is available to check for control experiments, correlation analyses, or reward hacking mitigation, so this load-bearing premise is currently unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DuPO, a dual learning-based preference optimization framework that generates annotation-free feedback via a generalized duality. The primal task's input is decomposed into known and unknown components, and a dual task is constructed to reconstruct the unknown component from the primal output and known information. The reconstruction quality is used as a self-supervised reward to optimize the primal task. The abstract reports substantial empirical gains in translation (2.13 COMET over 756 directions), mathematical reasoning (+6.4 points on three challenge benchmarks), and inference-time reranking (+9.3 points). The framework is claimed to be scalable, general, and annotation-free.","tokens_in":965,"tokens_out":1989,"duration_ms":21579,"significance":"If the reported gains hold, DuPO would be a significant contribution to LLM self-verification and preference optimization. The approach is attractive because it removes the need for costly labels and extends beyond strictly dual task pairs, potentially covering non-invertible tasks. The reported gains are large across two diverse task families (translation and math), and the reranking result suggests broader utility. However, significance is conditional on the empirical claims being reproducible and on the reconstruction-quality reward genuinely tracking task correctness rather than surface-level fluency. The manuscript as provided (abstract only) does not yet establish these conditions.","major_comments":[{"comment":"The reported gains (2.13 COMET over 756 directions, +6.4 points on three math benchmarks, +9.3 points as reranker) are presented without any experimental details: no baselines, no error bars or standard deviations, no number of runs, no dataset versions, and no ablations. Since the provided manuscript is only the abstract, these claims are currently unverifiable. This is load-bearing because the paper's contribution is primarily empirical.","section":"Abstract (empirical claims)"},{"comment":"The core self-supervised reward is defined as 'the quality of this reconstruction,' but the abstract does not specify how reconstruction quality is measured (e.g., likelihood, token-level accuracy, external metric) nor provide evidence that it correlates with actual task correctness. For mathematical reasoning, a fluent but incorrect solution may still permit a plausible reconstruction of the hidden variable; for translation, a fluent back-translation can arise from an incorrect forward translation. Without a correlation analysis or control experiments, the claimed gains could reflect optimization of a proxy (fluency/self-consistency) rather than true correctness.","section":"Abstract (reward definition)"},{"comment":"The abstract asserts that DuPO is a 'scalable, general, and annotation-free paradigm for LLM optimization' based on results in translation and mathematical reasoning. This is a strong claim. No evidence is provided for generality across other task types, and no theoretical analysis or negative results are mentioned. The lack of any supporting detail in the available text makes the generality claim speculative at this stage.","section":"Abstract (generality claim)"}],"minor_comments":[{"comment":"The terms 'generalized duality' and 'dual task' are used informally. A formal definition or reference would clarify the contribution, especially since the abstract distinguishes DuPO from 'traditional dual learning.'","section":"Abstract (terminology)"}],"recommendation":"uncertain","confidential_remarks":"This review is based solely on the abstract because the full text was not provided. The central claims are plausible but empirically unsubstantiated in the available material. I cannot recommend acceptance or rejection without the full paper. The editor should obtain the full manuscript before making a decision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing to know: this is a real conceptual idea wrapped in an abstract that makes large empirical promises. The known/unknown decomposition of the input, with the dual task reconstructing the unknown part, is a sensible generalization of dual learning beyond strict invertibility. That is genuinely new relative to RLVR and standard back-translation setups, and the reported numbers (2.13 COMET over 756 directions, +6.4 math, +9.3 reranker) would be impressive if they hold. The fact that the reward is annotation-free and uses the model's own reconstruction quality is also a neat trick that could matter in practice.\n\nWhat the paper does well, on the evidence available, is frame a coherent optimization loop: pick the known/unknown split, define the dual reconstruction, score it, and use that as preference feedback. That is a clean recipe. If the full text includes correlation analyses between reconstruction scores and actual task correctness, plus proper baselines (RLVR, self-consistency, other self-reward methods), this could be a solid contribution.\n\nNow the soft spots. The load-bearing premise is that reconstruction quality is a faithful reward for correctness. The abstract gives no evidence for that. A fluent back-translation does not imply the forward translation is semantically correct; a math model can reconstruct a plausible problem statement from a wrong solution. The risk of rewarding fluency, self-similarity, or reconstruction ease is real. The stress-test note is on point. Also, full experimental details are missing: no baselines, no ablations, no error bars, no discussion of reward hacking. Those are gaps in the abstract, not necessarily in the paper, but they leave the central claim unverified.\n\nThe reported gains might be partly explained by the reward correlating with surface quality rather than task correctness. Without a correlation analysis or at least a sanity check showing that higher reconstruction reward tracks better COMET or math accuracy on held-out data, I would not take the numbers at face value.\n\nWho is this for? People working on LLM post-training, especially those trying to escape the label bottleneck of RLVR. A reader who wants a new optimization paradigm will find the idea worth engaging with. But the empirical case is not yet made from the abstract alone.\n\nRecommendation: if the full paper contains the missing evidence, send it to peer review. The idea is important enough to merit referee time. If the full paper is as thin as the abstract, desk reject. Based on the abstract, I would lean toward 'send to peer review' because the framework is promising and deserves a careful look.\n\nWould I cite it? Not yet, not without seeing the full results. But I would bring it to a reading group to discuss whether reconstruction rewards can be trusted.","headline":"DuPO's generalized-duality idea is genuinely interesting, but the abstract alone cannot support the claim that reconstruction quality is a faithful reward; the full paper needs scrutiny.","tokens_in":1358,"tokens_out":983,"would_cite":false,"duration_ms":14092,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DuPO claims an LLM can generate its own training rewards by reversing the task it is trying to solve, and that these self-made rewards improve translation, math reasoning, and reranking without human labels.","keywords":["dual learning","self-supervised reward","preference optimization","LLM alignment","translation quality","mathematical reasoning","inference-time reranking"],"falsifier":"Construct a task where a wrong but stylistically consistent output makes the hidden input trivial to reconstruct, while a correct output is harder to reverse, then check whether DuPO's self-supervised reward prefers the wrong output. If it does, the reward signal is not a faithful proxy for correctness.","tokens_in":716,"feed_emoji":"🔄","tokens_out":1907,"duration_ms":20998,"temperature":0.7,"pith_summary":"DuPO claims that an LLM can generate its own training feedback for almost any task by reversing the task: split the input into a known part and an unknown part, run the model on the task, then ask the model to recover the unknown part from the output plus the known part. The quality of that reconstruction serves as a self-supervised reward to optimize the primal task. The paper reports that this annotation-free reward improves average translation quality by 2.13 COMET across 756 directions, lifts mathematical reasoning accuracy by an average of 6.4 points on three challenge benchmarks, and adds 9.3 points when used as an inference-time reranker. If true, this matters because standard verifiable-reward RL requires costly labels and only works for tasks with checkable answers, while traditional dual learning is restricted to invertible task pairs like translation and back-translation. DuPO positions itself as a scalable, general, and annotation-free paradigm for LLM optimization.","feed_headline":"LLM learns from rewards made by reversing its own task","feed_subtitle":"No labels needed: recovering hidden input from output steers translation, math, and reranking.","key_machinery":"The key mechanism is a generalized duality: decomposing a task's input into known and unknown components, defining the dual task as reconstructing the unknown component from the output and known information, and using reconstruction quality as a self-supervised reward. This carries the argument because it converts almost any task into a form of verifiable self-feedback without external labels.","core_discovery":"The central discovery is a generalized dual-task construction. For a primal task whose input can be decomposed into known information and unknown information, DuPO constructs a dual task: reconstruct the unknown input component from the primal task's output plus the known information. The quality of this reconstruction is used as a self-supervised reward to optimize the primal task, and because the same LLM can instantiate both tasks, the reward is generated without human annotations. The paper claims this extends dual learning beyond strictly dual task pairs, makes non-invertible tasks optimizable, and produces substantial gains in translation, mathematical reasoning, and inference-time rer","pith_inferences":["A risk the paper leaves implicit is that reconstruction quality might reward stylistic consistency or self-similarity rather than genuine correctness; an output that is confidently wrong but easy to reconstruct could be preferred over a correct output that is harder to reverse.","The decomposition into known and unknown input components is a design choice that may not be unique; different decompositions could produce different rewards, and the paper does not discuss how to choose the decomposition optimally.","The method could potentially be combined with other self-supervised signals, such as consistency across paraphrases or output fluency, to create a richer reward than reconstruction quality alone.","A direct testable extension would be applying DuPO to a task where the hidden input component is explicitly designed to be misleading, to see whether the reward signal remains aligned with true correctness."],"forward_implications":["If reconstruction quality tracks task correctness, LLMs could be optimized on open-ended tasks that currently lack automated rewards, without human annotation.","The same dual-reward mechanism could be applied to other generation tasks, such as code generation, summarization, or dialogue, whenever a hidden input component can be defined.","Because the reward is self-generated, DuPO could scale to many languages, domains, and tasks where collecting preference pairs or verifiable labels is impractical.","Using the self-supervised reward as an inference-time reranker offers a compute-for-accuracy tradeoff that could be combined with other decoding strategies.","The framework suggests a path toward continual self-improvement, where a model generates its own training signal from task structure rather than external feedback."],"supporting_citations":[],"fun_headline_variants":["Self-verifying LLMs by reversing their own tasks","Dual-task reconstruction yields label-free rewards","LLMs learn from reconstructing hidden input parts","DuPO: generalized duality for annotation-free LLM feedback","Reversing tasks trains LLMs without labels"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The whole scheme stands or falls on whether the quality of reconstructing the hidden input component from the output actually tracks true task correctness, rather than rewarding outputs that merely make reconstruction easy.","fun_headline_variants_meta":{"raw":{"variants":["Self-verifying LLMs by reversing their own tasks","Dual-task reconstruction yields label-free rewards","LLMs learn from reconstructing hidden input parts","DuPO: generalized duality for annotation-free LLM feedback","Reversing tasks trains LLMs without labels"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000263,"raw_usage":{"total_tokens":1432,"prompt_tokens":737,"completion_tokens":695,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":481,"completion_tokens_details":{"reasoning_tokens":621}},"tokens_in":481,"tokens_out":695,"duration_ms":7481,"temperature":1.0,"reasoning_tokens":621,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T18:29:31.622691+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a task where a wrong but stylistically consistent output makes the hidden input trivial to reconstruct, while a correct output is harder to reverse, then check whether DuPO's self-supervised reward prefers the wrong output. If it does, the reward signal is not a faithful proxy for correctness.","supporting_citations":[],"review_version":1}