{"id":"4b421850-b15a-48a1-9100-e7a4a380c968","arxiv_id":"2507.01335","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Training 2B/7B language models on right-to-left text factorization yields a reverse posterior scorer that improves math reasoning when combined with forward likelihood.","lead":"This paper trains language models to predict text from right to left, instead of the usual left-to-right direction, at 2B and 7B scale. It shows the reverse model can score how well a candidate answer reconstructs its question, and using this score to rerank math answers improves accuracy on AIME 2024 and AMC 2023.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central mechanism rests on an unmeasured posterior-degradation premise; Table 4 compares only against random selection, so the gains are not yet isolated from forward-likelihood or SFT-overlap confounds.","rationale":"The paper's formal derivation is internally consistent: Proposition 1 follows from its premise, and the noisy-channel rewriting in Eq. 8 is algebraically correct under the stated assumptions. The weakness is empirical, not logical. The reader's CONDITIONAL verdict is therefore appropriate: the direction is plausible and the release is valuable, but the central mechanism is supported only by a single qualitative case, while the reported gains have not been separated from forward-likelihood selection, length or style correlations, or possible SFT overlap with evaluation distributions. The proposed controlled reranking experiment would settle whether posterior degradation actually holds in the evaluated regimes and whether it drives the Table 4 improvements. No change to the reader's verdict is needed.","tokens_in":20116,"tokens_out":8198,"duration_ms":107050,"concrete_test":"On the exact candidate sets generated for Table 4 (N=64 per problem), label each candidate correct or incorrect using ground truth, then bin candidates by forward likelihood P_FLM(y|x) and separately by length or step count. Within each bin, test whether correct candidates have systematically higher P_RLM(x|y), using a matched-pair Wilcoxon test or rank correlation. Then add two reranking baselines on the same candidate sets: select by maximum P_FLM(y|x) alone and select by maximum P_RLM(x|y) alone. If the within-bin posterior gap is absent, or if the forward-likelihood baseline matches Reverse Reward's accuracy, the reported gains are not evidence of posterior degradation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Proposition 1 (Section 5.1) is a conditional statement: it shows that if hallucinated responses have lower reverse posterior than correct responses at comparable forward likelihood, then Reverse Reward ranks correctly. The entire empirical case depends on this 'posterior degradation' premise, but the only direct evidence offered is the single qualitative example in Table 7 (Remark 1). Table 4 compares Reverse Reward against greedy decoding and Best-of-N random; neither control isolates the reverse posterior. Because R(x,y) in Eq. 7 contains P_FLM(y|x)^(1-lambda), the reported gains could be reproduced by selecting high forward-likelihood candidates, or by correlated length or style preferences, without any posterior-degradation effect. A second confound is that the RLM was SFT on 100,000 OpenMathInstruct-2 examples; if those examples overlap with GSM8K, MATH-500, AIME, or AMC instances or solution styles, the score may reflect memorization rather than a general reconstruction penalty. The paper's Limitations explicitly state that posterior approximation under distribution shift is not established. Thus the load-bearing premise is unverified at exactly the point where the proof needs it.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper trains LEDOM, a purely reverse autoregressive language model at 2B/7B scale on 435B tokens, with matched forward models on identical architecture and data. It reports behavioral differences (abductive reasoning, question synthesis, reversal-curse resolution) and then proposes Reverse Reward, a bidirectional scoring method that combines forward likelihood P(y|x) with reverse posterior P(x|y) to rerank or beam-search forward generations on math benchmarks. The empirical section reports consistent gains over greedy decoding and Best-of-N random selection on GSM8K, MATH-500, AIME 2024, and AMC 2023, with a formal Proposition 1 claiming that posterior degradation is penalized.","tokens_in":20339,"tokens_out":5321,"duration_ms":64780,"significance":"If the empirical claims survive scrutiny, the contribution is substantial: LEDOM is, to my knowledge, the first open large-scale purely reverse autoregressive model, and the bidirectional scoring idea is a parameter-light verification signal that transfers across forward models of varying quality. The release of models, code, and data is a genuine strength, and the matched FLM/RLM training setup is a clean experimental design for isolating the effect of factorization direction. The consistency of the Table 4 gains across three base models and four benchmarks, including large gains on AIME and AMC, is a meaningful empirical signal. However, the central proof is conditional on an unverified posterior-degradation premise, the SFT data used to train the reverse reward model overlaps with two of the evaluation benchmarks, and key hyperparameters are unreported. These issues currently prevent the paper from establishing the mechanism it claims.","major_comments":[{"comment":"Proposition 1 is a direct restatement of the posterior-degradation assumption: it proves that if PRLM(x|y') < PRLM(x|y*) at comparable forward likelihood, then R ranks y* above y'. This is formally valid but does not establish that bidirectional scoring penalizes hallucination independently of the assumed premise. Moreover, Eq. (8) shows that at fixed forward likelihood the combined score is log P(y|x) - lambda log P(y) + c, so the ordering is exactly the ordering of -log P(y), i.e., of the reverse posterior through Bayes' rule. The paper should be explicit that the proposition is a conditional guarantee and that the substantive claim is the empirical posterior-degradation hypothesis, not a derived theorem.","section":"Section 5.1, Proposition 1 and Eq. (8)"},{"comment":"The only direct evidence for posterior degradation is the single qualitative Carla-download example in Table 7. This is insufficient to support the load-bearing premise of Proposition 1 and the interpretation of Table 4. The authors should report a quantitative evaluation on a sample of Best-of-N candidates: e.g., the distribution of PRLM(x|y) for correct versus incorrect responses matched on forward likelihood, with separation statistics and error bars. The Limitations section itself concedes that the posterior approximation under distribution shift is not established, so without such measurements the reported gains cannot be attributed to the reverse-posterior mechanism.","section":"Section 6.4, Remark 1, Table 7"},{"comment":"The reverse reward model is SFT on 100,000 examples from OpenMathInstruct-2, which is constructed from GSM8K and MATH (the source of MATH-500). This creates a direct overlap between the reward model's fine-tuning distribution and two of the four evaluation benchmarks in Table 4, so the GSM8K and MATH-500 gains may reflect memorization of solution styles or instances rather than a general reconstruction penalty. The authors should quantify the overlap, filter any overlapping instances, or report results on held-out benchmarks (e.g., AIME 2024 and AMC 2023 are less subject to this concern) with the SFT-leakage controlled.","section":"Section 6.1, Appendix D.2.1, Table 4"},{"comment":"Two central reproducibility details are missing or inconsistent. Table 4's caption and Section 6.5 state that Best-of-N samples 64 candidates, while Appendix D.2.3 says N=4 for response-level reranking; these must be reconciled. In addition, the combination weight lambda in Eq. (7) is never reported anywhere in the main text or appendices, even though the method's behavior depends entirely on lambda (lambda=0 reduces R to forward likelihood). The authors should report the exact lambda value(s) used and ideally include a sensitivity analysis over lambda.","section":"Section 6.2, Table 4 caption, Appendix D.2.3"},{"comment":"Step-wise beam search applies R(x,s<=t) to partial sequences, but Eq. (6) defines RRLM for a complete response y. Scoring a partial reasoning chain with a model trained to reconstruct the prompt from a full response is an approximation that is not discussed or validated. Since beam search is presented as a distinct contribution and one table row (OpenMath2 AMC 2023) shows its benefit, the authors should either justify this approximation or provide an ablation comparing step-level and response-level scoring under matched compute.","section":"Section 5.2 and Algorithm 1"}],"minor_comments":[{"comment":"Table 8 is a beam-search case on MATH-500, but its caption says 'A specific case of Best-of-N by Qwen-Math on GSM8K'; the caption should be corrected to match the content.","section":"Table 8 caption"},{"comment":"The claim that longer or more variable responses imply H(Y)>H(X) is heuristic; entropy is a property of the distribution, not of sequence length alone. The statement should be qualified or supported by an argument about the actual distributions.","section":"Section 2.2, Eq. (4)"},{"comment":"The notation in Eq. (6) uses T for the prompt length while conditioning on response tokens y; the index ranges should be made explicit to avoid confusion about what is being summed and conditioned on.","section":"Eq. (6)"},{"comment":"The phrase 'precious token prediction' appears to be a typo; the intended term is likely 'prefix token prediction' or 'prior token prediction'.","section":"Appendix D.2.1"},{"comment":"Figure 3 reports accuracy of 'FLM' with Reverse Reward beam search, but the base model is not named in the caption or text; it should identify which forward model was used and clarify whether the curve is for MATH-500 or GSM8K, since both appear on different y-axes.","section":"Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The most serious issue is the SFT overlap with GSM8K and MATH via OpenMathInstruct-2; if the authors cannot demonstrate that the gains persist after removing overlapping data, the empirical section's main claim is compromised. The incomplete reporting of lambda and the N=64 vs N=4 discrepancy are also more than cosmetic, since they affect reproducibility. If these points are fixed with additional experiments, the paper could be suitable for publication; as it stands, the central mechanism is unverified."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe thing to know about this paper is that its real contribution is the reverse model, not the verification proof. Training a purely reverse autoregressive LM at 2B/7B on 435B tokens and releasing it is genuinely new, and the behavioral observations—abductive reasoning, question synthesis, reversal-curse resolution—are worth taking seriously. The Reverse Reward reranking also produces consistent improvements on GSM8K/MATH/AIME/AMC across three different base models, which is a useful empirical signal. The related work is handled honestly.\n\nWhere the paper is soft is exactly where it claims to be rigorous. Proposition 1 says: if hallucinated outputs have lower reverse posterior than correct outputs at comparable forward likelihood, then the bidirectional score ranks them correctly. That is true, but it restates the premise; it does not prove that reverse posteriors behave this way. The only direct evidence is the single case in Table 7, and the limitations section concedes that posterior approximation under distribution shift is not established. So the load-bearing premise is unmeasured.\n\nThe experimental design has two confounds. First, the baselines are greedy decoding and random Best-of-N. Random selection does not isolate the reverse posterior; a forward-likelihood-only selection baseline would be needed to show the gains come from the reverse term rather than from the P_FLM component in Eq. 7. Second, the reverse model was SFT on 100k OpenMathInstruct-2 examples, a dataset built from GSM8K and MATH. Overlap with the evaluation sets is plausible and could explain part of the effect; the paper does not address this. There are also reporting gaps: the combination weight lambda is never given, and the main text says Best-of-N samples 64 candidates while the appendix says N=4.\n\nNone of this makes the paper a write-off. The resource is real, the behavior analysis is interesting, and the gains are consistent enough to merit a proper look. But the abstract oversells the proof and the method is under-specified. A serious referee should ask for proper controls, lambda and N reporting, and contamination analysis, and should temper the language about what is proven.\n\nThe target audience is researchers working on LM verification, math reasoning, and bidirectional factorization. The model and reranking idea will be useful to them. This deserves peer review—not a desk reject, but a major-revision path.","headline":"Open-source reverse LM is the real news; the verification proof assumes its own central premise, but the paper deserves a serious referee.","tokens_in":20920,"tokens_out":3668,"would_cite":false,"duration_ms":44583,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A purely right-to-left language model can verify forward answers: reranking by reverse posterior — how well an answer reconstructs its question — lifts AIME 2024 accuracy by 6.6 points and AMC 2023 by 15 points.","keywords":["reverse language model","right-to-left factorization","noisy channel decoding","posterior verification","hallucination reranking","reversal curse","abductive inference","mathematical reasoning"],"falsifier":"Measure posterior degradation directly at scale: on a math benchmark, sample correct and incorrect candidate solutions from a forward model with matched forward likelihood and record how often the reverse posterior ranks the correct solution higher; then repeat with the reverse reward model applied out-of-domain, for example without the OpenMathInstruct fine-tune or on response distributions unlike its training data. If the ranking rate is near chance, or if the Table 4 gains on AIME 2024 and AMC 2023 disappear under those conditions, posterior verification is not what produced the reported improvement.","tokens_in":19916,"feed_emoji":"🔄","tokens_out":15548,"duration_ms":112952,"temperature":0.7,"pith_summary":"The paper's claim is that the left-to-right factorization of language is only half the story. Training a model on the same data, architecture, and tokenizer but predicting tokens right-to-left produces complementary abilities — abductive inference, question synthesis, and a natural grasp of inverse relations — rather than merely a worse forward model. That reverse model then serves a practical purpose: by Bayes' theorem, its posterior $P(\\text{question} \\mid \\text{answer})$ measures whether a candidate answer genuinely reconstructs the original prompt, a verification signal forward likelihood alone cannot supply. Combined with forward likelihood in a noisy-channel score, this 'Reverse Reward' reranks candidate math solutions, and the paper reports that it beats both greedy decoding and random Best-of-N selection across three math-specialized forward models, with gains up to 6.6 points on AIME 2024 and 15 points on AMC 2023. If correct, the contribution is a label-free, parameter-light verifier that transfers across forward models of very different quality.","feed_headline":"Reverse-trained LM lifts math accuracy by up to 15 points","feed_subtitle":"A back-to-front model scores how well an answer reconstructs its question, catching hallucinated reasoning chains.","key_machinery":"The load-bearing object is the reverse autoregressive factorization $P(x) = \\prod_{t=1}^{T} P(x_t \\mid x_{t+1:T})$, trained by reversing the token sequence and applying a standard causal Transformer; at every position the hidden state encodes what comes after, not what came before. Its downstream instrument is the Reverse Reward score $R(x,y) = P_{\\mathrm{FLM}}(y \\mid x)^{1-\\lambda} \\cdot P_{\\mathrm{RLM}}(x \\mid y)^{\\lambda}$, the noisy-channel combination of forward likelihood with the reverse posterior $P_{\\mathrm{RLM}}(x \\mid y)$, the probability that a candidate response reconstructs the original prompt when read back-to-front. The argument rests on two identities: Bayes' theorem, which reduces the score to $\\log P(y \\mid x) - \\lambda \\log P(y) + c$ — forward likelihood regularized by a marginal response-complexity penalty — and the conditional-entropy asymmetry $H(Y \\mid X) - H(X \\mid Y) = H(Y) - H(X)$, which shows reverse scoring is the tighter, more concentrated evaluation signal whenever responses are longer or more variable than prompts, as in reasoning tasks.","core_discovery":"The paper's central claim is that reverse posterior scoring penalizes hallucination. Proposition 1 states: for a prompt $x$, if a correct response $y^*$ and a hallucinated response $y'$ have comparable forward likelihood $P_{\\mathrm{FLM}}(y \\mid x)$, and the hallucinated response exhibits 'posterior degradation' — $P_{\\mathrm{RLM}}(x \\mid y') < P_{\\mathrm{RLM}}(x \\mid y^*)$ — then the combined bidirectional score $R(x,y) = P_{\\mathrm{FLM}}(y \\mid x)^{1-\\lambda} \\cdot P_{\\mathrm{RLM}}(x \\mid y)^{\\lambda}$ ranks the correct response higher for any $\\lambda > 0$. Because $\\log R(x,y) = \\log P(y \\mid x) - \\lambda \\log P(y) + c$, bidirectional scoring is exactly noisy-channel decoding, the information-theoretic scheme that treats a received output as a corrupted rendering of a hidden message and decodes by combining likelihood with a prior; here it is forward likelihood regularized by a marginal-complexity penalty that suppresses generic, prompt-independent responses. The empirical version of the claim is Table 4: Reverse Reward improves accuracy on GSM8K, MATH-500, AIME 2024, and AMC 2023 over greedy decoding and uniform random Best-of-N across DeepSeekMath, QwenMath, and OpenMath2, with the largest gains, up to +6.6 points on AIME 2024 and +15 points on AMC 2023, on competition-level problems. The paper also claims the reverse model itself develops distinct capabilities — abductive inference, question synthesis, backward-from-goal derivation — and resolves the reversal curse naturally, while matching forward models on semantic-understanding tasks at 2B scale.","pith_inferences":["The noisy-channel argument is not math-specific. A direct extension would test Reverse Reward on other hallucination-prone settings — summarization faithfulness, open-domain factuality, instruction following — where a faithful output should reconstruct its prompt better than a fabricated one; the paper does not run these tests.","Posterior degradation is itself a measurable diagnostic: ranking candidate answers by reverse posterior alone, against gold or human labels on a broad suite, would isolate how much of the reranking signal is fidelity to the prompt rather than generic fluency preferences.","Because the reverse model is architecture-identical to the forward model, the same GPU pool hosts both directions at roughly equal per-pass cost; whether the verification signal strengthens or saturates as both models scale is an open question the paper explicitly leaves.","The safety asymmetry the paper observes — reverse generation completing unsafe prompts that forward safeguards block — implies alignment must be direction-aware; a concrete probe would test whether safety fine-tuning on forward data transfers to a reversed factorization at all."],"forward_implications":["Reverse posterior scoring is a label-free verifier: it adds one reverse-model forward pass per candidate and improves accuracy across forward models whose greedy GSM8K accuracy spans 42.0% to 95.6%, so it complements rather than duplicates forward model quality.","Bidirectional scoring is equivalent to forward likelihood with a marginal-complexity penalty $-\\lambda \\log P(y)$, so the reranker suppresses generic, prompt-independent responses instead of merely favoring fluent or long ones.","Finer verification granularity helps on some tasks: step-level beam search with the same score prunes hallucinated derivation paths early and further improves AMC 2023 and GSM8K, though it underperforms on AIME 2024, where pruning long chains can discard partially correct beams.","The reverse factorization itself resolves the reversal curse and generates premises from conclusions, making abductive inference, question synthesis, and backward-from-goal derivation available as engineering resources.","The two directions fail on different tasks and make different errors on the same tasks; that complementarity, the paper argues, is what makes bidirectional combination fruitful beyond the specific math-verification application demonstrated here."],"supporting_citations":[{"why":"Supplies the direct precedent Reverse Reward extends: a small reverse model providing unsupervised feedback and best-of-N reranking for forward generations.","marker":"Varun et al. (2025)"},{"why":"Provides the noisy-channel formulation that justifies combining forward likelihood with reverse posterior as channel decoding.","marker":"Shannon (1948)"},{"why":"Establishes the neural noisy channel, the prior application of Bayes-combined scoring that this paper generalizes to language-model verification.","marker":"Yu et al. (2017)"},{"why":"Demonstrates simple and effective noisy-channel modeling for machine translation, the empirical precedent for the bidirectional reranking.","marker":"Yee et al. (2019)"},{"why":"Defines the reversal curse that forward models suffer and that the reverse factorization resolves, motivating the complementary-direction claim.","marker":"Berglund et al. (2023)"},{"why":"Supplies GSM8K, the grade-school math benchmark on which Reverse Reward reranking is evaluated.","marker":"Cobbe et al. (2021)"},{"why":"Supplies MATH-500, the competition-level benchmark used to measure the reranking gains.","marker":"Lightman et al. (2023)"},{"why":"DeepSeekMath-7B, one of the three forward baselines whose outputs Reverse Reward reranks.","marker":"Shao et al. (2024)"},{"why":"OpenMath2 baseline; its OpenMathInstruct-2 dataset is also the source of the 100,000 fine-tuning examples for the reverse reward model.","marker":"Toshniwal et al. (2025)"},{"why":"QwenMath-7B, the strongest forward baseline, which reaches 96.1% GSM8K with Reverse Reward.","marker":"Yang et al. (2024)"}],"fun_headline_variants":["Reverse posterior scoring punishes hallucinated reasoning","Backward model's posterior scoring boosts math accuracy by 15%","Noisy-channel decoding with reverse likelihood lifts math scores","Reverse LM catches hallucinations by checking backward plausibility","Bidirectional scoring improves math reasoning via reverse posterior"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything rests on one empirical premise: that a hallucinated response — one whose reasoning inserts steps absent from the original premises — actually scores lower under the reverse posterior than a correct response with comparable forward likelihood. The paper states this premise behind Proposition 1, points to a single qualitative case (Table 7) as support, and its own Limitations section concedes that the quality of the reverse posterior under distribution shift is not established.","fun_headline_variants_meta":{"raw":{"variants":["Reverse posterior scoring punishes hallucinated reasoning","Backward model's posterior scoring boosts math accuracy by 15%","Noisy-channel decoding with reverse likelihood lifts math scores","Reverse LM catches hallucinations by checking backward plausibility","Bidirectional scoring improves math reasoning via reverse posterior"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001022,"raw_usage":{"total_tokens":4391,"prompt_tokens":1106,"completion_tokens":3285,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":722,"completion_tokens_details":{"reasoning_tokens":3210}},"tokens_in":722,"tokens_out":3285,"duration_ms":73671,"temperature":1.0,"reasoning_tokens":3210,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T20:54:40.941134+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure posterior degradation directly at scale: on a math benchmark, sample correct and incorrect candidate solutions from a forward model with matched forward likelihood and record how often the reverse posterior ranks the correct solution higher; then repeat with the reverse reward model applied out-of-domain, for example without the OpenMathInstruct fine-tune or on response distributions unlike its training data. If the ranking rate is near chance, or if the Table 4 gains on AIME 2024 and AMC 2023 disappear under those conditions, posterior verification is not what produced the reported improvement.","supporting_citations":[],"review_version":1}