Reverse Thinking Makes LLMs Stronger Reasoners

Chen-Yu Lee; Hamid Palangi; Justin Chih-Yao Chen; Long Le; Mohit Bansal; Rujun Han; Sayna Ebrahimi; Swaroop Mishra; Tomas Pfister; Vincent Perot

REVIEW 3 major objections 5 minor 3 cited by

Reverse Thinking Makes LLMs Stronger Reasoners

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Training LLMs to reason backward lifts accuracy by 13.5% across 12 reasoning benchmarks.

desk verdict A solid distillation recipe that likely helps, but the paper overstates the 'reverse thinking' mechanism it claims to establish. read the letter →

arxiv 2411.19865 v2 pith:BDNQKISA submitted 2024-11-29 cs.CL cs.AIcs.LG

Justin Chih-Yao Chen , Zifeng Wang , Hamid Palangi , Rujun Han , Sayna Ebrahimi , Long Le , Vincent Perot , Swaroop Mishra

show 3 more authors

Mohit Bansal Chen-Yu Lee Tomas Pfister

This is my paper · ORCID

classification cs.CLcs.AIcs.LG

keywords reversethinkingbackwardreasoningknowledgedistillationchain-of-thoughtmulti-tasklearningLLMdataaugmentation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a language model becomes a stronger reasoner when it is also trained to think in reverse. RevThink augments each training question with teacher-generated backward reasoning—a reversed question and its solution—and trains a smaller student model on three joint objectives: solve the original question, write its backward question, and solve that backward question. Across 12 reasoning datasets spanning commonsense, math, tabular, logical, and natural-language-inference tasks, the authors report an average gain of 13.53% over the student's zero-shot performance and 6.84% over the strongest knowledge-distillation baseline, with the same test-time cost as zero-shot prompting. The result matters because it suggests reverse thinking can be instilled during training rather than used only as an expensive verification step at inference.

What carries the argument

The load-bearing object is the augmented training tuple $(Q, R_f, Q_b, R_b)$: the original question, teacher-generated forward reasoning, the backward question, and the backward reasoning. The training loss is a joint multi-task objective, $L = \frac{1}{3n}\sum_{i}[\ell(S(Q^{(i)}), R_f^{(i)}) + \ell(S(Q^{(i)}), Q_b^{(i)}) + \ell(S(Q_b^{(i)}), R_b^{(i)})]$, where $\ell$ is the cross-entropy between predicted and target tokens. Data quality is enforced by keeping only samples whose forward reasoning leads to the correct answer and whose backward reasoning is verified by the teacher to be consistent with the original question. This machinery makes reverse thinking a training-time regularizer rather than a test-time verification loop.

What would settle it

Train a student with the same RevThink pipeline but replace the backward question and reasoning with template-generated inverses that pass a deliberately weak consistency filter, or with random backward content; if the accuracy gains persist, the effect is not due to genuine forward-backward consistency. Alternatively, run the pipeline on a set of problems with no invertible structure, such as pure number theory, and check whether the improvement over standard distillation disappears.

Watch

Extended reading notes

Core claim

The central discovery is that bidirectional thinking can be transferred from a larger teacher model to a smaller student through data augmentation and multi-task learning. For each question–answer pair, the teacher generates a forward chain-of-thought, a backward question that inverts the original, and backward reasoning that solves the inverted question; only triples whose forward reasoning matches the ground-truth answer and whose backward reasoning passes a teacher consistency check are kept. The student is trained to produce forward reasoning from the question, to produce the backward question from the question, and to produce backward reasoning from the backward question. At test time the student generates only forward reasoning, so backward thinking is internalized during training. The paper reports consistent accuracy gains over unidirectional knowledge distillation on eight held-in benchmarks, plus gains on four held-out datasets, along with sample efficiency and positive scaling with model size.

Load-bearing premise

The load-bearing premise is that the teacher's backward questions and backward reasoning are genuinely inverse to the forward question, and that the consistency filter reliably separates true inverses from spurious ones; if the teacher produces plausible-sounding but not truly inverse reasoning, the student trains on noisy signal.

Editorial extensions

If this is right

At inference the student is prompted exactly like a zero-shot model, yet accuracy is reported to rise by about 6–7 points over standard chain-of-thought distillation, so the backward-thinking gain comes at essentially no extra inference compute.
Using only 10% of the training data, RevThink is reported to outperform unidirectional distillation trained on the full dataset on StrategyQA, indicating the backward-augmented signal is information-dense.
Accuracy transfers to held-out datasets including reversed math problems, suggesting the learned backward skill is not just memorized benchmark-specific patterns.
The method scales with student size, and a 7B-parameter student trained with RevThink is reported to surpass the zero-shot performance of a much larger instruction-tuned model.
RevThink stacks with other augmentation methods such as answer augmentation, producing larger gains than either method alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Gains are likely strongest on tasks with well-defined inverses (arithmetic, multiple-choice classification, date arithmetic) and weakest on tasks like number theory whose backward problem is not unique; the reported per-topic MATH breakdown already points this way, and a direct test would rank datasets by invertibility and correlate it with the gain.
Because the backward-question objective alone (without backward reasoning) already improves accuracy, part of the effect may be a general regularization against overfitting to question surface form rather than specifically inverse computation; this could be probed by training with backward content that is plausible but not truly inverse while still passing the consistency filter.
A natural extension is to remove the teacher entirely: let the student generate its own backward questions and filter them by self-consistency, turning the pipeline into a self-distillation loop that would no longer require access to a larger model.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The result is probably real: training a 7B student on forward CoT plus teacher-generated backward questions and backward reasoning improves accuracy across 12 datasets. The paper is worth engaging. But the central mechanism—'learning to think backward'—is not actually verified, because the consistency filter only checks that the backward reasoning's answer appears in the original question, not that the backward question is a genuine inverse. That gap matters for the story, not necessarily for the recipe.

What is new here is the specific combination: a single pipeline that augments data with backward questions and backward reasoning, then trains the student with three joint objectives. The ablations are informative. The w/o-verification ablation shows filtering helps, and Table 2 shows that joint objectives beat separating them into instruction-tuned instances. That is useful evidence, and the gains over SKD and answer augmentation are consistent across two student models.

The stress-test concern is correct and should be addressed. The verification prompt in Appendix C accepts any Qb whose answer appears in Q1 and is judged correct. A same-answer paraphrase like '2+2=?' versus '2*2=?' would pass. The paper never measures whether Qb is actually inverse, so the reported gains could partly come from extra related QA supervision rather than reverse thinking. This does not sink the method—the gains are still gains—but the title's claim is under-supported. A human evaluation of Qb quality or a stricter inverse filter would tighten the attribution.

Two smaller issues: the abstract's sample-efficiency claim is based only on StrategyQA, not the broader pattern the text implies, and there are no error bars or significance tests. The code and data link is still a placeholder, which is an easy fix and should be required before publication. The held-out generalization results are suggestive but cover only four datasets.

This paper is for readers working on knowledge distillation or data augmentation for reasoning. The recipe is simple, the experiments are broad, and the mechanism question is genuinely interesting. I would send it out for peer review, asking for code/data, error bars, and an evaluation of backward question quality. The conditional verdict is about right: accept the empirical recipe with modest confidence, but treat the 'reverse thinking' explanation as not yet established.

Referee Report

3 major / 5 minor

Summary. The paper proposes Reverse-Enhanced Thinking (RevThink), a training-time framework in which a teacher model generates, for each question, a forward chain of thought, a backward question, and backward reasoning; a consistency check keeps tuples whose backward answer matches the original question; and a student model is trained with three objectives: Q→R_f, Q→Q_b, and Q_b→R_b, while inference uses only forward reasoning. Experiments are reported on 12 datasets across commonsense, math, tabular, NLI, and logical reasoning, with Mistral-7B-Instruct and Gemma-7B-Instruct as students, comparing against zero-shot, SKD, Distilling Step-by-Step, and several data-augmentation baselines. The paper also reports sample-efficiency results, OOD generalization to four held-out datasets, scaling across model sizes, and a combination with Answer Augmentation.

Significance. If the empirical results hold, RevThink is a simple and practical training-time intervention that improves 7B-scale reasoning models across diverse domains while keeping test-time cost identical to zero-shot prompting. The paper's strengths are the breadth of evaluation (two student models, several distillation and augmentation baselines, ablations, held-out datasets, scaling), the positive out-of-distribution generalization results in Table 3, and the clean multi-task objective in Eq. (1). However, the central attribution of the gains to 'reverse thinking' is not directly validated: the data-generation and filtering procedure checks answer consistency but does not verify that the backward question is a genuine inverse of the original question. In addition, all results are point estimates without error bars or significance tests, and the abstract's headline sample-efficiency claim is supported on only one dataset. These issues do not by themselves invalidate the empirical finding, but they need to be addressed before the mechanism claim can be accepted.

major comments (3)

[Section 3.2 / Appendices B and C] The consistency filter does not establish that Q_b is a genuine inverse of Q. The backward-question prompt in Appendix B explicitly tells the teacher that 'The correct answer in your generated question must be present in the input question,' and the verification prompt in Appendix C accepts a pair when A2 'can be found in Q1' and is judged correct. A tuple such as Q='2+2=?' with answer 4 and Q_b='2*2=?' with answer 4 would therefore pass, even though Q_b is not an inverse of Q. The w/o-Verification ablation in Table 5 shows that filtering helps, but it does not separate selection of true inverses from selection of answer-consistent examples. To support the paper's central claim, please measure Q_b inverseness on a sample (e.g., human or LLM judgment against a clear rubric) and include a control condition in which Q_b is an answer-preserving paraphrase rather than an inverse; without such evidence the reported gains could come from extra related QA supervision rather than from learning to think backward.
[Abstract and Section 5.2 (Fig. 3)] The sample-efficiency claim that 'using only 10% of the correct forward reasoning from the training data, it outperforms a standard fine-tuning method trained on 10x more forward reasoning' is supported only on StrategyQA. In Figure 3, the ANLI, GSM8K, and Date Understanding panels do not show the 10% point crossing the full-data SFT accuracy; the paper's own text in Section 5.2 identifies StrategyQA as the crossover case. Please either qualify the abstract claim to the datasets where the crossover holds or provide additional datasets exhibiting the same effect; as written, the abstract overstates the evidence.
[Tables 1, 3, 4 and Figures 3-5] All reported results are point estimates from single runs with greedy decoding; no error bars, random seeds, or significance tests are reported. Because some of the headline differences are small (e.g., a few points on ARC, ANLI, and Gemma GSM8K in Table 1, and several ablation comparisons in Figure 4), the reader cannot judge which gains are reliable. Please report means and standard deviations over at least three seeds, or provide bootstrap confidence intervals, for the main tables and the key ablations.

minor comments (5)

[Footnote 1] The code/data availability statement points to 'this url' rather than an actual link; please provide a working URL so that the experiments are reproducible.
[Section 3.3, Eq. (1)] The phrase 'conditioned on the (b) generated backward question' is ambiguous: in Eq. (1), objective (c) uses the teacher-provided Q_b as input, not a question generated by the student model at training time. Please clarify that the student is trained on teacher-generated Q_b separately from objective (b).
[Section 4 / Table 6] The citation 'Date Understanding (bench authors, 2023)' should be formatted properly, e.g., as BIG-Bench authors (2023), matching the reference entry.
[Figure 4 caption] The bar labels in Figure 4 are difficult to parse; please make them consistent with the caption's X→Y notation so that each bar's learning source is immediately clear.
[Section 5.2] The sentence 'REVTHINK shows sample efficiency by largely outperforming SFT given any portion of the training data' is stronger than the plotted evidence in Figure 3; please soften it to match what the figure actually shows.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: RevThink is an empirical data-augmentation and distillation pipeline whose central claim rests on held-out evaluations, not on a definitional or self-cited reduction.

full rationale

The paper's derivation chain is: the teacher model generates forward reasoning, a backward question, and backward reasoning (Section 3.2); a ground-truth correctness check and a teacher consistency check filter the augmented set; the student is trained with the three losses in Eq. (1), namely Q-to-Rf, Q-to-Qb, and Qb-to-Rb; and test-time performance is measured by prompting only Q and comparing against held-out test sets (Tables 1 and 3). No trained parameter or predicted quantity is defined in terms of the target result, and no baseline number is obtained from a fitted value. The self-citations (e.g., Chen et al. 2023 and 2024 in Related Work; Hsieh et al. 2023, which shares authors) are contextual and are not the load-bearing justification for the framework. The Appendix C verification filter checks that the backward answer A2 is present and correct in Q1 rather than proving that the backward question Qb is a genuine inverse; this is a legitimate concern about mechanism attribution and data quality, but it is not circularity, because the reported gains are not derived by assuming the inverse property. The claimed improvement is an empirical result against external benchmarks, so no circular step can be exhibited.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The central claim depends on a set of design choices (teacher, prompts, hyperparameters) and assumptions about data quality. There are no geometric or physical free parameters; the free parameters are standard ML hyperparameters. No new entities are postulated.

free parameters (7)

LoRA rank = 32
Rank for Low-Rank Adaptation; set by authors, not tuned per dataset.
Learning rate = 5e-6 (Mistral), 2e-4 (Gemma)
Per-model learning rates; no search reported.
Training epochs = 3 (MATH/GSM8K), 10 (other domains)
Epoch count differs by domain; hand-chosen.
Teacher model = Gemini-1.5-Pro-001
The teacher generates and verifies all backward data; a different teacher may yield different results.
Student model = Mistral-7B-Instruct-v0.3, Gemma-7B-Instruct
Two instruction-tuned 7B models; choice affects results.
Data filtering criteria = Correct forward reasoning + teacher-verified consistency
Retention rate varies by dataset (e.g., ANLI 883/2000); this selection shapes the training distribution.
Backward question prompt templates = Hand-crafted prompts with in-context examples (Appendix B)
The quality of backward questions depends on these templates.

assumptions (4)

domain assumption Teacher-generated backward reasoning, verified for consistency by the same teacher, is a reliable training signal for improving forward reasoning.
Section 3.2; if the backward data is noisy, the student may not benefit.
domain assumption The answer extraction function g(Rf) correctly identifies whether the model's reasoning leads to the ground-truth answer.
Section 3.2; the filtering step assumes g is accurate.
standard math Cross-entropy next-token prediction is an appropriate objective for distilling reasoning traces.
Eq. 1; standard in the distillation literature (e.g., SKD).
domain assumption The chosen benchmarks are representative of reasoning ability and the held-out sets measure generalization rather than memorization.
Section 4; contamination or benchmark-specific tuning would weaken the claim.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Reverse Thinking Makes LLMs Stronger Reasoners." pith.science (2026). https://pith.science/paper/BDNQKISA

@misc{pith2026241119865,
  author       = {Pith},
  title        = {Pith review of: Reverse Thinking Makes LLMs Stronger Reasoners},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BDNQKISA}},
  note         = {Machine review of arXiv:2411.19865}
}

read the original abstract

Reverse thinking plays a crucial role in human reasoning. Humans can reason not only from a problem to a solution but also in reverse, i.e., start from the solution and reason towards the problem. This often enhances overall reasoning performance as it enables consistency checks between their forward and backward thinking. To enable Large Language Models (LLMs) to perform reverse thinking, we introduce Reverse-Enhanced Thinking (RevThink), a framework composed of data augmentation and learning objectives. In RevThink, we augment the dataset by collecting structured forward-backward reasoning from a teacher model, consisting of: (1) the original question, (2) forward reasoning, (3) backward question, and (4) backward reasoning. We then employ three objectives to train a smaller student model in a multi-task learning fashion: (a) generate forward reasoning from a question, (b) generate a backward question from a question, and (c) generate backward reasoning from the backward question. Experiments across 12 datasets covering commonsense, math, and logical reasoning show an average 13.53% improvement over the student model's zero-shot performance and a 6.84% improvement over the strongest knowledge distillation baselines. Moreover, our method demonstrates sample efficiency -- using only 10% of the correct forward reasoning from the training data, it outperforms a standard fine-tuning method trained on 10x more forward reasoning. RevThink also exhibits strong generalization to out-of-distribution held-out datasets.

Figures

Figures reproduced from arXiv: 2411.19865 by the authors.

**Figure 2.** REVTHINK consists of two stages: (1) Data augmentation and (2) Student model learning. First, given a dataset D = {(Q(i) , A(i) )} n i=1, we augment it by prompting the teacher model to generate forward reasoning, backward question, and backward reasoning. We keep instances only with correct forward reasoning (validated by the ground truth) and consistent forward-backward reasoning (validated by the teacher model). … view at source ↗

**Figure 3.** Comparison of REVTHINK and the SFT baseline with different sample sizes. Notably, REVTHINK shows sample efficiency by largely outperforming SFT given any portion of the training data. Furthermore, our method with only 10% of training data outperforms SFT with the full training data on StrategyQA. StrategyQA ANLI GSM8K Date 40 50 60 70 80 Accuracy (%) 52.4 38.8 54.7 50.1 69.4 44.9 57.1 48.5 72.0 43.2 56.6 56.5 70.3 4… view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Comparison of different learning sources and the combination of input/output. We denote [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: The average token counts per sample used in training versus the test-time accuracy. The dashed line shows the regression over the baselines. Our method outperforms the baselines with only a slight increase in token count. Note that REVTHINK generates a comparable numb…

**Figure 6.** Figure 6: REVTHINK scales effectively with student model size. Notably, Mistral-7B + REVTHINK outperforms Mistral-8x22B (the red dashed line) by 8.36%. REVTHINK scales positively with model size. We apply REVTHINK on StrategyQA using models of varying sizes (all models are inst…

**Figure 7.** Figure 7: The break down analysis of the MATH dataset. [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Answer-Conditioned Chains of Thought Degrade Verifiable-Reasoning Distillation in Large Language Models
cs.CL 2026-07 conditional novelty 6.0 of 10

Answer-conditioned chains of thought are harmful supervision for reasoning distillation—causing up to ~27-point accuracy drops—and the damage is invisible to correctness filters.
VLMs Can Aggregate Scattered Training Patches
cs.CV 2025-06 conditional novelty 6.0 of 10

Open-source VLMs can infer image IDs or safety labels after training only on scattered patches of those images, a capability that can be abused to bypass image moderation.
RECALL: Library-Like Behavior In Language Models is Enhanced by Self-Referencing Causal Cycles
cs.CL 2025-01 reject novelty 3.0 of 10

Repeated phrases in training data can act as bridges that help autoregressive language models recover preceding text, a trick the paper formalizes and exploits with a two-step prompt.

Reference graph

Works this paper leans on

21 extracted references · 18 canonical work pages · cited by 3 Pith papers

[2]

Ensure that the {n} new answer choices are inversely correlated with the {n} input question’s choices

work page
[3]

Make sure only one answer choice in your generated question is correct and reasonable

work page
[4]

Advances in Neural Information Processing Systems, 36

Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629. Shan You, Chang Xu, Chao Xu, and Dache...

arXiv 2022
[5]

The generated question and answer choices should be semantically different from the input question. {in_context_samples} {input_question} Prompt for Backward Question Generation (Math Reasoning Problem) Your task is to generate an inverse question, based on the input question and its correct answer. Follow these rules:

work page
[10]

Use the correct answer from the input question to create a new, related but inverse question

work page
[11]

Make sure there exists only one correct answer in your generated question

work page
[12]

The correct answer in your generated question must be present in the input question

work page
[13]

{in_context_samples} {input_question} The prompt Ibq we use to generate backward questions

The generated question should be semantically different from the input question. {in_context_samples} {input_question} The prompt Ibq we use to generate backward questions. They are general templates, where we insert specific {in_context_samples} depending on the task. Below, we provide the in-context examples for each task. In-context samples for Strateg...

work page

Show all 21 references

[16]

**The Police were a rock band:** Their members were musicians, not law enforcement officers
[17]

**Law enforcement officers have specific powers:** These powers are granted by the government and include the ability to make arrests
[18]

The Police

**Being in a band called "The Police" does not grant legal authority:** Band names are artistic choices and have no bearing on legal powers. **Therefore, the members of The Police could not perform lawful arrests.** The answer is **no** Backward Question (Qb): If the members o...
[19]

OUTPUT: If 5 6 X is 25, what is the value of unknown variable X? INPUT: Twenty-seven increased by twice a number is 39

What is the mean of the separate set of five numbers? INPUT: What is 5 6 of 30? The correct answer is 25. OUTPUT: If 5 6 X is 25, what is the value of unknown variable X? INPUT: Twenty-seven increased by twice a number is 39. What is the number? The correct answer is 6. OUTPUT...

2021
[20]

The Police

The question implies there are two possible meanings for "The Police"
[21]

This is the definition of police officers

One meaning refers to a group that can make lawful arrests. This is the definition of police officers
[22]

The other meaning refers to a group that does not have the authority to make lawful arrests
[23]

Since the question states this group cannot make lawful arrests, they cannot be police officers
[24]

OUTPUT: The mean of one set of six numbers is 24, and the mean of the set of all eleven numbers is

What is the mean of the set of all eleven numbers? The correct answer is 19. OUTPUT: The mean of one set of six numbers is 24, and the mean of the set of all eleven numbers is
[25]

The Police

Therefore, "The Police" in this context likely refers to a group known for something other than law enforcement, making them likely musicians. The answer is (A) Example on GSM8K Original Question (Q): James writes a 3-page letter to 2 different friends twice a week. How many p...

2021
[2016]

In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 86–96, Berlin, Germany

Improving neural machine translation models with monolingual data. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 86–96, Berlin, Germany. Association for Computational Lin- guistics. Xiaoyu Shen, Hui Su...
[2018]

In Proceed- ings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4316–4327, Brussels, Belgium

NEXUS network: Connecting the preceding and the following in dialogue generation. In Proceed- ings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4316–4327, Brussels, Belgium. Association for Computational Linguistics. Noah Shinn, Federico Ca...

2018 arXiv
[2024]

arXiv preprint arXiv:2406.01006

Semcoder: Training code language mod- els with comprehensive semantics. arXiv preprint arXiv:2406.01006. Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenen- baum, and Igor Mordatch. 2023. Improving factual- ity and reasoning in language models through multia- gent debate. a...

2023 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [2]

Ensure that the {n} new answer choices are inversely correlated with the {n} input question’s choices

work page

[2] [3]

Make sure only one answer choice in your generated question is correct and reasonable

work page

[3] [4]

Advances in Neural Information Processing Systems, 36

Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629. Shan You, Chang Xu, Chao Xu, and Dache...

arXiv 2022

[4] [5]

The generated question and answer choices should be semantically different from the input question. {in_context_samples} {input_question} Prompt for Backward Question Generation (Math Reasoning Problem) Your task is to generate an inverse question, based on the input question and its correct answer. Follow these rules:

work page

[5] [10]

Use the correct answer from the input question to create a new, related but inverse question

work page

[6] [11]

Make sure there exists only one correct answer in your generated question

work page

[7] [12]

The correct answer in your generated question must be present in the input question

work page

[8] [13]

{in_context_samples} {input_question} The prompt Ibq we use to generate backward questions

The generated question should be semantically different from the input question. {in_context_samples} {input_question} The prompt Ibq we use to generate backward questions. They are general templates, where we insert specific {in_context_samples} depending on the task. Below, we provide the in-context examples for each task. In-context samples for Strateg...

work page