From Sufficiency to Reflection: Reinforcement-Guided Thinking Quality in Retrieval-Augmented Reasoning for LLMs

Jeff Z. Pan; Jie He; Victor Guti\'errez-Basulto

REVIEW 4 major objections 3 minor 12 references

From Sufficiency to Reflection: Reinforcement-Guided Thinking Quality in Retrieval-Augmented Reasoning for LLMs

T0 review · 4 major / 3 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Reinforcement-trained RAG models reason better when rewards score the thinking, not just the final answer.

desk verdict Useful multi-reward RL recipe for RAG reasoning, but the headline LeTS comparison is uncontrolled; deserves review with revisions. read the letter →

arxiv 2507.22716 v2 pith:LDYNFVJD submitted 2025-07-30 cs.CL

Jie He , Victor Gutiérrez-Basulto , Jeff Z. Pan This is my paper

classification cs.CL

keywords retrieval-augmentedgenerationreinforcementlearningmulti-hopquestionansweringprocessrewardssufficiencyrewardreasoningqualityreflectionmechanismGRPO

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that outcome-only reinforcement learning is not enough for retrieval-augmented reasoning: models trained only on final-answer rewards still fail by retrieving too little, reasoning incorrectly, or producing answers that contradict their own reasoning. The authors propose TIRESRAG-R1, a think-retrieve-reflect training framework with four reward signals that jointly supervise answer correctness, information sufficiency, reasoning quality, and reflection-driven error correction. If the central claim is right, the same RL recipe that produces strong reasoning in math can be made substantially more reliable for knowledge-intensive questions. The paper reports that TIRESRAG-R1 outperforms prior RAG reasoning methods on four multi-hop QA datasets, with an average exact-match improvement of 5.8% over the leading process-reward baseline, and that it generalizes to single-hop open-domain QA.

What carries the argument

The load-bearing machinery is a GRPO training loop over think-search-answer-reflect trajectories, where each trajectory is scored by four rewards: an F1 answer reward, a binary sufficiency reward from a judge that sees the question, the reasoning trajectory, and the gold answer, a continuous thinking reward assessing logical soundness, alignment with retrieved evidence, error awareness, and conciseness, and a reflection reward that is positive only when the model revises a wrong first answer into a correct one. Auxiliary rewards are annealed away late in training, and the advantage is reweighted by a sigmoid function of the average sufficiency score so that harder questions receive stronger optimization pressure. This reward architecture is what converts the paper's three observed failure modes into explicit learning signals.

What would settle it

Train the same TIRESRAG-R1 pipeline with a sufficiency judge that does not receive the gold answer, keeping all other hyperparameters fixed, and compare exact match on the four test sets; if performance drops substantially, the gold-answer-aware judge is contributing answer leakage rather than measuring genuine contextual sufficiency.

Watch

Extended reading notes

Core claim

The central claim is that a retrieval-augmented reasoning model trained with GRPO learns better and more stable multi-hop question answering when its reward function measures the reasoning process itself, not just whether the final answer matches. TIRESRAG-R1 uses a locally deployed judge to score whether the retrieved context plus reasoning is sufficient to derive the gold answer, a separate judge to score reasoning quality on four criteria, and a reflection reward that rewards correcting a wrong first answer while penalizing changing a correct one. The paper also introduces a difficulty-aware advantage reweighting based on the average sufficiency score of a question's rollouts, a consistency penalty, and a group-filtering step that drops saturated questions. On HotpotQA, 2WikiMultiHopQA, MuSiQue, and Bamboogle, the method improves exact match over an answer-only Search-R1 baseline by roughly 3 to 7 points, and it also outperforms the process-reward LeTS baseline by 5.8 EM points on average.

Load-bearing premise

The method assumes that a locally deployed language-model judge can reliably score whether a reasoning trajectory contains enough information to derive the gold answer and how good the reasoning quality is, even though the sufficiency judge also sees the gold answer, so a biased or overly lenient judge would degrade all auxiliary rewards and the training would lose its advantage.

Editorial extensions

If this is right

Training with process rewards makes RL-RAG training more stable: a group-filtering step prevents late-stage collapse, and ablations show that removing any single auxiliary reward drops performance below naive GRPO.
The sufficiency reward is the most load-bearing auxiliary signal: removing it causes the largest performance drop, suggesting that models otherwise tend to answer without gathering enough evidence.
The reflection mechanism addresses the failure where a valid reasoning chain still yields a wrong final answer, a case that outcome-only methods cannot correct during training.
The method generalizes beyond its multi-hop training distribution to single-hop open-domain QA, outperforming baselines on NQ, PopQA, and TriviaQA.
Improved reasoning quality does not come at the cost of inference efficiency: the trained model uses fewer search steps and shorter thinking than naive GRPO.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the sufficiency judge's access to the gold answer makes it lenient, then part of the reported gain may come from answer leakage into the reward signal; this could be tested by training with a judge that does not see the gold answer and comparing the resulting performance.
Because the reflection reward is sparse, a natural extension is to mine reflection-rich trajectories from the trained model and use them for supervised warm-start before RL, which the paper lists as future work but does not itself test.
The difficulty-aware reweighting uses average sufficiency reward as a proxy for question difficulty; one could extend it to difficulty estimates based on retrieval result noise or number of required reasoning hops, which the paper does not explore.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Worth a read if you work on RL-trained RAG. The paper proposes a concrete training recipe—GRPO with sufficiency, thinking, and reflection rewards, plus difficulty reweighting and sample filtering—and shows it consistently beats re-trained baselines on four multi-hop QA datasets. The ablations are actually informative, and the method generalizes to single-hop tasks. That's a solid empirical contribution.

The main caveat is the headline 5.8% EM gain over LeTS. The authors say LeTS has no public code and they take the numbers from the original paper (App. C.3). Every other baseline was retrained in-house on the same 8,148-example set and evaluated with the same script. So the LeTS comparison is not apples-to-apples; differences in training data, splits, normalization, or decoding could explain part of the gap. The claim that they improve over process-and-outcome reward hybridization is therefore currently unquantified. That's a real soft spot, but not fatal—the gains against Search-R1 and R1-Searcher are controlled and consistent.

The other soft spots are smaller. Hyperparameters like retrieval count, penalty weight, and reward weights are tuned on the same test benchmarks (Figs. 3, 5, 8), and there are no significance tests or seed variance. That makes the reported numbers optimistically selected. The dependency on a Qwen3-8B judge for sufficiency and thinking rewards is a risk; the sufficiency prompt includes the gold answer, which may inflate scores. The authors acknowledge this in the Limitations section, which is honest, but the degree of reliance is high.

What the paper does well: the preliminary analysis of overthinking/underthinking and the error taxonomy (Table 2) is genuinely useful. The ablation table (Table 7) shows each reward component matters, and the filter module is critical for stability—that's a nice practical insight. The prompts and hyperparameters are detailed enough to reproduce, and they ship code and data.

Who this is for: researchers pushing on process rewards for agentic RAG. It won't change the world, but it's a meaningful step beyond outcome-only rewards. It deserves serious peer review, but a referee should push for a controlled LeTS rerun or a reframed claim, plus some variance reporting. I'd take it as a workshop or main-conference paper with revisions.

Referee Report

4 major / 3 minor

Summary. The paper proposes TIRESRAG-R1, a reinforcement-learning framework for retrieval-augmented reasoning that combines a think–retrieve–reflect generation process with a multi-dimensional reward function (answer, sufficiency, thinking quality, reflection), a difficulty-aware advantage reweighting scheme, and a group-filtering mechanism for training stability. The authors report experiments on HotpotQA, 2WikiMultiHopQA, MuSiQue, and Bamboogle, claiming average EM gains of 5.8 points over the LeTS baseline and consistent improvements over re-trained RL baselines, plus generalization to single-hop benchmarks. They also provide failure-mode analyses, ablations, and efficiency analyses.

Significance. If the results hold, the paper makes a useful contribution by showing that process-level rewards for retrieval sufficiency and reasoning quality, combined with reflection, can improve both accuracy and reasoning stability in small RL-trained RAG models. The strengths are the systematic failure analysis, the component ablations, the compatibility check with Reinforce++, the efficiency comparison, and the release of code and data. The main quantitative claims, however, are currently limited by an uncontrolled comparison to LeTS and by apparent selection of hyperparameters on the test benchmarks, so the extent of the improvement over the state of the art is not yet firmly established.

major comments (4)

[5.1, Table 3, App. C.3] The headline claim of an average 5.8-point EM improvement over LeTS is not based on a controlled comparison. Appendix C.3 states that LeTS has no public code and its results are reproduced from the original paper, whereas every other RL baseline was retrained on the same 8,148-example training set and evaluated with the same script and splits. If LeTS used different training data, test splits, answer normalization, backbone, or decoding settings, the reported gain could be an artifact of protocol differences. This is load-bearing because LeTS is the relevant process-reward baseline for the paper's contribution. Please either rerun LeTS under the unified protocol when code becomes available, or explicitly present the comparison as reference-based and remove the quantitative "additional gains of 5.8 and 4.2 points" phrasing from Section 5.1.
[5, App. C.1, Figs. 3, 5, 8] Several hyperparameters appear to be selected on the same four test benchmarks used for the final report: the number of retrieved documents, the consistency penalty weight, the reward-mixture weights, the annealing schedule, and the reward backbone. No confidence intervals, standard deviations across seeds, or significance tests are reported. As a result, the reported improvements may be optimistically selected and the stability claims are not quantified. Please report at least mean and standard deviation over multiple seeds for the main table, or use a validation split for hyperparameter selection and report test-set numbers for the chosen configuration.
[4.3, Eqs. (2)-(3), App. J] The sufficiency and thinking rewards are produced by a locally deployed Qwen3-8B judge, but the paper does not validate this judge against human annotations or an alternative judge. Because these rewards are the main auxiliary training signals, their reliability is load-bearing for the method. Please report judge agreement on a sample (e.g., 100 trajectories scored by humans) or a comparison with a stronger judge, and discuss how the inclusion of the gold answer in the sufficiency prompt (Fig. 11) affects leniency. The backbone sensitivity results in Fig. 8 are a useful start, but they do not establish absolute accuracy.
[4.4 'Group Filtering' and Algorithm 1, line 14] The pseudocode in Algorithm 1 appears to invert the group-filtering logic. The main text says the filter removes saturated queries, i.e., groups where all rollouts are correct or all incorrect, but the condition 'if 0.1 < r_i < 0.9 for all i' would keep only groups where every rollout has an intermediate reward, and would discard the mixed groups that carry the most learning signal. Please correct the condition to match the intended filter (e.g., skip a group when max_i r_i < 0.1 or min_i r_i > 0.9).

minor comments (3)

[Eq. (7)] The consistency penalty term appears to use AT_i twice; the surrounding text suggests it should be AS_i · AT_i · AA_i. Please check and correct.
[Algorithm 1, line 2] The dynamic weight formula is written as exp((T−0.9t)/10), which differs from the main-text formula exp((t−0.9T)/10) in Section 4.3; the two schedules behave differently early in training. Please align the pseudocode with the main text.
[Abstract and Section 3.1] The claim of being 'the first to define overthinking and underthinking in RAG' should be moderated or supported with a comparison to prior definitions of overthinking and underthinking in reasoning models, since the terms have been used in adjacent RL-reasoning literature.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: reward definitions and benchmark evaluation are externally grounded; the LeTS comparison and test-set hyperparameter tuning are protocol-fairness concerns, not derivation-level circularity.

full rationale

TIRESRAG-R1 is an empirical RL method. The answer, sufficiency, thinking, and reflection rewards (Eqs. 1–5) are computed from the gold answer, a local LLM judge, and CEM; they are training signals, not predictions. Final evaluation uses EM, F1, CEM, and a separate GPT-4o judge (App. C.2), independent of the training reward backbone. The difficulty-aware reweighting (Eq. 6) does define difficulty via the policy's own sufficiency score, but it is a weighting heuristic whose effect is tested by ablation (Table 7); it does not make the benchmark outcome equal to its input. The only author self-citation is the Research baseline (Chen et al. 2025), which is retrained in-house and is not load-bearing. The paper honestly reports scope limits (Qwen-3B only, reward-model quality, reflection sparsity). Two empirical caveats remain: LeTS numbers are imported from the original paper (App. C.3) rather than rerun, so the 5.8% headline gain is not a controlled comparison, and hyperparameters are explored on the four test sets (App. G.1, Figs. 3, 5, 8), risking optimistic selection. These affect confidence in the numbers, but they are not circular derivations: no claimed result is equivalent, by the paper's equations, to its own inputs.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The central claim rests on an external LLM judge for reward scoring and on several tuned hyperparameters; these are the main elements pulled in without independent evidence.

free parameters (5)

Difficulty reweighting parameters A, B, rho0, k = A=0.4, B=1.5, rho0=0.75, k=10.0
Tuned on the four evaluation datasets; sensitivity shown in Appendix G.1 and Figure 3.
Reward weights w_t, w_s, w_r = 0.6, 0.3, 0.3
Chosen via ablation; the paper reports that giving the thinking reward the highest weight works best.
Dynamic weight schedule offset and steepness = offset 0.9T, steepness 10
Chosen from annealing-strategy comparison in Appendix G.2 and Figure 5.
Consistency penalty lambda_p = 0.1
Penalty magnitude for advantage consistency; sensitivity analysis shown in Figure 3.
Top-k retrieval = 5
Retrieves 5 documents per search; Figure 3 shows 5 outperforms 3 and 7.

assumptions (3)

domain assumption Qwen3-8B judge scores sufficiency and thinking quality accurately
The three auxiliary rewards are produced by this judge; if its scores are unreliable the training signal degrades. Invoked in Section 4.3 and Appendix J.
domain assumption BGE-large-en-v1.5 with KILT 2019 Wikipedia supplies enough evidence for the QA datasets
Retrieval quality is assumed sufficient for the queries; all models use the same retriever and corpus. Invoked in Appendices B and C.1.
domain assumption The GRPO algorithm and F1 answer reward are valid training choices
The method builds on DeepSeek-R1's GRPO; the paper does not re-derive it. Invoked in Section 4.2.

how reviews work

0 comments

Cite this review

Pith. "Pith review of From Sufficiency to Reflection: Reinforcement-Guided Thinking Quality in Retrieval-Augmented Reasoning for LLMs." pith.science (2026). https://pith.science/paper/LDYNFVJD

@misc{pith2026250722716,
  author       = {Pith},
  title        = {Pith review of: From Sufficiency to Reflection: Reinforcement-Guided Thinking Quality in Retrieval-Augmented Reasoning for LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LDYNFVJD}},
  note         = {Machine review of arXiv:2507.22716}
}

read the original abstract

Reinforcement learning-based retrieval-augmented generation (RAG) methods enhance the reasoning abilities of large language models (LLMs). However, most rely only on final-answer rewards, overlooking intermediate reasoning quality. This paper analyzes existing RAG reasoning models and identifies three main failure patterns: (1) information insufficiency, meaning the model fails to retrieve adequate support; (2) faulty reasoning, where logical or content-level flaws appear despite sufficient information; and (3) answer-reasoning inconsistency, where a valid reasoning chain leads to a mismatched final answer. We propose TIRESRAG-R1, a novel framework using a think-retrieve-reflect process and a multi-dimensional reward system to improve reasoning and stability. TIRESRAG-R1 introduces: (1) a sufficiency reward to encourage thorough retrieval; (2) a reasoning quality reward to assess the rationality and accuracy of the reasoning chain; and (3) a reflection reward to detect and revise errors. It also employs a difficulty-aware reweighting strategy and training sample filtering to boost performance on complex tasks. Experiments on four multi-hop QA datasets show that TIRESRAG-R1 outperforms prior RAG methods and generalizes well to single-hop tasks. The code and data are available at: https://github.com/probe2/TIRESRAG-R1.

Figures

Figures reproduced from arXiv: 2507.22716 by the authors.

**Figure 1.** An example showing different reasoning trajectories for answering a multi-hop query. It compares insufficient information, incorrect predictions, and fully correct reasoning. et al., 2022), which enables LLMs to generate intermediate reasoning steps before arriving at a final answer, significantly enhancing performance on reasoning tasks (Wang et al., 2023; Pan et al., 2023; Snell et al., 2025). Despite these adv… view at source ↗

**Figure 2.** Overall architecture of TIRESRAG-R1, which integrates search, reasoning, and reflection with a dynamic [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Effect of different hyperparameters. Method Hotpotqa 2wikimultihopqa Musique Bamboogle EM F1 CEM EM F1 CEM EM F1 ACC CEM F1 CEM Naive GRPO 36.4 48.5 41.6 46.4 53.1 54.0 16.2 26.0 19.6 36.0 49.2 40.8 TIRESRAG-R1-Instruct 41.0 54.2 46.0 52.8 59.6 60.8 19.4 30.0 23.2 44.0 54.7 47.2 w/o Filter 18.8 24.6 26.8 21.4 26.5 28.8 6.0 11.4 9.4 19.2 28.8 25.6 w/o Difficulty 38.2 50.4 43.6 49.2 54.0 55.4 17.0 27.0 21.4 35.2 49.3 … view at source ↗

Figures from the paper (12 more)

**Figure 4.** Figure 4: Visualization of different reward weight [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]

**Figure 5.** Figure 5: Comparison of different reward weight scheduling strategies. • Impact of number of retrieved documents. As the number of retrieved documents increases from 3 to 5 (ours), the model performance improves, with EM increasing by 3.95 points on average. When increasing from…

**Figure 7.** Figure 7: Training dynamics of thinking rewards over [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]

**Figure 8.** Figure 8: Comparison of different reward backbone models. cient and thinking scoring. We observe consistent improvements with Qwen3-3B: on Musique, EM improves from 16.6 to 17.6; on Bamboogle, EM improves from 41.6 to 42.8. A similar trend is seen comparing Qwen2.5-3B and Qwen2.…

**Figure 9.** Figure 9: Prompt template for instruction model training. [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]

**Figure 10.** Figure 10: Prompt template for base model training. [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]

**Figure 11.** Figure 11: Prompt used to evaluate context sufficiency. [PITH_FULL_IMAGE:figures/full_fig_p022_11.png]

**Figure 12.** Figure 12: Prompt used to evaluate thinking quality [PITH_FULL_IMAGE:figures/full_fig_p023_12.png]

**Figure 13.** Figure 13: Prompt used for LLM-as-Judge evaluation. [PITH_FULL_IMAGE:figures/full_fig_p023_13.png]

**Figure 14.** Figure 14: Case study where the naive GRPO model, despite having sufficient information, mistakenly infers the [PITH_FULL_IMAGE:figures/full_fig_p024_14.png]

**Figure 15.** Figure 15: Case study where the naive GRPO model, despite retrieving sufficient information, follows an incorrect [PITH_FULL_IMAGE:figures/full_fig_p025_15.png]

**Figure 16.** Figure 16: Case study where the naive GRPO model, despite retrieving the correct birth years (1872 for Dennis E. [PITH_FULL_IMAGE:figures/full_fig_p026_16.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

12 extracted references · 7 canonical work pages

[1]

Logical Soundness - Is the reasoning coherent and structured?

work page
[2]

Contextual Alignment - Does it use retrieved evidence correctly?

work page
[3]

Error Awareness - Does it avoid unsupported assumptions?

work page
[4]

Chunyang Meng, Shijie Song, Haogang Tong, Maolin Pan, and Yang Yu

Understanding r1-zero-like training: A critical perspective.Preprint, arXiv:2503.20783. Chunyang Meng, Shijie Song, Haogang Tong, Maolin Pan, and Yang Yu. 2024. Deepscaler: Holistic au- toscaling for microservices based on spatiotemporal gnn with adaptive graph learning. InProceedings of the 38th IEEE/ACM International Conference on Au- tomated Software E...

arXiv 2024
[5]

Shamane Siriwardhana, Rivindu Weerasekera, Elliott Wen, Tharindu Kaluarachchi, Rajib Rana, and Suranga Nanayakkara

Search and refine during think: Autonomous retrieval-augmented reasoning of llms.Preprint, arXiv:2505.11277. Shamane Siriwardhana, Rivindu Weerasekera, Elliott Wen, Tharindu Kaluarachchi, Rajib Rana, and Suranga Nanayakkara. 2023. Improving the domain adaptation of retrieval augmented generation (RAG) models for open domain question answering.Trans- actio...

arXiv 2023
[6]

decomposes tasks into executable plans for targeted retrieval, and ITER-RETGEN (Shao et al.,

work page
[8]

all-correct

uses answer reward plus a format reward. Research(Chen et al., 2025) also incorporates format reward. To ensure fairness, we re-trained these RL-based baselines on our training set us- ing the authors’ released code and hyperparameter settings, without using their checkpoints.LeTS (Zhang et al., 2025a) combines step-level rewards with answer rewards and i...

work page 2025
[12]

Important: - Judge only the thinking process, not the answer

Clarity and Precision - Is it concise, relevant, and non-redundant? Scoring: 0.0: Completely flawed reasoning 1.0: Perfect reasoning Intermediate (e.g., 0.3, 0.7) are allowed. Important: - Judge only the thinking process, not the answer. - Reward accurate, grounded, and structured reasoning. Your Output: A single float-type score from {{0.0, 0.1, 0.2, ......

work page 1956

Show all 12 references

[2023]

search–thinking–answer

incorporates intermediate generation to itera- tively reformulate queries. In parallel, modular and hybrid RAG frameworks (Gao et al., 2024b; Zhang et al., 2024; Zhou et al., 2024) have introduced com- ponentized systems that integrate query rewriting, evidence aggregation, an...

2025
[2024]

Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasu- pat, and Ming-Wei Chang

The llama 3 herd of models.Preprint, arXiv:2407.21783. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasu- pat, and Ming-Wei Chang. 2020. Realm: retrieval- augmented language model pre-training. InProceed- ings of the 37th International Conference on Machine Learning, ICML’20. J...

2020 arXiv
[2025]

Anoushka Gade and Jorjeta Jetcheva

Deepseek-r1: Incentivizing reasoning capa- bility in llms via reinforcement learning.Preprint, arXiv:2501.12948. Anoushka Gade and Jorjeta Jetcheva. 2024. It’s about time: Incorporating temporality in re- trieval augmented language models.Preprint, arXiv:2401.13222. Yunfan Gao...

2024 arXiv
[9474]

Jiarui Li, Ye Yuan, and Zehua Zhang

Curran Associates, Inc. Jiarui Li, Ye Yuan, and Zehua Zhang. 2024a. Enhancing llm factual accuracy with rag to counter hallucina- tions: A case study on domain-specific queries in pri- vate knowledge-bases.Preprint, arXiv:2403.10446. Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao...

arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Logical Soundness - Is the reasoning coherent and structured?

work page

[2] [2]

Contextual Alignment - Does it use retrieved evidence correctly?

work page

[3] [3]

Error Awareness - Does it avoid unsupported assumptions?

work page

[4] [4]

Chunyang Meng, Shijie Song, Haogang Tong, Maolin Pan, and Yang Yu

Understanding r1-zero-like training: A critical perspective.Preprint, arXiv:2503.20783. Chunyang Meng, Shijie Song, Haogang Tong, Maolin Pan, and Yang Yu. 2024. Deepscaler: Holistic au- toscaling for microservices based on spatiotemporal gnn with adaptive graph learning. InProceedings of the 38th IEEE/ACM International Conference on Au- tomated Software E...

arXiv 2024

[5] [5]

Shamane Siriwardhana, Rivindu Weerasekera, Elliott Wen, Tharindu Kaluarachchi, Rajib Rana, and Suranga Nanayakkara

Search and refine during think: Autonomous retrieval-augmented reasoning of llms.Preprint, arXiv:2505.11277. Shamane Siriwardhana, Rivindu Weerasekera, Elliott Wen, Tharindu Kaluarachchi, Rajib Rana, and Suranga Nanayakkara. 2023. Improving the domain adaptation of retrieval augmented generation (RAG) models for open domain question answering.Trans- actio...

arXiv 2023

[6] [6]

decomposes tasks into executable plans for targeted retrieval, and ITER-RETGEN (Shao et al.,

work page

[7] [8]

all-correct

uses answer reward plus a format reward. Research(Chen et al., 2025) also incorporates format reward. To ensure fairness, we re-trained these RL-based baselines on our training set us- ing the authors’ released code and hyperparameter settings, without using their checkpoints.LeTS (Zhang et al., 2025a) combines step-level rewards with answer rewards and i...

work page 2025

[8] [12]

Important: - Judge only the thinking process, not the answer

Clarity and Precision - Is it concise, relevant, and non-redundant? Scoring: 0.0: Completely flawed reasoning 1.0: Perfect reasoning Intermediate (e.g., 0.3, 0.7) are allowed. Important: - Judge only the thinking process, not the answer. - Reward accurate, grounded, and structured reasoning. Your Output: A single float-type score from {{0.0, 0.1, 0.2, ......

work page 1956