Pith. sign in

REVIEW 5 major objections 4 minor 1 cited by

Retrieval Feedback Memory Enhancement Large Model Retrieval Generation Method

T0 review · 5 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read RFM-RAG stores curated passages in a growing evidence pool, trains a model to stop retrieval when evidence is sufficient, and reports higher accuracy than prior RAG methods on three QA benchmarks.

desk verdict RFM-RAG's accuracy claims are confounded because GPT-3.5-turbo is used inside the proposed pipeline but not in the baselines; the RAG idea is plausible but the evaluation needs a controlled ablation. read the letter →

arxiv 2508.17862 v1 pith:3XCQZ7JJ submitted 2025-08-25 cs.IR

Leqian Li , Dianxi Shi , Jialu Zhou , Xinyu Wei , Mingyue Yang , Songchang Jin , Shaowu Yang This is my paper
classification cs.IR
keywords retrieval-augmentedgenerationiterativeretrievaldynamicevidencepoolknowledgegapdetectionfeedbackmodelmulti-hopquestionansweringopen-domainQAtermination
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's claim is that retrieval-augmented generation works better when retrieval is stateful: instead of independent rounds of retrieval, RFM-RAG accumulates curated passages in a growing evidence pool and only retrieves more when the pool is judged incomplete. The loop is driven by knowledge-gap detection: relational triples extracted from the question, with unknown values as placeholders, show which entities are still under-covered, and those entities become the next query. A small trained model, the R-Feedback Model, combines an entity-coverage score and a semantic-relevance score, and decides when the evidence is sufficient; only then is the final answer generated. Across 2WikiMultihopQA, NaturalQA, and StrategyQA, the method reports exact-match and accuracy gains over no retrieval, single-round RAG, and adaptive retrieval baselines, including a 17.9-point exact-match gain on Mistral-7b over no retrieval. The same stop signal also reduces retrieval steps by 12-35% relative to a fixed three-round policy, cutting latency and avoiding the noise that extra, redundant passages can add.

What carries the argument

The load-bearing mechanism is the loop formed by the dynamic evidence pool and the R-Feedback Model. Each round retrieves passages for a generated query qi, an LLM uses chain-of-thought prompting to filter, organize, and deduplicate them into Ei, and these curated passages accumulate as E = {E0, E1, ...}. The R-Feedback Model is a small feed-forward network with two hidden layers; its input is a syntactic coverage feature Sf (the average, over entities extracted from the question, of how often each entity appears in the pool relative to pool length) fused with a semantic relevance feature Gf produced by a cross-encoder over the question and the pool. Its output logit determines whether the p

What would settle it

A direct test would compare the R-Feedback Model's stop decisions on a separate set of NaturalQA/StrategyQA examples annotated as sufficiently answered or not; if decisions are no better than random or than always using three retrieval rounds, the reported gains cannot be attributed to the learned termination. A cheaper check is the distribution of retrieval steps: RFM-RAG should stop well before three rounds on easy questions and use three on hard ones; a flat distribution would indicate the model is not actually tracking sufficiency.

Watch

Extended reading notes

Core claim

The central claim is that a persistent, curated evidence pool plus an explicit learned sufficiency check outperforms both single-round retrieval-augmented generation and adaptive retrieval methods. The paper characterizes knowledge gaps as entities and relational triples: an LLM extracts key entities from the question, replaces missing values with placeholders, computes an entity-coverage feature against the accumulated evidence, and builds the next query from entities that remain under-covered. The R-Feedback Model is a small two-hidden-layer network that takes the average coverage feature and a cross-encoder semantic-relevance feature and outputs a stop/continue decision; its training pair

Load-bearing premise

The R-Feedback Model that decides when to stop retrieving is trained only on artificially constructed example pairs from 2WikiMultihopQA and then used unchanged on NaturalQA and StrategyQA; the method stands on those stopping judgments transferring to other tasks and corpora, and the paper reports no direct test of that transfer.

Editorial extensions

If this is right

  • RAG systems should treat retrieval as a stateful accumulation process: passages found in earlier rounds remain available for the final answer, which prevents previously retrieved facts from being lost between retrievals.
  • A learned termination signal can replace fixed retrieval budgets: RFM-RAG's R-Feedback Model stops early on enough questions to cut retrieval steps by 12-35% while keeping or improving accuracy.
  • Explicit entity-gap detection is a workable query-generation strategy for multi-hop questions; queries built from under-covered entities retrieve missing facts rather than simply rephrasing the original question.
  • Smaller language models can reach competitive QA accuracy when the evidence pool is complete, suggesting that retrieval-side completeness matters separately from generator-side reasoning strength.
  • With the retrieval cap at three rounds, the method is practical for latency-sensitive settings; the sufficiency model's early exit is what avoids unnecessary third-round retrievals.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A straightforward extension the paper does not explore is to make the R-Feedback Model output a ranked list of under-covered entities instead of a binary stop signal, turning termination into targeted retrieval; the per-entity coverage features already contain that information.
  • The strongest unresolved question is cross-task transfer of the sufficiency classifier, since it is trained only on 2WikiMultihopQA. Measuring its stop decisions against reference labels on NaturalQA and StrategyQA would show whether the reported latency and accuracy gains generalize.
  • A minimal diagnostic would test whether the simple entity-coverage average alone reproduces most of the R-Feedback Model's decisions; if it does, the method's main contribution is the evidence-pool construction and gap-based query generation rather than the learned termination.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes RFM-RAG, an iterative retrieval-augmented generation framework that maintains a dynamic evidence pool. Each retrieval round uses an LLM (GPT-3.5-turbo) to curate retrieved passages, extracts relational triples with <X> placeholders to form knowledge gaps, and generates new queries from those gaps. A separate R-Feedback Model—a feed-forward classifier over entity-coverage and cross-encoder semantic features—decides when the evidence pool is sufficient and terminates retrieval. The method is evaluated on 2WikiMultihopQA, NaturalQA, and StrategyQA using Gemma-2b and Mistral-7b, against no-retrieval, vanilla RAG, Adaptive-RAG, DRAGIN, and Probing-RAG. The paper reports average EM/ACC gains and reduced retrieval steps.

Significance. The paper's core idea—stateful evidence accumulation, targeted query generation from identified knowledge gaps, and learned early stopping—is plausible and potentially useful. The authors also state an intent to release a training dataset for the R-Feedback Model, which could be a contribution. Strengths include evaluation on multiple datasets and two base LLMs, a case study, and a clear architectural description. However, the empirical evidence as presented is not sufficient to support the central claim. The proposed pipeline uses GPT-3.5-turbo for evidence curation and query generation while baselines use only the small open generation model; the evaluation is a single run with no error bars; the R-Feedback Model is trained and evaluated on the same dataset family without out-of-distribution validation; and some per-dataset results contradict the averaged claims. These issues are load-bearing for the claim that RFM-RAG outperforms previous methods.

major comments (5)
  1. [Dynamic Evidence Pool Construction; Experimental Setups] The comparison in Table 2 is confounded. Equation (1) instructs GPT-3.5-turbo to curate retrieved passages, and Equations (3) and (6) use the same auxiliary LLM for entity/triple extraction and placeholder filling. The baselines (No Retrieval, Vanilla RAG, Adaptive-RAG, DRAGIN, Probing-RAG) use only the small generation model (Gemma-2b or Mistral-7b). Thus the reported gains may be due to GPT-3.5-turbo's query rewriting and evidence organization ability rather than to the dynamic evidence pool or R-Feedback Model. A controlled ablation replacing GPT-3.5-turbo with the same base model, or giving the same auxiliary LLM to the comparison methods, is required to support the central claim.
  2. [Implementation Details] The paper states: 'we conducted experiments with a single run.' No error bars, confidence intervals, or significance tests are reported. Several differences in Table 2 are small (e.g., Gemma-2b StrategyQA ACC: RFM-RAG 63.2 vs. DRAGIN 62.4; Mistral-7b StrategyQA ACC: RFM-RAG 72.6 vs. DRAGIN 70.3). With a single run, these differences are within sampling noise, so the claim that RFM-RAG 'outperforms previous methods' is not established.
  3. [Training R-Feedback Model; Experimental Setups] The R-Feedback Model is trained on synthetic (q,E) pairs derived from 2WikiMultihopQA gold evidence chains and then applied, without retraining or calibration, to 2WikiMultihopQA, NaturalQA, and StrategyQA. This creates an in-distribution advantage on 2WikiMultihopQA, and no evidence is given that the sufficiency classifier transfers to the other two datasets. The paper also does not report whether the 500 2Wiki test questions are disjoint from the questions used to build the R-Feedback training samples. Without an OOD evaluation of termination decisions or a training/test overlap check, the claimed accuracy and latency benefits in Table 3 are not supported.
  4. [Experimental Results, Main Results] The headline improvements are averages over datasets and mask a regression. On Gemma-2b, 2WikiMultihopQA ACC falls from 43.0 (No Retrieval) to 37.6 (RFM-RAG), and from 38.4 (Vanilla RAG) to 37.6. The text's claim of ACC improvement by 3.5 and 4.5 percentage points on Gemma-2b is an average across three datasets. A method that improves average accuracy via large gains on two datasets while degrading on a third does not support the unqualified 'improves overall system accuracy' claim without per-dataset significance testing.
  5. [Table 3] The fixed-iteration ablation table is internally difficult to interpret and appears to contradict the text. For Mistral-7b on NaturalQA, the RFM row lists EM 33.4 with R-Step 2.34, while the wo-RFM row lists 35.8; if the latter is the EM at the maximum 3 retrieval steps, the fixed baseline is 2.4 points higher, not lower as the text claims. The column labels and arrow notation should be clarified, and the claim that 'unnecessary retrieval beyond knowledge saturation leads to a reduction in accuracy' needs to be supported by the actual numbers.
minor comments (4)
  1. [Methodology, Eq. (4)/(7)] The entity-coverage feature S_fk divides occurrence count by evidence-pool length, but the units of L_E are not defined and the threshold θ is not specified. Equation (7) also reuses |E| for a different quantity than Equation (2), which defines E as the evidence pool.
  2. [Appendices] Appendices A, B, and C are referenced but not included in the arXiv text. Hyperparameters for the R-Feedback Model, prompt templates, corpus construction details, and the promised R-Feedback training dataset release are therefore unavailable to readers.
  3. [Table 3] The table formatting needs correction: the headers 'R-Step ∆ EM' and 'R-Step ∆ ACC' are ambiguous, and the arrow notation and numeric values should be separated into explicit columns so that the reader can verify the claimed deltas.
  4. [Throughout] There are minor naming inconsistencies and typos: the abstract and intro use different expanded forms of RFM-RAG ('Retrieval Feedback and Memory Retrieval Augmented Generation' vs. 'Retrieval-Feedback Augmented Memory Enhanced...'), and the datasets section has '2WikimultihopQA' instead of '2WikiMultihopQA'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the benchmark gains are empirical results, not derivations from the method's own definitions.

full rationale

The paper's central claim is that RFM-RAG improves QA accuracy on three public benchmarks. This is an empirical claim, not a first-principles derivation, and the reported quantities (EM/ACC) are not defined in terms of the method's own parameters. The R-Feedback Model is a trained feed-forward classifier; its training labels are synthetic pairs built from 2WikiMultihopQA gold evidence chains, and the same dataset is used among the evaluation benchmarks. That raises a legitimate in-distribution/generalization concern, but it is not circularity: the final answer EM/ACC is not the classifier's training target, so the benchmark numbers are not forced by construction. Similarly, the use of GPT-3.5-turbo for evidence curation/query formulation while baselines use only the small QA model is a serious control/confound issue, but it is an experimental-fairness problem, not a circular reduction; no parameter is fitted to the benchmark answers and no result is equivalent to its input by definition. The paper contains no load-bearing self-citation chain and no imported uniqueness theorem. The single-run evaluation is a reliability limitation, not circularity, and the R-Feedback Model's out-of-distribution behavior on NaturalQA and StrategyQA is a generalization risk rather than a self-referential step. Therefore the circularity score is 0.

Assumptions & free parameters 5 free parameters · 6 assumptions · 3 invented entities

The method rests on several hand-chosen or fitted quantities (coverage threshold, retrieval cap, R-Feedback weights, cross-encoder, few-shot counts) and on domain assumptions about sufficiency estimation, transferability, BM25 adequacy, and answer extraction. None of these are independently validated outside the paper's own pipeline.

free parameters (5)
  • Coverage threshold theta = not specified
    Entity coverage feature Sfk (Eq. 4) is compared to preset threshold theta to decide whether an entity is missing (Eq. 5). The value is chosen by hand and never reported.
  • Maximum retrieval rounds = 3
    Chosen because 'diminishing accuracy gains and significant latency increases beyond three retrieval rounds' (Experimental Setups). Affects all comparisons.
  • R-Feedback Model weights = trained on 10,000 synthetic pairs from 2WikiMultihopQA
    The termination decision is produced by a trained feed-forward network with two hidden layers. Hyperparameters are relegated to the missing Appendix A.
  • Cross-encoder weights = not specified
    Semantic relevance feature Gf comes from a cross-encoder, whose identity and training are not disclosed.
  • Few-shot exemplar counts = 4-shot (2Wiki, NQ), 6-shot (StrategyQA)
    Evaluation setup choices that could affect absolute scores, though shared across methods.
assumptions (6)
  • domain assumption Knowledge sufficiency is representable by entity coverage and cross-encoder relevance
    Section 'R-Feedback Model' defines Sf and Gf as the only inputs to the termination classifier.
  • domain assumption The R-Feedback model trained on synthetic 2WikiMultihopQA pairs transfers to NaturalQA and StrategyQA without retraining
    Used across all benchmarks in Experimental Results; no OOD validation of the classifier is reported.
  • domain assumption LLMs can answer accurately in a single pass if given sufficient knowledge
    Stated in Methodology; motivates early termination; not tested directly.
  • domain assumption BM25 sparse retrieval is sufficient for fair comparison
    Implemented via ElasticSearch 'for all methods to ensure fairness.'
  • domain assumption Regex-based answer extraction maps free-form outputs to labels faithfully
    Experimental Setups.
  • standard math Standard probability and cross-entropy machinery
    Feed-forward network with cross-entropy loss.
invented entities (3)
  • R-Feedback Model
    purpose: Classifies whether the evidence pool is sufficient to answer the question, deciding when to stop retrieval.
    No direct evaluation of the classifier is given; its usefulness is only inferred from downstream QA numbers, and its training data comes from the same benchmark family used for one of the test sets.
  • Dynamic evidence pool
    purpose: Persistent, deduplicated store of retrieved evidence across rounds.
    It is an internal memory structure; no direct measurement of its quality is provided.
  • Relational triple placeholders <X>
    purpose: Represents missing knowledge gaps in query generation.
    A prompting construct; no independent validation that placeholders identify true gaps.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Retrieval Feedback Memory Enhancement Large Model Retrieval Generation Method." pith.science (2026). https://pith.science/paper/3XCQZ7JJ

@misc{pith2026250817862,
  author       = {Pith},
  title        = {Pith review of: Retrieval Feedback Memory Enhancement Large Model Retrieval Generation Method},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3XCQZ7JJ}},
  note         = {Machine review of arXiv:2508.17862}
}
read the original abstract

Large Language Models (LLMs) have shown remarkable capabilities across diverse tasks, yet they face inherent limitations such as constrained parametric knowledge and high retraining costs. Retrieval-Augmented Generation (RAG) augments the generation process by retrieving externally stored knowledge absent from the models internal parameters. However, RAG methods face challenges such as information loss and redundant retrievals during multi-round queries, accompanying the difficulties in precisely characterizing knowledge gaps for complex tasks. To address these problems, we propose Retrieval Feedback and Memory Retrieval Augmented Generation(RFM-RAG), which transforms the stateless retrieval of previous methods into stateful continuous knowledge management by constructing a dynamic evidence pool. Specifically, our method generates refined queries describing the models knowledge gaps using relational triples from questions and evidence from the dynamic evidence pool; Retrieves critical external knowledge to iteratively update this evidence pool; Employs a R-Feedback Model to evaluate evidence completeness until convergence. Compared to traditional RAG methods, our approach enables persistent storage of retrieved passages and effectively distills key information from passages to construct clearly new queries. Experiments on three public QA benchmarks demonstrate that RFM-RAG outperforms previous methods and improves overall system accuracy.

Figures

Figures reproduced from arXiv: 2508.17862 by the authors.

Figure 1
Figure 1. RFM-RAG employs an LLM to distill knowl [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. RFM-RAG dynamically constructs an evidence pool by processing retrieved results and formulates targeted queries [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Prompt1 for Organizing Passages Using LLMs. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Prompt3 for Extracting Placeholder-Represented [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: EM and ACC scores for QA without retrieval and RFM-RAG based on Gemma-2b, Mistral-7b, and Gemma3-4b [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. PATH-Bench: Path-Dependent Evaluation of Lifelong Agents

    cs.AI 2026-08 conditional novelty 6.0 of 10

    A benchmark that controls task order to measure transfer, retention, and path-dependence in lifelong LLM agents, applied to code and tool-use tasks.

Reference graph

Works this paper leans on

36 extracted references · 11 canonical work pages · cited by 1 Pith paper

  1. [1]

    Jiang, A

    Albert Q. Jiang, A. M., Alexandre Sablayrolles; et al. 2023. Mistral 7B. arXiv preprint arXiv:2310.06825

    arXiv 2023

  2. [2]

    Asai, A.; Wu, Z.; Wang, Y.; Sil, A.; and Hajishirzi, H. 2024. Self-rag: Learning to retrieve, generate, and critique through self-reflection

    work page 2024

  3. [3]

    Baek, I.; Chang, H.; Kim, B.; Lee, J.; and Lee, H. 2024. Probing-rag: Self-probing to guide language models in selective document retrieval. arXiv preprint arXiv:2410.13339

    work page Pith review arXiv 2024

  4. [4]

    Banerjee, S.; and Lavie, A. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, 65--72

    2005

  5. [5]

    B.; Lespiau, J.-B.; Damoc, B.; Clark, A.; et al

    Borgeaud, S.; Mensch, A.; Hoffmann, J.; Cai, T.; Rutherford, E.; Millican, K.; Van Den Driessche, G. B.; Lespiau, J.-B.; Damoc, B.; Clark, A.; et al. 2022. Improving language models by retrieving from trillions of tokens. In International conference on machine learning, 2206--2240. PMLR

    work page 2022

  6. [6]

    D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al

    Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J. D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33: 1877--1901

    2020

  7. [7]

    W.; Sutton, C.; Gehrmann, S.; et al

    Chowdhery, A.; Narang, S.; Devlin, J.; Bosma, M.; Mishra, G.; Roberts, A.; Barham, P.; Chung, H. W.; Sutton, C.; Gehrmann, S.; et al. 2023. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240): 1--113

    work page 2023

  8. [8]

    E.; and Schockaert, S

    Es, S.; James, J.; Anke, L. E.; and Schockaert, S. 2024. Ragas: Automated evaluation of retrieval augmented generation. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, 150--158

    work page 2024

Show all 36 references
  1. [9]

    R.; Kry \'s ci \'n ski, W.; McCann, B.; Xiong, C.; Socher, R.; and Radev, D

    Fabbri, A. R.; Kry \'s ci \'n ski, W.; McCann, B.; Xiong, C.; Socher, R.; and Radev, D. 2021. Summeval: Re-evaluating summarization evaluation. Transactions of the Association for Computational Linguistics, 9: 391--409

    2021

  2. [10]

    Geva, M.; Khashabi, D.; Segal, E.; Khot, T.; Roth, D.; and Berant, J. 2021. Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies. Transactions of the Association for Computational Linguistics, 9: 346--361

    2021

  3. [11]

    Guu, K.; Lee, K.; Tung, Z.; Pasupat, P.; and Chang, M. 2020. Retrieval augmented language model pre-training. In International conference on machine learning, 3929--3938. PMLR

    2020

  4. [12]

    D.; Sugawara, S.; and Aizawa, A

    Ho, X.; Nguyen, A.-K. D.; Sugawara, S.; and Aizawa, A. 2020. Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps. arXiv preprint arXiv:2011.01060

    2020 arXiv

  5. [13]

    Izacard, G.; and Grave, E. 2020. Leveraging passage retrieval with generative models for open domain question answering. arXiv preprint arXiv:2007.01282

    2020 arXiv

  6. [14]

    J.; and Park, J

    Jeong, S.; Baek, J.; Cho, S.; Hwang, S. J.; and Park, J. C. 2024. Adaptive-rag: Learning to adapt retrieval-augmented large language models through question complexity. arXiv preprint arXiv:2403.14403

    2024 arXiv

  7. [15]

    F.; Gao, L.; Sun, Z.; Liu, Q.; Dwivedi-Yu, J.; Yang, Y.; Callan, J.; and Neubig, G

    Jiang, Z.; Xu, F. F.; Gao, L.; Sun, Z.; Liu, Q.; Dwivedi-Yu, J.; Yang, Y.; Callan, J.; and Neubig, G. 2023. Active retrieval augmented generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 7969--7992

    2023

  8. [16]

    Khandelwal, U.; Levy, O.; Jurafsky, D.; Zettlemoyer, L.; and Lewis, M. 2019. Generalization through memorization: Nearest neighbor language models. arXiv preprint arXiv:1911.00172

    2019 arXiv

  9. [17]

    Kwiatkowski, T.; Palomaki, J.; Redfield, O.; Collins, M.; Parikh, A.; Alberti, C.; Epstein, D.; Polosukhin, I.; Devlin, J.; Lee, K.; et al. 2019. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7: 453--466

    2019

  10. [18]

    u ttler, H.; Lewis, M.; Yih, W.-t.; Rockt \

    Lewis, P.; Perez, E.; Piktus, A.; Petroni, F.; Karpukhin, V.; Goyal, N.; K \"u ttler, H.; Lewis, M.; Yih, W.-t.; Rockt \"a schel, T.; et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems, 33: 9459--9474

    2020

  11. [19]

    Lin, C.-Y. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, 74--81

    2004

  12. [20]

    Maynez, J.; Narayan, S.; Bohnet, B.; and McDonald, R. 2020. On faithfulness and factuality in abstractive summarization. arXiv preprint arXiv:2005.00661

    2020 arXiv

  13. [21]

    Papineni, K.; Roukos, S.; Ward, T.; and Zhu, W.-J. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, 311--318

    2002

  14. [22]

    Paranjape, B.; Lundberg, S.; Singh, S.; Hajishirzi, H.; Zettlemoyer, L.; and Ribeiro, M. T. 2023. Art: Automatic multi-step reasoning and tool-use for large language models. arXiv preprint arXiv:2303.09014

    2023 arXiv

  15. [23]

    Radford, A.; Narasimhan, K.; Salimans, T.; Sutskever, I.; et al. 2018. Improving language understanding by generative pre-training

    2018

  16. [24]

    Ram, O.; Levine, Y.; Dalmedigos, I.; Muhlgay, D.; Shashua, A.; Leyton-Brown, K.; and Shoham, Y. 2023. In-context retrieval-augmented language models. Transactions of the Association for Computational Linguistics, 11: 1316--1331

    2023

  17. [25]

    Robertson, S.; Zaragoza, H.; et al. 2009. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends in Information Retrieval , 3(4): 333--389

    2009

  18. [26]

    E.; and Jones, K

    Robertson, S. E.; and Jones, K. S. 1976. Relevance weighting of search terms. Journal of the American Society for Information science, 27(3): 129--146

    1976

  19. [27]

    Su, W.; Tang, Y.; Ai, Q.; Wu, Z.; and Liu, Y. 2024. DRAGIN: dynamic retrieval augmented generation based on the information needs of large language models. arXiv preprint arXiv:2403.10081

    2024 arXiv

  20. [28]

    Team, G.; Kamath, A.; Ferret, J.; Pathak; et al. 2025. Gemma 3 technical report. arXiv preprint arXiv:2503.19786

    2025 arXiv

  21. [29]

    Team, G.; Mesnard, T.; Hardin, C.; Dadashi, R.; Bhupatiraju, S.; Pathak; et al. 2024. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295

    2024 arXiv

  22. [30]

    Thorne, J.; Vlachos, A.; Christodoulopoulos, C.; and Mittal, A. 2018. FEVER: a large-scale dataset for fact extraction and VERification. arXiv preprint arXiv:1803.05355

    2018 arXiv

  23. [31]

    Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.-A.; Lacroix, T.; Rozi \`e re, B.; Goyal, N.; Hambro, E.; Azhar, F.; et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  24. [32]

    Trivedi, H.; Balasubramanian, N.; Khot, T.; and Sabharwal, A. 2022. Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. arXiv preprint arXiv:2212.10509

    2022 arXiv

  25. [33]

    V.; Zhou, D.; et al

    Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Xia, F.; Chi, E.; Le, Q. V.; Zhou, D.; et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35: 24824--24837

    2022

  26. [34]

    Zhou, C.; Neubig, G.; Gu, J.; Diab, M.; Guzman, P.; Zettlemoyer, L.; and Ghazvininejad, M. 2020. Detecting hallucinated content in conditional neural sequence generation. arXiv preprint arXiv:2011.02593

    2020 arXiv

  27. [35]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...

  28. [36]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.