Pith. sign in

REVIEW 5 major objections 3 minor 2 cited by

Ask, Retrieve, Summarize: A Modular Pipeline for Scientific Literature Summarization

T0 review · 5 major / 3 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read XSum's question-and-editor design tops SurveySum summarization baselines.

desk verdict A tidy, modular RAG pipeline with a plausible design, but the headline gains rest on a single unvalidated small-model evaluator, so the evidence is not yet as strong as the abstract claims. read the letter →

arxiv 2505.16349 v1 pith:SCAJ5Y35 submitted 2025-05-22 cs.CL

Pierre Achkar , Tim Gollub , Martin Potthast This is my paper · ORCID
classification cs.CL
keywords scientificliteraturesummarizationmulti-documentretrieval-augmentedgenerationquestioneditormodulecitation-awareSurveySumLLM-basedevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the quality of a scientific multi-document summary depends not only on retrieving the right passages but on asking the right retrieval questions in the first place. It presents XSum, a modular retrieval-augmented generation pipeline whose question-generation module derives five questions from each paper's title and abstract, and whose editor module later composes the retrieved question-answer pairs into a single citation-rich survey section. On the SurveySum benchmark, XSum is reported to outperform the two published SurveySum pipelines on every measured metric, with the largest gains on Ref-F1 (0.76 vs. 0.72/0.64), G-Eval (4.2 vs. 4.0/3.1), and CheckEval (0.97 vs. 0.76/0.61). The authors care because dynamic, content-derived queries may be a broadly applicable fix for retrieval-based summarization, not just for this dataset.

What carries the argument

The pipeline's central machinery is the pair of new modules around a standard RAG loop. A question-generation module prompts an LLM with each paper's title and abstract to produce k=5 broad questions, which are embedded with SPECTER2 and used to retrieve the top 100 chunks from a FAISS index; a ColBERT2 late-interaction reranker narrows this to the 20 most relevant chunks, and a second LLM answers each question from those chunks, citing them and abstaining when evidence is insufficient. An editor module then ingests all question-answer pairs and writes one flowing summary that must keep the citations from the answers. The assumed mechanism is that questions derived from document metadata create queries semantically closer to the content than a static section title, so the retrieved chunks are more on-target, and the two-pass structure separates evidence collection from composition.

What would settle it

Run a small human evaluation on a sample of SurveySum sections: ask annotators to rank XSum and the two baseline summaries on coverage, coherence, and citation correctness, and compare their ranking with the CheckEval gap of 0.97 for XSum versus 0.76 for the best baseline. Alternatively, recompute all three pipelines' G-Eval and CheckEval scores with a different judge model (for instance a larger or instruction-tuned evaluator) and check whether XSum still leads by comparable margins.

Watch

Extended reading notes

Core claim

On its own terms, the paper's claim is that replacing static section-title queries with LLM-generated questions taken from the papers' titles and abstracts makes retrieval more relevant, and that a second LLM 'editor' pass that fuses the resulting question-answer pairs produces summaries that are more coherent and more faithful to the cited literature. The evidence is a head-to-head comparison on SurveySum: XSum reaches ROUGE-1 0.51, ROUGE-2 0.10, ROUGE-L 0.24, BERTScore 0.62, Ref-F1 0.76, G-Eval 4.2, and CheckEval 0.97, versus best-baseline values of 0.49, 0.10, 0.23, 0.59, 0.72, 4.0, and 0.76 respectively. The authors also observe the trade-off that their summaries are more verbose and less selective than human-written survey sections, which they attribute to the difficulty of matching expert-level prioritization.

Load-bearing premise

The load-bearing assumption is that G-Eval and CheckEval scores produced by Phi-3-small-8k-instruct, with no human calibration on SurveySum and with all baseline scores recomputed by the authors, are faithful measures of summary quality; if that evaluator silently prefers XSum's verbosity or citation-dense style, the headline advantage could shrink or disappear.

Editorial extensions

If this is right

  • If the comparison holds, dynamic question generation is a cheap drop-in replacement for static section-title queries in RAG summarizers, since it only needs titles and abstracts.
  • The editor module's citation-preservation rule suggests that citation accuracy can be engineered by carrying citations from retrieved answers into the final text rather than asking the generator to cite from memory.
  • The design is modular: question templates and editor instructions can be swapped for domain-specific or audience-specific summarization without changing the retrieval core.
  • The reported results imply that the main headroom for RAG-based scientific summarization may now be stylistic — the authors' own examples show verbosity and reduced selectivity — rather than factual grounding.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next test the paper does not run is an ablation: rerun XSum with static title queries, or without the editor, to isolate how much of the CheckEval gain is due to each module (the authors list this as future work).
  • The LLM-judge scoring may partly reward XSum's verbosity and citation density; a human preference study on SurveySum would determine whether the headline gap reflects usefulness or style, and the authors note the absence of qualitative analysis.
  • The same question-generation idea could transfer to other multi-document tasks such as evidence synthesis or literature review drafting, where retrieval queries are typically underspecified, and to corpora that have abstracts but not full-text indexes.
  • Because XSum assumes the input papers are already chosen, combining it with a topical paper-retrieval step would turn it from a summarizer into an end-to-end literature-review system (a limitation the paper acknowledges).
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 3 minor

Summary. The paper introduces XSum, a modular RAG-based pipeline for multi-document summarization of scientific literature. XSum adds two components to a standard retrieval-augmented generator: a question-generation module that creates five queries per paper from its title and abstract, and an editor module that composes retrieved question-answer pairs into a final summary while enforcing citation inclusion. The pipeline is evaluated on the SurveySum dataset against two baseline pipelines (Pipeline_1 and Pipeline_2), with ROUGE, BERTScore, Ref-F1, G-Eval, and CheckEval scores. All metrics, including baseline scores, were recomputed by the authors using Phi-3-small-8k-instruct as the evaluator. The paper reports that XSum outperforms both baselines on every metric, with the largest gain in CheckEval (0.97 vs. 0.76), and concludes that the question-generation and editor modules drive the improvement.

Significance. If the reported improvements are reliable, XSum would be a useful, transparent modular framework for scientific multi-document summarization. The paper contributes an external benchmark evaluation with released code, a consistent evaluation protocol in which all baselines are rescored by the same evaluator, and detailed worked examples with scores. The design choices—dynamic query generation and a citation-enforcing editor—are clearly motivated and easy to adapt. However, the current evidence base is narrow: the evaluation relies entirely on LLM-based metrics from a single small model, with no human judgments, no variance or significance estimates, and no ablations isolating the two proposed modules. The significance for the broader literature is therefore conditional on a stronger validation of the measurement instruments and the causal attribution.

major comments (5)
  1. [Section 4.3, Section 4.4, Table 1] All conclusions rest on a single run of each pipeline with no confidence intervals, significance tests, or human evaluation. The differences between XSum and Pipeline_2 are small on most metrics (ROUGE-1 +0.02, ROUGE-L +0.01, BERTScore +0.03, Ref-F1 +0.04, G-Eval +0.2); only the CheckEval gap (+0.21) is sizable. Section 6 explicitly concedes the lack of qualitative analysis, and Section 5 lists an ablation as future work. The authors should report per-section score distributions or bootstrap intervals, and ideally a human evaluation on a sample, or temper the abstract's claim of 'considerable improvements' to the specific metrics where the difference is robust.
  2. [Appendix A.2, Example 2] The low-scoring XSum output is visibly off-topic: it discusses LIME/SHAP and model-agnostic explanation in general, while the gold section is about GPT-2-based counterfactual generation. The same summary receives G-Eval 4.5/5 and CheckEval 0.11. This demonstrates that G-Eval does not penalize topical drift on SurveySum, and that a single CheckEval score can be an extreme outlier. Given the reported XSum mean CheckEval of 0.97, the authors should show the full score distribution and the relationship between G-Eval and CheckEval to establish that the aggregate is meaningful rather than an artifact of evaluator instability.
  3. [Section 3.5, editor prompt] The instruction 'Always include the citations (e.g., [BIBREF14], [BIBREF16]) mentioned in the answers in the final section' obliges the editor to transfer every citation from the retrieved answers into the final summary. This mechanically increases Ref-F1, which is computed against the gold citation list, and can inflate checklist-based coverage scores, creating a confound between citation density and summary relevance. Since Section 5 states that no ablation was performed, the contribution of the editor module—and of this specific instruction—to the Table 1 scores is not established.
  4. [Section 4.4] The paper does not specify whether the Pipeline_1 and Pipeline_2 summaries were regenerated with the same generator as XSum (gpt4o-mini) or were taken from the original SurveySum runs. Section 2 describes the original pipelines as using a different LLM (e.g., GPT-4 in Pipeline 1). If the baselines were not regenerated with the same generator, the comparison is confounded: score differences may reflect the generator model rather than the proposed retrieval and editing modules. The authors should state the exact generator used for each baseline and, ideally, rerun the baselines with the same generator used for XSum.
  5. [Section 3.2, Section 4.5, Section 5] The discussion in Section 4.5 attributes XSum's strong performance to the question-generation and editor modules, but the paper provides no ablation that isolates these components. Section 5 explicitly lists an ablation study as future work. Without such an experiment, the causal claim in Section 4.5 is not supported by the data presented; the current evidence only shows that the full XSum pipeline scores higher than the two baselines, not which module is responsible.
minor comments (3)
  1. [Section 3.2] The terminology for HyDE and HyQE is inconsistent: the text first writes 'HyQE (Hypothetical Document Embeddings)' and two sentences later 'HyQE (Hypothetical Query Embeddings)'. The methods are HyDE (Hypothetical Document Embeddings) and HyQE (Hypothetical Query Embeddings); please correct the labels.
  2. [Section 3.3] The chunk size of 150 tokens with 20-token overlap is described as determined 'by experimentation' without reporting the alternatives considered or the sensitivity of the final results to this choice. A short table or a reference to an appendix would improve reproducibility.
  3. [Figure 3] Figure 3 does not depict the ColBERT2 reranking step described in Section 3.4, even though the text presents reranking as an integral part of the retrieval process. Adding the reranker to the figure would make the diagram consistent with the methodology.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the pipeline is an empirical system evaluated on an external benchmark; evaluator-validity concerns are measurement issues, not derivation-from-input circularity.

full rationale

XSum makes no analytic derivation that reduces to its inputs. The central claim is an empirical comparison on the external SurveySum dataset against two published baseline pipelines, using published metrics (ROUGE, BERTScore, Ref-F1, G-Eval, CheckEval). The pipeline components (question generation, retrieval, reranking, editing) are neither fitted to the reference summaries nor defined in terms of the evaluation scores. The references to SurveySum, G-Eval, CheckEval, and ColBERT are external prior work by other authors, not self-citations, and none of them is invoked to forbid alternatives or to justify a uniqueness claim. The authors' decision to recompute baseline G-Eval and CheckEval scores with their own Phi-3-small-8k-instruct evaluator is a potential reliability and fairness concern, and the Appendix A.2 example showing low CheckEval for an off-topic summary is relevant to whether the reported metric gaps are meaningful; however, this is a question of measurement validity and reproducibility, not circularity. The editor prompt's instruction to include citations from the answers can affect Ref-F1, but citation inclusion is part of the system design and is compared against the same external reference citations for all pipelines; it is not a fitted parameter renamed as a prediction. No circular step can be exhibited from the paper's own equations or construction, so the appropriate score is 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central evaluation relies on several unvalidated domain assumptions: SurveySum is a trustworthy benchmark, the Phi-3-based G-Eval/CheckEval scores correlate with quality, the retrieval stack returns sufficient context, and citation markers in retrieved chunks equal correct attribution. The system's own hyperparameters (k=5 questions, 150/20 chunking, top-100/top-20 retrieval, sampling parameters) are hand-selected without ablations, and chunking was tuned on the same benchmark. No new entities are introduced; all components are existing models or datasets.

free parameters (4)
  • Number of generated questions per paper (k) = 5
    Chosen in Section 3.2; no sensitivity analysis; directly controls how many retrieval queries and Q&A pairs feed the editor.
  • Chunk size and overlap = 150 tokens, 20-token overlap
    Selected by experimentation per Section 3.3; affects retrieval granularity and final summary; tuning on the evaluation benchmark without a described validation split.
  • Retrieval counts (initial and reranked) = top 100, then top 20
    Set in Section 3.4; no ablation or analysis of how these thresholds affect metric outcomes.
  • Generation sampling parameters = temperature 0.3, top-p 0.95
    Set in Section 4.3 for controlled outputs; no exploration of sensitivity.
assumptions (5)
  • domain assumption LLM-based metrics (G-Eval, CheckEval) computed by Phi-3-small-8k-instruct are valid proxies for summary quality.
    Sections 4.2 and 4.4 use these as headline evidence; no human validation or calibration on SurveySum is provided.
  • domain assumption SurveySum ground-truth survey sections are reliable references for MDS evaluation.
    Section 4.1 treats SurveySum as the benchmark; no analysis of annotation quality or ambiguity in reference sections is given.
  • domain assumption The retrieval stack (SPECTER2 embeddings, FAISS, ColBERT2 reranking, top-100 to top-20) retrieves sufficient relevant content for each question.
    Section 3.4 assumes this without retrieval recall or oracle experiments; if relevant chunks are missed, summaries lose content.
  • ad hoc to paper Generating five questions from titles/abstracts improves retrieval over using the section title as a static query.
    This is the core design hypothesis (Sections 3.2 and 4.5) but no ablation isolates question generation from the editor module.
  • domain assumption Citation markers from retrieved chunks correspond to genuine attribution in the summary.
    Ref-F1 assumes citation overlap is meaningful; the editor prompt explicitly requires including all citations from answers, which can reward copying citation labels.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Ask, Retrieve, Summarize: A Modular Pipeline for Scientific Literature Summarization." pith.science (2026). https://pith.science/paper/SCAJ5Y35

@misc{pith2026250516349,
  author       = {Pith},
  title        = {Pith review of: Ask, Retrieve, Summarize: A Modular Pipeline for Scientific Literature Summarization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SCAJ5Y35}},
  note         = {Machine review of arXiv:2505.16349}
}
read the original abstract

The exponential growth of scientific publications has made it increasingly difficult for researchers to stay updated and synthesize knowledge effectively. This paper presents XSum, a modular pipeline for multi-document summarization (MDS) in the scientific domain using Retrieval-Augmented Generation (RAG). The pipeline includes two core components: a question-generation module and an editor module. The question-generation module dynamically generates questions adapted to the input papers, ensuring the retrieval of relevant and accurate information. The editor module synthesizes the retrieved content into coherent and well-structured summaries that adhere to academic standards for proper citation. Evaluated on the SurveySum dataset, XSum demonstrates strong performance, achieving considerable improvements in metrics such as CheckEval, G-Eval and Ref-F1 compared to existing approaches. This work provides a transparent, adaptable framework for scientific summarization with potential applications in a wide range of domains. Code available at https://github.com/webis-de/scolia25-xsum

Figures

Figures reproduced from arXiv: 2505.16349 by the authors.

Figure 1
Figure 1. Overview of Pipeline 1: The system segments full-text papers into overlapping chunks, ranks them using monoT5-3B based on the section title, and selects the top-ranked chunks for LLM-based summarization. Reference Papers FAISS Document Pre-Processing RAG Final Summary Full Text Section Title Chunks Relevant Chunks Reranking [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Overview of Pipeline 2: Instead of ranking with a neural model, this pipeline encodes chunks as dense embeddings using SPECTER2, stores them in a FAISS vector database, retrieves them based on section title queries, and applies reranking before LLM-based summarization. Beyond SurveySum, several other datasets have been developed for MDS, particularly in the biomedical domain. Datasets such as Cochrane-auto and MS2 f… view at source ↗
Figure 3
Figure 3. Overview of the XSum Pipeline. The pipeline processes reference papers into summaries through modular steps. Document Pre-Processing segments papers into chunks, encodes them as embeddings, and stores them in a FAISS database. Question Generation uses an LLM to generate questions from titles and abstracts. In Question Answering, a RAG framework retrieves relevant chunks and generates answers with an LLM. Finally, th… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Reproducible, Scalable Pipeline for Synthesizing Autoregressive Model Literature

    cs.IR 2025-08 conditional novelty 6.0 of 10

    A scalable literature-synthesis pipeline that retrieves, filters, extracts, summarizes, and converts AR-model papers into runnable training scripts, with F1 > 0.85 extraction and three reproduction case studies.

  2. AI4Research: A Survey of Artificial Intelligence for Scientific Research

    cs.CL 2025-07 conditional novelty 4.0 of 10

    A survey that organizes AI-for-research work into five tasks, comprehension, survey, discovery, writing, and peer review, and compiles associated tools and benchmarks.

Reference graph

Works this paper leans on

24 extracted references · 8 canonical work pages · cited by 2 Pith papers

  1. [1]

    Krenn, L

    M. Krenn, L. Buffoni, B. C. Coutinho, S. Eppel, J. G. Foster, A. Gritsevskiy, H. Lee, Y. Lu, J. P. Moutinho, N. Sanjabi, R. Sonthalia, N. M. Tran, F. Valente, Y. Xie, R. Yu, M. Kopp, Forecasting the future of artificial intelligence with machine learning-based link prediction in an exponentially growing knowledge network, Nat. Mac. Intell. 5 (2023) 1326–1...

    doi:10.1038/s42256-023-00735-0 2023

  2. [2]

    Nanba, M

    H. Nanba, M. Okumura, Towards multi-paper summarization using reference information, in: T. Dean (Ed.), Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, IJCAI 99, Stockholm, Sweden, July 31 - August 6, 1999. 2 Volumes, 1450 pages, Morgan Kaufmann, 1999, pp. 926–931. URL: http://ijcai.org/Proceedings/99-2/Papers/038.pdf

    work page 1999

  3. [3]

    Zhang, P

    H. Zhang, P. S. Yu, J. Zhang, A systematic survey of text summarization: From statistical methods to large language models, CoRR abs/2406.11289 (2024). URL: https://doi.org/10.48550/arXiv.2406. 11289. doi:10.48550/ARXIV.2406.11289. arXiv:2406.11289

    arXiv doi:10.48550/arxiv.2406.11289 2024

  4. [4]

    L. C. Fernandes, G. B. Guedes, T. S. Laitz, T. S. Almeida, R. F. Nogueira, R. A. Lotufo, J. Pereira, Sur- veysum: A dataset for summarizing multiple scientific articles into a survey section, CoRR abs/2408.16444 (2024). URL: https://doi.org/10.48550/arXiv.2408.16444. doi: 10.48550/ARXIV. 2408.16444. arXiv:2408.16444

    work page Pith review arXiv doi:10.48550/arxiv.2408.16444 2024

  5. [5]

    Y. Lee, J. Kim, J. Kim, H. Cho, P. Kang, Checkeval: Robust evaluation framework using large language model via checklist, CoRR abs/2403.18771 (2024). URL: https://doi.org/10.48550/arXiv. 2403.18771. doi:10.48550/ARXIV.2403.18771. arXiv:2403.18771

    work page doi:10.48550/arxiv.2403.18771 2024

  6. [6]

    Y. Liu, D. Iter, Y. Xu, S. Wang, R. Xu, C. Zhu, G-eval: NLG evaluation using GPT-4 with better human alignment, CoRR abs/2303.16634 (2023). URL: https://doi.org/10.48550/arXiv.2303.16634. doi:10.48550/ARXIV.2303.16634. arXiv:2303.16634

    arXiv doi:10.48550/arxiv.2303.16634 2023

  7. [7]

    Agarwal, R

    N. Agarwal, R. S. Reddy, K. Gvr, C. P. Rosé, SciSumm: A multi-document summarization system for scientific articles, in: S. Kurohashi (Ed.), Proceedings of the ACL-HLT 2011 System Demonstrations, Association for Computational Linguistics, Portland, Oregon, 2011, pp. 115–120. URL: https: //aclanthology.org/P11-4020/

    work page 2011

  8. [8]

    Fabbri, I

    A. Fabbri, I. Li, T. She, S. Li, D. Radev, Multi-news: A large-scale multi-document summa- rization dataset and abstractive hierarchical model, in: A. Korhonen, D. Traum, L. Màrquez (Eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Lin- guistics, Association for Computational Linguistics, Florence, Italy, 2019, pp. 1074–1...

    doi:10.18653/v1/p19-1102 2019

Show all 24 references
  1. [9]

    Y. Liu, M. Lapata, Hierarchical transformers for multi-document summarization, in: A. Korhonen, D. Traum, L. Màrquez (Eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Florence, Italy, 201...

    2019 doi

  2. [10]

    Shinde, T

    K. Shinde, T. Roy, T. Ghosal, An extractive-abstractive approach for multi-document sum- marization of scientific articles for literature review, in: A. Cohan, G. Feigenblat, D. Freitag, T. Ghosal, D. Herrmannova, P. Knoth, K. Lo, P. Mayr, M. Shmueli-Scheuer, A. de Waard, L. L...

    2022

  3. [11]

    P. Wang, S. Li, K. Pang, L. He, D. Li, J. Tang, T. Wang, Multi-document scientific summariza- tion from a knowledge graph-centric view, in: N. Calzolari, C. Huang, H. Kim, J. Pustejovsky, L. Wanner, K. Choi, P. Ryu, H. Chen, L. Donatelli, H. Ji, S. Kurohashi, P. Paggio, N. Xue...

    2022

  4. [12]

    A. Asai, J. He, R. Shao, W. Shi, A. Singh, J. C. Chang, K. Lo, L. Soldaini, S. Feldman, M. D’Arcy, D. Wadden, M. Latzke, M. Tian, P. Ji, S. Liu, H. Tong, B. Wu, Y. Xiong, L. Zettlemoyer, G. Neubig, D. S. Weld, D. Downey, W. Yih, P. W. Koh, H. Hajishirzi, Openscholar: Synthesiz...

    2024 doi arXiv

  5. [13]

    Bakker, J

    J. Bakker, J. Kamps, Cochrane-auto: An aligned dataset for the simplification of biomedical abstracts, in: M. Shardlow, H. Saggion, F. Alva-Manchego, M. Zampieri, K. North, S. Štajner, R. Stodden (Eds.), Proceedings of the Third Workshop on Text Simplification, Accessibility a...

    2024 doi

  6. [14]

    DeYoung, I

    J. DeYoung, I. Beltagy, M. van Zuylen, B. Kuehl, L. L. Wang, MS2: multi-document summa- rization of medical studies, CoRR abs/2104.06486 (2021). URL: https://arxiv.org/abs/2104.06486. arXiv:2104.06486

    2021 arXiv

  7. [15]

    Y. Lu, Y. Dong, L. Charlin, Multi-xscience: A large-scale dataset for extreme multi-document summarization of scientific articles, in: B. Webber, T. Cohn, Y. He, Y. Liu (Eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, ...

    2020 doi

  8. [16]

    L. Gao, X. Ma, J. Lin, J. Callan, Precise zero-shot dense retrieval without relevance labels, CoRR abs/2212.10496 (2022). URL: https://doi.org/10.48550/arXiv.2212.10496. doi: 10.48550/ARXIV. 2212.10496. arXiv:2212.10496

    2022 doi arXiv

  9. [17]

    W. Zhou, J. Zhang, H. Hasson, A. Singh, W. Li, Hyqe: Ranking contexts with hypothetical query embeddings, CoRR abs/2410.15262 (2024). URL: https://doi.org/10.48550/arXiv.2410.15262. doi:10.48550/ARXIV.2410.15262. arXiv:2410.15262

    2024 doi arXiv

  10. [18]

    Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, Q. Guo, M. Wang, H. Wang, Retrieval-augmented generation for large language models: A survey, CoRR abs/2312.10997 (2023). URL: https://doi.org/10.48550/arXiv.2312.10997. doi: 10.48550/ARXIV. 2312.10997. arXiv:2312.10997

    2023 doi arXiv

  11. [19]

    Santhanam, O

    K. Santhanam, O. Khattab, J. Saad-Falcon, C. Potts, M. Zaharia, Colbertv2: Effective and efficient retrieval via lightweight late interaction, CoRR abs/2112.01488 (2021). URL: https://arxiv.org/abs/ 2112.01488. arXiv:2112.01488

    2021 arXiv

  12. [20]

    Lin, ROUGE: A package for automatic evaluation of summaries, in: Text Summarization Branches Out, Association for Computational Linguistics, Barcelona, Spain, 2004, pp

    C.-Y. Lin, ROUGE: A package for automatic evaluation of summaries, in: Text Summarization Branches Out, Association for Computational Linguistics, Barcelona, Spain, 2004, pp. 74–81. URL: https://aclanthology.org/W04-1013/

    2004

  13. [21]

    Zhang, V

    T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, Y. Artzi, Bertscore: Evaluating text generation with BERT, CoRR abs/1904.09675 (2019). URL: http://arxiv.org/abs/1904.09675. arXiv:1904.09675

    2019 arXiv

  14. [22]

    A. R. Fabbri, W. Kryscinski, B. McCann, C. Xiong, R. Socher, D. R. Radev, Summeval: Re-evaluating summarization evaluation, Trans. Assoc. Comput. Linguistics 9 (2021) 391–409. URL: https: //doi.org/10.1162/tacl_a_00373. doi:10.1162/TACL\_A\_00373

    2021 doi

  15. [23]

    Z. Li, X. Xu, T. Shen, C. Xu, J. Gu, Y. Lai, C. Tao, S. Ma, Leveraging large language models for NLG evaluation: Advances and challenges, in: Y. Al-Onaizan, M. Bansal, Y. Chen (Eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP...

    2024

  16. [24]

    Faysse, H

    M. Faysse, H. Sibille, T. Wu, B. Omrani, G. Viaud, C. Hudelot, P. Colombo, Colpali: Efficient document retrieval with vision language models, CoRR abs/2407.01449 (2024). URL: https://doi. org/10.48550/arXiv.2407.01449. doi:10.48550/ARXIV.2407.01449. arXiv:2407.01449. A. Append...

    2024 doi arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.