Pith. sign in

REVIEW 5 major objections 6 minor 42 references

A New HOPE: Domain-agnostic Automatic Evaluation of Text Chunking

T0 review · 5 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A new metric, HOPE, scores chunking quality by how semantically independent the resulting passages are.

desk verdict A genuinely useful chunking-evaluation metric, but the headline correlations and gains are not backed by the statistical analysis. read the letter →

arxiv 2505.02171 v1 pith:RG6JXY3L submitted 2025-05-04 cs.CL cs.AI

Henrik Br{aa}dland , Morten Goodwin , Per-Arne Andersen , Alexander S. Nossum , Aditya Gupta This is my paper · ORCID
classification cs.CLcs.AI
keywords textchunkingRetrieval-AugmentedGenerationevaluationmetricsemanticindependenceconceptunityinformationpreservationembeddingsimilarityRAGperformance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that document chunking, the step that cuts source texts into passages before retrieval, can be evaluated without human labels by a new metric called HOPE. HOPE measures three things: concept unity inside a passage, semantic independence between passages, and preservation of the original document's information. Across seven domains, HOPE scores correlate with RAG performance indicators, and the semantic independence component shows the strongest association. If correct, chunking decisions no longer have to be tested by end-task performance alone; they can be scored directly and optimized.

What carries the argument

The central object is the HOPE metric, defined as $\mathrm{HOPE} = \frac{1}{3}(\zeta_{\mathrm{inf}} + \zeta_{\mathrm{sem}} + \zeta_{\mathrm{con}})$, built on three formal principles of chunking. Concept unity $\zeta_{\mathrm{con}}$ is computed by having an LLM generate statements from a passage and measuring pairwise cosine similarity of their embeddings; semantic independence $\zeta_{\mathrm{sem}}$ is computed by generating questions from a passage, answering them with and without additional context, and measuring the cosine similarity of the two answer sets; information preservation $\zeta_{\mathrm{inf}}$ is computed by sampling sentence triples from the source document, generating one true and three false claims, and testing whether an LLM can identify the true claim using retrieved passages. The metric requires no annotated data and uses embeddings and LLMs in a pipeline that maps each chunking configuration to a single score.

What would settle it

Take a set of documents, chunk each with a fixed set of chunking methods, compute HOPE scores and RAG performance indicators, then fit a model with document and domain as random effects. If the correlation between semantic independence and factual correctness disappears or drops below significance once document identity is controlled, the central claim is not supported.

Watch

Extended reading notes

Core claim

The paper introduces HOPE, a domain-agnostic, automatic evaluation metric that quantifies chunking quality as the arithmetic mean of three components: concept unity, semantic independence, and information preservation. The central empirical finding is that semantic independence between passages is the dominant driver of RAG performance, with gains of up to 56.2% in factual correctness and 21.1% in answer correctness when comparing extremes of the semantic independence distribution. Conversely, the traditional assumption that passages should each convey a single core concept shows minimal or even negative impact on performance. These results are presented as evidence for the paper's second principle of chunking and as a challenge to the first.

Load-bearing premise

The reported correlations pool 1048 document-chunking pairs without accounting for the fact that many pairs come from the same document, so the apparent predictive power of HOPE could be driven by stable differences between documents rather than by the chunking method itself.

Editorial extensions

If this is right

  • Chunking methods can be compared automatically and without human labels, so expensive end-to-end RAG evaluations are not needed for every chunking decision.
  • Strategies that increase semantic independence, such as decontextualizing passages during chunking, are likely to improve factual correctness and answer correctness in RAG systems.
  • The paper's finding that concept unity correlates negatively with RAG performance suggests that single-concept chunking should not be treated as a default goal.
  • HOPE, or its semantic independence component, could be used as a reward signal for optimizing chunk boundaries in a learning-based or search-based chunking system.
  • Because HOPE correlates with RAG metrics across domains, it provides a common benchmark for comparing future chunking methods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A controlled ablation that rewrites chunks to maximize semantic independence while holding information preservation fixed would directly test whether the observed correlation is causal, a step the paper does not take.
  • The reported 56.2% and 21.1% gains compare extreme readings of the correlated distribution; a randomized intervention study would bound the true causal effect size.
  • If HOPE is used to compare chunking methods, its LLM-generated statements and questions should be audited for diversity, since the paper notes the concept unity component currently spans only about 10% of its possible range.
  • The metric's independence from downstream task evaluation makes it a candidate for online monitoring of chunking quality in deployed RAG systems, though the multiple LLM invocations make per-document scoring computationally heavy.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes HOPE (Holistic Passage Evaluation), an automatic, domain-agnostic metric for evaluating document chunking in retrieval-augmented generation. It formalizes three chunking principles—concept unity, semantic independence, and information preservation—and operationalizes each with LLM-based generation and embedding-based similarity computations, aggregating the three sub-scores as an equal-weight arithmetic mean (Eq. 10). Using 131 documents from seven domains and eight chunking configurations, the authors compute HOPE scores and RAGAS-based RAG performance metrics, then report Pearson correlations (Table 2) and claim that semantic independence is the most important chunking property, with performance gains up to 56.2% in factual correctness and 21.1% in answer correctness. They also report that concept unity shows minimal or negative impact, challenging traditional assumptions.

Significance. If the empirical claims were supported, HOPE would be a practically valuable contribution: it is automatic, annotation-free, designed to be domain-agnostic, and decomposes chunking quality into three interpretable properties. The comparison against a BLEU baseline is a reasonable sanity check, and the use of established RAGAS metrics lends external validity to the outcome measures. The paper also provides a formal definition of chunking and makes falsifiable predictions about the role of semantic independence. However, the current statistical evidence does not support the headline claims: the pooled correlation analysis ignores the nested structure of the data, and the reported performance gains come from a confounded min–max comparison. The central contribution is therefore defensible only after a reanalysis that accounts for document- and domain-level clustering.

major comments (5)
  1. [Section 4.3, Table 2] The Pearson correlations are computed over 1,048 observations formed from 131 documents × 8 chunking configurations, but the observations are nested: each document's source text and question set reappears across its 8 chunkings, and documents share domain-level characteristics. The p-values in Table 2 therefore overstate significance. For the largest reported correlation (ρ=0.136 for ζ_sem vs. FC), treating n=131 documents as the effective sample yields t≈1.56 (p≈0.12), not p<0.05. Please reanalyze with a mixed-effects model with random intercepts for document and domain, or with per-document correlations across chunking configurations, and use cluster-robust or block-bootstrap standard errors.
  2. [Section 5 and Figure 7] The headline 'performance gain of up to 56.2% in factual correctness and 21.1% in answer correctness' is not an estimate of the effect of semantic independence. It is a min–max comparison of RAG scores at the extremes of the pooled ζ_sem distribution, and those extremes are confounded with chunking method, chunk size, document, and domain. A causal or even 'essential' role for semantic independence requires either a within-document comparison (e.g., comparing chunkings of the same document that differ in ζ_sem) or a regression controlling for chunking configuration and document effects. As reported, the magnitudes are not interpretable as the effect of semantic independence.
  3. [Section 4.1 and domain-agnostic claim] The paper claims a domain-agnostic metric based on seven domains, but no per-domain correlations or interaction tests are reported. With as few as 5 documents in the debate domain and 10 in several others, the pooled correlations could be driven by one or two domains. Please report per-domain correlation tables or a model with domain interactions to support the domain-agnostic claim.
  4. [Equation (2) and Section 3.2] The concept-unity score sums over all ordered pairs of statements, including i=j. Since θ(s_i,s_i)=1, the diagonal term inflates ζ_con by an amount that depends on |S|, and the double sum counts each unordered pair twice. This affects the very submetric used to support the conclusion that 'traditional assumptions about maintaining concept unity within passages show minimal impact.' Please exclude the diagonal (and state whether pairs are ordered) or justify the current normalization.
  5. [Section 3.5, Eq. (10)] HOPE is defined as an equal-weight arithmetic mean of the three submetrics, but no justification or sensitivity analysis is given. Table 2 shows that the submetrics have different signs and significance levels (e.g., ζ_con correlates negatively with CR, while ζ_sem correlates positively), so the aggregate HOPE score and its correlations could depend on the chosen weights. Please provide a sensitivity analysis (e.g., leave-one-out submetric analysis, or regression-based weighting) or a theoretical rationale for equal weighting.
minor comments (6)
  1. [Abstract] The abstract states 'correlates significantly (p > 0.13)'; this should read 'ρ > 0.13', since p is a significance level, not an effect size.
  2. [Section 1.3 and Figure 2] The heading 'Principals of Chunking' contains a typo and should be 'Principles'; the Figure 2 caption 'Statments' should be 'Statements'.
  3. [Section 2.1] The text says 'event for strong models like GPT-4'; this should be 'even for strong models like GPT-4'.
  4. [Table 2 and BLEU baseline] The BLEU baseline is not described: please specify whether BLEU is computed between the original document and the concatenated passages, and which tokenization/reference is used.
  5. [Section 3.3] The choice k=3 for the number of retrieved context passages in the semantic-independence computation is stated without justification; adding a robustness check over k would strengthen the metric.
  6. [Reproducibility] The manuscript states that the link to the HOPE implementation is removed during review; for an automatic metric paper, the code and all LLM prompts should be included in the supplementary material to make the results reproducible.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: HOPE's sub-metrics are computed independently of RAGAS outcomes, and the paper's correlational claims are external validations rather than fitted predictions.

full rationale

I walked the paper's derivation chain. The three HOPE sub-metrics are defined directly from the three chunking principles using LLM and embedding computations: concept unity (Eqs. 2-3), semantic independence (Eqs. 4-6), and information preservation (Eqs. 8-9). None of these equations contains a fitted parameter tuned to RAGAS scores; RAGAS is used only as an external outcome variable in the correlation analysis. The aggregation in Eq. 10 is an unweighted arithmetic mean, which is a stated design choice rather than an inverse-engineered fit to the empirical results. The headline claims about performance gains (21.1% and 56.2%) are derived from comparing RAGAS scores at the extremes of the observed ζ_sem distribution, which is a data-exploratory comparison, not a prediction that HOPE itself generates from RAG outcomes. There are no load-bearing self-citations, no imported uniqueness theorems, and no ansatz smuggled in via citation; the cited works on claim decomposition, decontextualization, and RAGAS are external. The pooled-correlation and nesting concern raised by the reader is a legitimate statistical-inference issue about independence of observations, but it is not circularity, because the HOPE values are not constructed from the RAGAS ground truth. Accordingly, no circular step meets the required standard of exhibiting a reduction of the claimed result to its own inputs, and the honest finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

HOPE is a methodological construct, not a physical entity; the free parameters are design choices (k, equal weights, statement/question counts, segment length). The axioms are domain assumptions about embedding similarity, LLM reliability, representativeness of sampling, validity of RAGAS, and completeness of the three principles.

free parameters (4)
  • k, number of retrieved context passages in semantic independence = 3
    Set to 3 by hand in Section 3.3; affects the subset of passages used to test whether a passage's interpretation changes with additional context.
  • Equal weighting of HOPE sub-metrics = 1/3 each
    HOPE is the arithmetic mean of concept unity, semantic independence, and information preservation (Eq. 10); no justification is given for equal weights, and other weightings could change correlations.
  • Number of LLM-generated statements/questions per passage = unspecified
    Section 3.2 and 3.3 do not specify the size of the statement set S and question set Q; this affects the variance and stability of the sub-metrics.
  • Three-sentence segment sampling for information preservation = 3 consecutive sentences
    Section 3.4 samples segments of exactly three sentences; this captures only local information and may miss long-range dependencies, acknowledged in the text.
assumptions (5)
  • domain assumption Cosine similarity of embedding vectors is a valid measure of semantic similarity.
    Used throughout HOPE (Eq. 1) to compare statements, answers, and passages; the paper notes embedding methods may miss nuanced differences, but this is not validated against human judgments.
  • domain assumption LLM-generated statements and questions are faithful and diverse.
    Assumed in Section 3.5; Section 5.2 acknowledges that LLM bias can reduce diversity, which would directly impact concept unity and information preservation.
  • domain assumption Uniform sampling of three-sentence segments represents the document's information content.
    Section 3.4 uses U(D) to sample segments; the paper acknowledges the limitation for detecting complex relationships across longer distances.
  • domain assumption RAGAS scores are valid ground-truth measures of RAG performance.
    The paper uses RAGAS Answer Correctness, Factual Correctness, Response Relevancy, and Context Recall as dependent variables without questioning their validity; errors in RAGAS would propagate to the correlations.
  • ad hoc to paper The three principles (concept unity, semantic independence, information preservation) are the essential dimensions of chunking quality.
    These principles are introduced in Section 1.3 as the foundation of HOPE; other possible dimensions (e.g., passage length, retrieval robustness, query-conditional quality) are not considered.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A New HOPE: Domain-agnostic Automatic Evaluation of Text Chunking." pith.science (2026). https://pith.science/paper/RG6JXY3L

@misc{pith2026250502171,
  author       = {Pith},
  title        = {Pith review of: A New HOPE: Domain-agnostic Automatic Evaluation of Text Chunking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RG6JXY3L}},
  note         = {Machine review of arXiv:2505.02171}
}
read the original abstract

Document chunking fundamentally impacts Retrieval-Augmented Generation (RAG) by determining how source materials are segmented before indexing. Despite evidence that Large Language Models (LLMs) are sensitive to the layout and structure of retrieved data, there is currently no framework to analyze the impact of different chunking methods. In this paper, we introduce a novel methodology that defines essential characteristics of the chunking process at three levels: intrinsic passage properties, extrinsic passage properties, and passages-document coherence. We propose HOPE (Holistic Passage Evaluation), a domain-agnostic, automatic evaluation metric that quantifies and aggregates these characteristics. Our empirical evaluations across seven domains demonstrate that the HOPE metric correlates significantly (p > 0.13) with various RAG performance indicators, revealing contrasts between the importance of extrinsic and intrinsic properties of passages. Semantic independence between passages proves essential for system performance with a performance gain of up to 56.2% in factual correctness and 21.1% in answer correctness. On the contrary, traditional assumptions about maintaining concept unity within passages show minimal impact. These findings provide actionable insights for optimizing chunking strategies, thus improving RAG system design to produce more factually correct responses.

Figures

Figures reproduced from arXiv: 2505.02171 by the authors.

Figure 1
Figure 1. The three levels of holistic chunking evaluation. Left: The intrinsic passage properties are concerned with the [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Calculating the concept unity ¯𝜁𝑐𝑜𝑛: The passage in green is forwarded to an LLM to produce a set of statements S. The statements are then transformed into vector representations using an embedding model. The pairwise cosine similarity between the vectors is calculated. high semantic similarity, which we measure using the cosine simi￾larity of vector embeddings as shown in equation 2 and illustrated in [PITH_FULL_I… view at source ↗
Figure 3
Figure 3. Calculation of the semantic independence [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Calculation of the information preservation [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Documents and questions from diverse domains are processed through eight chunking configurations (four fixed-size, [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Distributions of the HOPE values for all 1048 combinations of documents and chunking methods. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: RAG performance indicators plotted against the [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 18 canonical work pages

  1. [1]

    Scott Barnett, Stefanus Kurniawan, Srikanth Thudumu, Zach Brannelly, and Mohamed Abdelrazek. 2024. Seven Failure Points When Engineering a Retrieval Augmented Generation System. section 5, 194–199. isbn: 9798400705915. doi: 10.1145/3644815.3644945

    arXiv 2024

  2. [2]

    Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries Chin-Yew. Text summarization branches out, 74–81. doi: 10.1253/jcj.34.1213

    work page doi:10.1253/jcj.34.1213 2004

  3. [3]

    Florin Cuconasu, Giovanni Trappolini, Federico Siciliano, Simone Filice, Cesare Campagnano, Yoelle Maarek, Nicola Tonellotto, and Fabrizio Silvestri. 2024. The Power of Noise: Redefining Retrieval for RAG Systems.SIGIR 2024 - Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 719–729. isbn: 979...

    work page doi:10.1145/3626772.365 2024

  4. [4]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova Google, and A I Language. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Tech. rep. https://github.com/tensorflow/tensor2ten sor

    work page 2019

  5. [5]

    Abhimanyu Dubey et al. 2024. The Llama 3 Herd of Models, 1–92. http://arxiv .org/abs/2407.21783

    arXiv 2024

  6. [6]

    Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, and Jonathan Larson. 2024. From Local to Global: A Graph RAG Approach to Query-Focused Summarization, 1–15. http://arxiv.org/abs/2 404.16130

    work page 2024

  7. [7]

    Shahul Es, Jithin James, Luis Espinosa-Anke, and Steven Schockaert. 2024. RAGAS: Automated Evaluation of Retrieval Augmented Generation. EACL 2024 - 18th Conference of the European Chapter of the Association for Com- putational Linguistics, Proceedings of System Demonstrations , 150–158. isbn: 9798891760912

    work page 2024

  8. [8]

    Anisha Gunjal and Greg Durrett. 2024. Molecular Facts: Desiderata for Decon- textualization in LLM Fact Verification. http://arxiv.org/abs/2406.20079

    arXiv 2024

Show all 42 references
  1. [9]

    Jiafeng Guo, Yinqiong Cai, Yixing Fan, Fei Sun, Ruqing Zhang, and Xueqi Cheng. 2022. Semantic Models for the First-Stage Retrieval: A Comprehensive Review. ACM Transactions on Information Systems , 40, 4, (Oct. 2022). doi: 10.11 45/3486250

    2022

  2. [10]

    Krass*, Lucia Zheng, Neel Guha, Christopher D

    Peter Henderson*, Mark S. Krass*, Lucia Zheng, Neel Guha, Christopher D. Manning, Dan Jurafsky, and Daniel E. Ho. 2022. Pile of law: learning responsible data filtering from the law and a 256gb open-source legal dataset. (2022). https: //arxiv.org/abs/2207.00220

    2022 arXiv

  3. [11]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring Massive Multitask Lan- guage Understanding. ICLR 2021 - 9th International Conference on Learning Representations

    2021

  4. [12]

    Kenton Lee, Ming Wei Chang, and Kristina Toutanova. 2020. Latent retrieval for weakly supervised open domain question answering. ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, 6086–6096. isbn: 9781950737482. doi:...

    2020 doi

  5. [13]

    Seungjun Lee, Jungseob Lee, Hyeonseok Moon, Chanjun Park, Jaehyung Seo, Sugyeong Eo, Seonmin Koo, and Heuiseok Lim. 2023. A Survey on Evaluation Metrics for Machine Translation. Mathematics, 11, 4, 1–22. doi: 10.3390/math1 1041006

    2023 doi

  6. [14]

    Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023. Towards General Text Embeddings with Multi-stage Contrastive Learning. http://arxiv.org/abs/2308.03281

    2023 arXiv

  7. [15]

    Lei Liang et al. 2024. KAG: Boosting LLMs in Professional Domains via Knowl- edge Augmented Generation, 1–33. http://arxiv.org/abs/2409.13731

    2024 arXiv

  8. [16]

    Lin Long, Rui Wang, Ruixuan Xiao, Junbo Zhao, Xiao Ding, Gang Chen, and Haobo Wang. 2024. On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey. isbn: 9798891760998. http://arxiv.org/abs/2406.15126

    2024 arXiv

  9. [17]

    Anurag Mishra. [n. d.] Five Levels of Chunking Strategies in RAG| Notes from Greg’s Video — anuragmishra_27746. https://medium.com/@anuragmishra_27 746/five-levels-of-chunking-strategies-in-rag-notes-from-gregs-video-7b7 35895694d. [Accessed 20-11-2024]. ()

    2024

  10. [18]

    Niklas Muennighoff, Nouamane Tazi, Loïc Magne, and Nils Reimers. 2023. MTEB: Massive Text Embedding Benchmark.EACL 2023 - 17th Conference of the European Chapter of the Association for Computational Linguistics, Proceedings of the Conference, 2006–2029. isbn: 9781959429449. do...

    2023 doi

  11. [19]

    Arvind Neelakantan et al. 2022. Text and Code Embeddings by Contrastive Pre-Training. http://arxiv.org/abs/2201.10005

    2022 arXiv

  12. [20]

    Benjamin Newman, Luca Soldaini, Raymond Fok, Arman Cohan, and Kyle Lo

  13. [21]

    OpenAI. 2023. GPT-4 Technical Report. 4, 1–100. http://arxiv.org/abs/2303.087 74

    2023

  14. [22]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a Method for Automatic Evaluation of Machine Translation. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL) , 311–318. doi: 10.1002/andp.19223712302

    2002 doi

  15. [23]

    Rajvardhan Patil, Sorio Boit, Venkat Gudivada, and Jagadeesh Nandigam. 2023. A Survey of Text Representation and Embedding Techniques in NLP. IEEE Access, 11, 36120–36146. doi: 10.1109/ACCESS.2023.3266377

    2023

  16. [24]

    Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models. NeurIPS. http://arxiv.org/abs/2104.08663

    2021 arXiv

  17. [25]

    Hugo Touvron et al. 2023. Llama 2: Open Foundation and Fine-Tuned Chat Models. http://arxiv.org/abs/2307.09288

    2023 arXiv

  18. [26]

    Lucy Lu Wang et al. 2020. CORD-19: the COVID-19 open research dataset. In Proceedings of the 1st Workshop on NLP for COVID-19 at ACL 2020 . Association for Computational Linguistics, Online, (July 2020). https://www.aclweb.org/an thology/2020.nlpcovid19-acl.1

    2020

  19. [27]

    Zhilin Wang, Alexander Bukharin, Olivier Delalleau, Daniel Egert, Gerald Shen, Jiaqi Zeng, Oleksii Kuchaiev, and Yi Dong. 2024. HelpSteer2-Preference: Complementing Ratings with Preferences, 1–26. http://arxiv.org/abs/2410.0125 7

    2024

  20. [28]

    Miriam Wanner, Seth Ebner, Zhengping Jiang, Mark Dredze, and Benjamin Van Durme. 2024. A Closer Look at Claim Decomposition, 153–175. isbn: 9798891761063. doi: 10.18653/v1/2024.starsem-1.13

    2024 doi

  21. [29]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. Advances in Neural Information Processing Systems, 35, NeurIPS, 1–14. isbn: 9781713871088

    2022

  22. [30]

    Jules White, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith, and Douglas C Schmidt. 2023. A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT. http://arxiv.org/abs/2302.11382

    2023 arXiv

  23. [31]

    Shangyu Wu et al. 2024. Retrieval-Augmented Generation for Natural Language Processing: A Survey. http://arxiv.org/abs/2407.13193

    2024 arXiv

  24. [32]

    Siye Wu, Jian Xie, Jiangjie Chen, Tinghui Zhu, Kai Zhang, and Yanghua Xiao

  25. [33]

    Zhiheng Xi et al. 2023. The Rise and Potential of Large Language Model Based Agents: A Survey. http://arxiv.org/abs/2309.07864

    2023 arXiv

  26. [34]

    Antonio Jimeno Yepes, Yao You, Jan Milczek, Sebastian Laverde, and Renyu Li

  27. [35]

    Qiu, and Lili Qiu

    Siyun Zhao, Yuqing Yang, Zilong Wang, Zhiyuan He, Luna K. Qiu, and Lili Qiu

  28. [36]

    Wayne Xin Zhao et al. 2023. A Survey of Large Language Models, (Mar. 2023). http://arxiv.org/abs/2303.18223

    2023 arXiv

  29. [37]

    http://arxiv.org/abs/2402.05131

    Financial Report Chunking for Effective Retrieval Augmented Generation. http://arxiv.org/abs/2402.05131

    arXiv

  30. [39]

    http://arx iv.org/abs/2409.14924

    Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wisely. http://arx iv.org/abs/2409.14924

    arXiv

  31. [41]

    Zijie Zhong, Hanwen Liu, Xiaoya Cui, Xiaofan Zhang, and Zengchang Qin

  32. [42]

    http://arxiv.org/abs/2406.00456

    Mix-of-Granularity: Optimize the Chunking Granularity for Retrieval- Augmented Generation, 1–17. http://arxiv.org/abs/2406.00456

    arXiv

  33. [2023]

    EMNLP 2023 - 2023 Conference on Em- pirical Methods in Natural Language Processing, Proceedings , 3194–3212

    A Question Answering Framework for Decontextualizing User-facing Snippets from Scientific Documents. EMNLP 2023 - 2023 Conference on Em- pirical Methods in Natural Language Processing, Proceedings , 3194–3212. isbn: 9798891760608. doi: 10.18653/v1/2023.emnlp-main.193

    2023 doi

  34. [2024]

    http://arxiv.org/abs/2404.03302

    How Easily do Irrelevant Inputs Skew the Responses of Large Language Models? 1–20. http://arxiv.org/abs/2404.03302

    arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.