Pith. sign in

REVIEW 5 major objections 5 minor 46 references

Multi-document reasoning is budget-constrained, not context-constrained, and a hierarchical bi-path retriever can match full-context LLMs using 94% fewer tokens.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 08:05 UTC pith:AIVYTPUA

load-bearing objection FABLE is a plausible new retrieval architecture, but the paper's central efficiency claim is undermined by an internal contradiction between the 65.7% full-document upper bound and the 92.5% ablation result for the same generator. the 5 major comments →

arxiv 2601.18116 v2 pith:AIVYTPUA submitted 2026-01-26 cs.CL

BEAR: Budgeted Evidence Allocation for Multi-Document Reasoning

Lin Sun , Linglin Zhang , Jingang Huang , Change Jia , Zhengwei Cheng , Xiangzheng Zhang This is my paper
classification cs.CL
keywords retrieval-augmented generationmulti-document reasoninghierarchical semantic indexingLLM-guided navigationbudget-constrained evidence allocationlong-context LLMmulti-hop question answeringtree-based retrieval
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper is trying to establish that multi-document reasoning is limited not by how much text a model can read, but by how a fixed query-time evidence budget is allocated across documents and semantic granularities. It argues that full-context inference is non-selective and expensive, while flat chunk retrieval returns locally relevant passages that lack the structure needed for cross-document synthesis. In response, it proposes a framework that pre-builds LLM-generated hierarchical semantic indexes and, at query time, navigates them coarse-to-fine through two complementary paths under an explicit token budget. If the reported results hold, the framework matches or beats a much larger full-context model while using about six percent of the tokens, and the efficiency gain comes from the hierarchy and bi-path design rather than from semantic chunking alone. A sympathetic reader would care because this reframes the retrieval-versus-long-context debate as an allocation problem with measurable cost savings.

Core claim

Central claim: multi-document reasoning is budget-constrained, not context-constrained—under a fixed query-time token budget, coarse-to-fine retrieval over LLM-built semantic hierarchies can match or beat full-context inference. The framework builds per-document trees (internal nodes = ToC title + summary; leaves = semantic chunks) and retrieves via two complementary paths: depth-limited LLM-guided selection and structure-aware propagation down tree edges, with budget-adaptive routing between document and node granularity. Reported: 92.07% completeness at 8K generation tokens vs 91.05% for a 517K-token full-context run (94% reduction); best retrieval-based EM/F1 on 2Wiki; +22.14 accuracy on

What carries the argument

Key machinery is the LLM-built semantic forest: per-document trees with typed internal nodes (ToC-style title + summary) and semantic-chunk leaves, embedded as title-path-plus-summary vectors. Query-time retrieval has two paths: depth-limited LLM-guided selection/navigation over high-level abstractions, and TreeExpansion, which ranks nodes by a three-signal score—direct similarity with depth decay, ancestor-inherited relevance, and child-aggregated relevance—then greedily fills the budget. A budget-adaptive router stops at document level when coarse results fit, otherwise descends to node level; fusion dedupes ancestor/descendant pairs and orders chunks positionally, LLM picks first. The mec

Load-bearing premise

Everything rests on the assumption that an LLM-generated title-plus-summary is a lossless proxy for the content in its descendant chunks: if a summary omits, distorts, or invents a fact the answer depends on, both the LLM navigation path and the summary-embedding path will miss it, and the saving disappears for that query.

What would settle it

Build a synthetic corpus where each gold answer sits in a single known chunk, then generate summaries that systematically omit that chunk's key fact while summarizing the rest; if the framework's completeness on those queries at its claimed 8K budget collapses toward zero while full-context inference still answers, the central proxy assumption is refuted. Simpler check: compare retrieval via internal-node embeddings against retrieval over leaf chunks and measure how often the gold evidence is found only through the leaf.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the reported 94% token reduction holds, long-context inference is not a substitute for structured retrieval on multi-document reasoning; retrieval architecture—not context size—becomes the dominant lever for faithful answers.
  • Budget-adaptive routing means per-query token cost scales with evidence complexity rather than corpus size: when document-level evidence fits the budget, the system stops before fine-grained traversal.
  • The bi-path ablations show LLM reasoning and structural propagation have complementary strengths—LLM-only wins at very tight budgets, fusion wins at moderate budgets—so the best path depends on the budget tier.
  • On BrowseComp-plus, replacing the retriever alone raised agent accuracy by 22 points with the same agent LLM, indicating retriever quality is a first-order constraint on agent performance.
  • The paper itself states the advantages diminish on highly unstructured corpora or queries that rely solely on keyword matching.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the 94% token savings become real cost savings only if the LLM doing selection/navigation is cheaper than the full-context feed; otherwise the economic win is smaller than the token reduction suggests.
  • Because internal titles/summaries are the only view of document content, a summarizer that systematically omits the exact fact a question needs would defeat the framework—a stress test for the lossless-proxy premise.
  • The structural scoring weights are uniform and untuned; the paper's own finding that LLM-only beats fusion at 1K tokens suggests query-adaptive weighting of the two paths is a natural improvement it does not explore.
  • The position-preserving ordering of retrieved chunks (LLM picks before expansion picks, both in document order) implies evidence ordering, not just selection, contributes to the gains—connecting to known positional-bias effects in long-context models.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes FABLE, a retrieval framework that builds a per-document hierarchical 'semantic forest' offline: documents are LLM-chunked, an LLM generates a tree with titles, summaries, and leaf chunks, and embeddings are indexed. At query time, retrieval runs two paths at document and node levels: LLM-guided traversal over non-leaf nodes and vector/structural propagation (TreeExpansion with fixed equal weights), with a budget controller that decides whether to return whole documents or node-level evidence. Claims: on DragonBall, FABLE(docs) reaches 92.07% completeness and low hallucination, surpassing Gemini-2.5-Pro full-context (91.05%) while using 31K tokens vs 517K (94% reduction); on HotpotQA and 2Wiki it is competitive or state-of-the-art among retrieval baselines; on BrowseComp-plus it improves agent accuracy and recall. Ablations attribute gains to hierarchy and bi-path fusion rather than semantic chunking alone.

Significance. Conditional significance is high. If the results survive scrutiny, FABLE would demonstrate that query-time token budgets, not context length, are the bottleneck for multi-document reasoning, and that LLM-built hierarchies plus bi-path retrieval can beat indiscriminate full-context inference at a fraction of the cost. The design is clearly specified, and the TreeExpansion score uses intentionally un-tuned uniform 1/3 weights, which is a genuine strength: the reported gains are not from overfitting weights to the benchmark. The evaluation spans synthetic QA, multi-hop QA, and an agent setting, giving breadth. However, the paper does not release code or data, and the central efficiency claim is currently supported only by cross-model comparisons and an internally inconsistent ablation. The contribution is therefore promising but not yet established.

major comments (5)
  1. [§1 bullet; Table 1; §4.2] The headline claim that FABLE matches full-context LLM performance (92.07% vs Gemini-2.5-Pro's 91.05%) is an uncontrolled cross-model comparison. Per §4.2, DragonBall uses DeepSeek-V3.2 throughout, so FABLE(docs) is generated by DeepSeek-V3.2, while the full-context baselines in Table 1 are Qwen3-32B and Gemini models. The 1.02-point gap could be entirely due to generator strength. The statement in §4.2 that 'relative comparisons are always performed under identical LLM settings' is not honored for this headline comparison. Please report a same-generator full-context DeepSeek-V3.2 row and derive the token-reduction claim from that controlled comparison.
  2. [§5.3.1 vs §5.3.2] The ablation section states that all ablation experiments use Qwen3-32B for LLM generation. §5.3.1 reports llm-chunks at 8K reaching 64.5% and explicitly calls 65.7% 'the upper bound of qwen3-32B with full document access'; this 65.7% matches Table 1's Qwen3-32B full-context completeness 65.73. Yet §5.3.2 reports FABLE(docs) at 4K reaching 92.5% with the same Qwen3-32B generator. A retrieval method cannot exceed the full-document upper bound on the same dataset and metric. Either 92.5% is wrong, 65.7% is not an upper bound, or §5.3.2 uses a different generator/split than §5.3.1. This is not a presentation nit: the paper's central efficiency claim ('retrieval beats full context') depends on reconciling these numbers.
  3. [§1 Contributions; Table 1] The contribution bullet reports '93.65% completeness' while Table 1 lists FABLE(docs) at 92.07%, with both sharing the same hallucination (5.37%) and irrelevance (2.52%) values. The paper therefore contains two different headline results for the same configuration. The authors should correct the bullet or the table, and all derived claims (e.g., 'surpassing Gemini' and the EM deltas) should be recomputed from the corrected value.
  4. [§3.1, Eqs. (3)-(5)] The retrieval architecture assumes that LLM-generated internal nodes (toc_path + summary) are faithful, lossless proxies for descendant leaf content: both Eq. (5) embedding retrieval and LLM navigation operate on these summaries. If a summary omits, distorts, or hallucinates evidence present only in a descendant chunk, no downstream step can recover it. The paper provides no analysis of summary fidelity or error propagation. This is especially relevant for the batch-wise construction in Eqs. (3)-(4), which is invoked for documents exceeding the LLM context window; the claim that merging partial trees 'maintains cross-part semantic coherence' is asserted but not validated. Since the efficiency advantage over long-context inference depends on compressing long documents into bounded summaries, please add an analysis or ablation measuring how often gold evidence is absent from internal nodes
  5. [§4.3] On DragonBall, Completeness, Hallucination, and Irrelevance are evaluated using DeepSeek-V3.2 as the LLM judge. Per §4.2, DeepSeek-V3.2 is also the backbone that constructs the semantic forests and serves as the generator for FABLE's rows. Using the same model both to build the index and to grade the output creates a potential circularity: the judge may favor the structure that the same model produced. Please report agreement with an alternative judge (e.g., a different model or human annotation) or justify why this is not a confound.
minor comments (5)
  1. [§3.2, TreeExpansion Details] The text says scores are computed 'via Eq. 4', but the composite score is defined in Eq. (7). Eq. (4) is the partial-tree merge operation. Please correct the cross-reference.
  2. [§5.3.2, §5.3.3] Gemini-2.5-Pro's completeness is cited as 91.1% in the ablation text but 91.05% in Table 1. Rounding is fine, but the values should be consistent across the paper.
  3. [References] References [13] and [14] are the same paper (Ho et al., 2020, COLING) and should be merged.
  4. [§5.3.4, Figures 4-5] The high-level claims in §5.3.4 (e.g., FABLE(Nodes) average 97.7%, +22.8, +35.5) are not backed by a table or error bars in the figures. Please provide the underlying numbers or a small table for reproducibility.
  5. [General] The paper states that code, prompts, and constructed forests 'will be released upon acceptance.' For a reproducibility-oriented evaluation, making the semantic forests or at least a hosted index available with the submission would substantially strengthen the reader's ability to verify the claims.

Circularity Check

0 steps flagged

No circular derivation: FABLE is empirical engineering with no fitted-parameter predictions, no self-citation chain, and no definitional tautology; noted evaluation confounds are correctness risks, not circularity.

full rationale

FABLE's claims are empirical comparisons rather than derivations from first principles. The retrieval pipeline (Eqs. 1–8) is defined independently of the target metrics: TreeExpansion uses fixed uniform weights (1/3 each) explicitly not tuned, no parameter is fitted to DragonBall/HotpotQA/2Wiki/BrowseComp-plus outcomes, and the efficiency claim (31K tokens vs. 517K full context) is a measured comparison, not a prediction derived from its inputs. There are no self-citations by the present authors; cited baselines and datasets are external. Two textual concerns are not circular: (i) the same model, DeepSeek-V3.2, both constructs the semantic forest and judges DragonBall completeness/hallucination/irrelevance (Secs. 4.2–4.3), which weakens evaluative independence but does not make any result true by definition; (ii) Sec. 5.3.1 reports a 65.7% full-document upper bound for Qwen3-32B while Sec. 5.3.2 reports FABLE(docs) at 4K achieving 92.5% with the same generator, an internal inconsistency affecting the validity of the headline comparison but not constituting circularity. Since no load-bearing step reduces to its own inputs, the circularity score is 0.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

FABLE introduces no new physical or ontological entities; its semantic forest is an engineering artifact. The central claims rest on domain assumptions about the fidelity of LLM-generated summaries, the coherence of merged partial trees, and the trustworthiness of an LLM-family judge. Free parameters are limited to architectural choices and the budget-sweep selection protocol; the uniform fusion weights are explicitly not fitted.

free parameters (3)
  • hierarchy depth D = 4
    Configured maximum tree depth (Section 4.2); no sensitivity analysis reported, yet it bounds LLM-visible summary context and affects both budget and recall.
  • FAISS TopK K_doc = not stated
    Vector-path document selection uses TopK FAISS (Algorithm 1, line 7) but the value of K_doc is never given, so the balance between the LLM path and vector path is not reproducible.
  • reported budget Bmax selection = 1K–128K; headline at ~8K
    Bmax is swept from 1K to 128K 'depending on experimental conditions' (Section 4.2); headline claims (92.07% completeness, 31K tokens, 94% reduction) correspond to a favorable point of the sweep rather than a fixed protocol.
axioms (4)
  • domain assumption LLM-generated node titles and summaries faithfully represent descendant chunk content (Eq. 5 embedding of toc_path + summary)
    Both retrieval paths — LLM navigation and TreeExpansion — operate over summaries; if summaries are lossy or hallucinated, the evidence upstream of the chunks is corrupted. This is the load-bearing proxy assumption.
  • domain assumption Batch-wise partial-tree construction and merging preserves cross-part semantic coherence for long documents (Eqs. 3-4)
    Documents exceeding the LLM context window are split, built into partial trees, and merged. The paper asserts coherence without evaluation, and this is exactly the regime where long-document evidence is most at risk.
  • domain assumption DeepSeek-V3.2 is a trustworthy oracle for completeness/hallucination/irrelevance on DragonBall
    DragonBall end-to-end metrics are 'evaluated using DeepSeek-V3.2' (Section 4.3), the same model family that performs the index construction — a self-consistent but self-referential evaluation loop.
  • domain assumption HippoRAG2 is the current SOTA structured-RAG baseline, having been 'shown to outperform RAPTOR, GraphRAG, and LightRAG in recent evaluations'
    Section 2.3 asserts this without a citation; it justifies benchmarking only HippoRAG2 among structured methods, but the supporting evidence is not provided in the paper.

pith-pipeline@v1.3.0-alltime-deepseek · 14395 in / 13586 out tokens · 141030 ms · 2026-08-03T08:05:10.984631+00:00 · methodology

0 comments
read the original abstract

We argue that multi-document reasoning is constrained not only by how much text a model can read, but also by how limited query-time evidence budget is allocated across documents and semantic granularities. Full-context inference exposes the model to broad evidence non-selectively and at high per-query cost, while flat chunk retrieval often returns locally relevant passages that are weakly organized for cross-document synthesis. We present \textbf{BEAR}, a framework for structured evidence allocation that builds hierarchical semantic indices offline and performs coarse-to-fine evidence access at query time through complementary \emph{exploration} and \emph{recovery} paths. This coarse-to-fine design can be viewed as structured evidence allocation under a fixed evidence-context budget. Across synthetic and real-world benchmarks, BEAR performs particularly strongly on DragonBall, remains competitive with strong retrieval-based baselines on HotpotQA, and yields the best retrieval-based result on 2Wiki under our evaluated protocol, while operating under substantially smaller \emph{query-time evidence budgets} than the reported long-context references. Additional analyses suggest that the gains are associated with hierarchy as an allocation substrate together with complementary exploration and recovery, rather than semantic chunking alone.

Figures

Figures reproduced from arXiv: 2601.18116 by Change Jia, Jingang Huang, Linglin Zhang, Lin Sun, Xiangzheng Zhang, Zhengwei Cheng.

Figure 1
Figure 1. Figure 1: Overview of FABLE, a forest-based adaptive bi-path LLM-enhanced retrieval framework [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Semantic tree construction and multi-granularity vector indexing: (1) semantic-aware chunking, (2) LLM-based [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Performance of retrieval and context construction strategies across varying input lengths. Long-context models use [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: bi-path performance across domains and languages. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: bi-path performance across different query types. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

46 extracted references · 1 canonical work pages

  1. [1]

    Anthropic. 2024. Introducing the Claude 3 Model Family. https://www.anthropic. com/news/claude-3-family. Accessed: 2024-01-16

    2024

  2. [2]

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2023. Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. arXiv:2310.11511 [cs.CL] https://arxiv.org/abs/2310.11511 9

    Pith/arXiv arXiv 2023

  3. [3]

    Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, et al. 2024. Longbench: A bilingual, multitask benchmark for long context understanding. InProceedings of the 62nd annual meeting of the association for computational linguistics (volume 1: Long papers). 3119–3137

    2024

  4. [4]

    Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu

  5. [5]

    Zijian Chen, Xueguang Ma, Shengyao Zhuang, Ping Nie, Kai Zou, Andrew Liu, Joshua Green, Kshama Patel, Ruoxi Meng, Mingyi Su, Sahel Sharifymoghad- dam, Yanxi Li, Haoran Hong, Xinyu Shi, Xuye Liu, Nandan Thakur, Crystina Zhang, Luyu Gao, Wenhu Chen, and Jimmy Lin. 2025. BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent...

    Pith/arXiv arXiv 2025

  6. [6]

    DeepSeek-AI. 2025. DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models

    2025

  7. [7]

    Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, and Jonathan Larson. 2024. From Local to Global: A Graph RAG Approach to Query-Focused Summarization.arXiv preprint arXiv:2404.16130 (2024)

    Pith/arXiv arXiv 2024

  8. [8]

    Luyu Gao, Xueguang Ma, Jimmy Lin, and Jamie Callan. 2022. Precise Zero- Shot Dense Retrieval without Relevance Labels. arXiv:2212.10496 [cs.IR] https: //arxiv.org/abs/2212.10496

    Pith/arXiv arXiv 2022

  9. [9]

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. 2024. Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv:2312.10997 [cs.CL] https://arxiv.org/abs/2312.10997

    Pith/arXiv arXiv 2024

  10. [10]

    Zirui Guo, Xiaohua Lian, Yanhua Yang, Hanzhi Huang, Shuwen Liu, Yixuan Feng, Yiding Liu, and Jinhao Li. 2024. LightRAG: Simple and Fast Retrieval-Augmented Generation.arXiv preprint arXiv:2410.05779(2024)

    Pith/arXiv arXiv 2024

  11. [11]

    Bernal Jiménez Gutiérrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, and Yu Su

  12. [12]

    Bernal Jiménez Gutiérrez, Yiheng Shu, Weijian Qi, Sizhe Zhou, and Yu Su. 2025. From RAG to Memory: Non-Parametric Continual Learning for Large Language Models. arXiv:2502.14802 [cs.CL] https://arxiv.org/abs/2502.14802

    Pith/arXiv arXiv 2025

  13. [13]

    InAdvances in Neural Information Processing Systems, Vol

    HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models. InAdvances in Neural Information Processing Systems, Vol. 37

  14. [14]

    Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reason- ing Steps. InProceedings of the 28th International Conference on Computational Linguistics, Donia Scott, Nuria Bel, and Chengqing Zong (Eds.). International Committee on Computational Linguistics, Barcelona, Sp...

    doi:10.18653/v1/2020.coling-main.580 2020

  15. [15]

    Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reason- ing Steps. InProceedings of the 28th International Conference on Computational Linguistics. 6609–6625

    2020

  16. [16]

    Jiajie Jin, Xiaoxi Li, Guanting Dong, Yuyao Zhang, Yutao Zhu, Yongkang Wu, Zhonghua Li, Ye Qi, and Zhicheng Dou. 2025. Hierarchical Document Refinement for Long-context Retrieval-augmented Generation. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Wanxiang Che, Joyce Nabende, Ekaterina Sh...

    2025

  17. [17]

    Haoyu Huang, Yongfeng Huang, Junjie Yang, Zhenyu Pan, Yongqiang Chen, Kaili Ma, Hongzhi Chen, and James Cheng. 2025. Retrieval-Augmented Generation with Hierarchical Knowledge.arXiv preprint arXiv:2503.10150(2025)

    arXiv 2025

  18. [18]

    Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense Passage Retrieval for Open- Domain Question Answering. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 6769–6781

    2020

  19. [19]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. InAdvances in Neural Information Processing Systems, Vol. 33. 9459–9474

    2020

  20. [20]

    Greg Kamradt. 2023. Needle In A Haystack - Pressure Testing LLMs. https: //github.com/gkamradt/LLMTest_NeedleInAHaystack. Accessed: 2024-01-16

    2023

  21. [21]

    Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2023. Lost in the Middle: How Language Models Use Long Contexts. InAdvances in Neural Information Processing Systems, Vol. 36

    2023

  22. [22]

    Niklas Muennighoff, Nouamane Tazi, Loïc Magne, and Nils Reimers. 2023. MTEB: Massive Text Embedding Benchmark. InProceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. 2014–2037

    2023

  23. [23]

    Chaofan Li, Zheng Liu, Shitao Xiao, and Yingxia Shao. 2023. Making Large Lan- guage Models A Better Foundation For Dense Retrieval. arXiv:2312.15503 [cs.CL]

    arXiv 2023

  24. [24]

    OpenAI. 2023. GPT-4 Turbo: Announcing New Models and Developer Prod- ucts. https://openai.com/blog/new-models-and-developer-products-announced- at-devday. Accessed: 2024-01-16

    2023

  25. [25]

    OpenAI. 2025. gpt-oss-120b & gpt-oss-20b Model Card. arXiv:2508.10925 [cs.CL] https://arxiv.org/abs/2508.10925

    Pith/arXiv arXiv 2025

  26. [26]

    Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. 2022. WebGPT: Browser-assisted question-answering with human feedback. arXiv:2112.09...

    Pith/arXiv arXiv 2022

  27. [27]

    Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. In-Context Retrieval-Augmented Lan- guage Models. InTransactions of the Association for Computational Linguistics, Vol. 11. 1316–1331

    2023

  28. [28]

    Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy Lillicrap, Jean-baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, et al. 2024. Gemini 1.5: Unlocking Multimodal Understanding Across Millions of Tokens of Context.arXiv preprint arXiv:2403.05530(2024)

    Pith/arXiv arXiv 2024

  29. [29]

    Smith, and Mike Lewis

    Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A. Smith, and Mike Lewis. 2023. Measuring and Narrowing the Compositionality Gap in Language Models. arXiv:2210.03350 [cs.CL] https://arxiv.org/abs/2210.03350

    Pith/arXiv arXiv 2023

  30. [30]

    Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language Models Can Teach Themselves to Use Tools. arXiv:2302.04761 [cs.CL] https://arxiv.org/abs/2302.04761

    Pith/arXiv arXiv 2023

  31. [31]

    Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. 2023. REPLUG: Retrieval-Augmented Black-Box Language Models.arXiv preprint arXiv:2301.12652(2023)

    Pith/arXiv arXiv 2023

  32. [32]

    Parth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna, Anna Goldie, and Christopher D. Manning. 2024. RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval. InInternational Conference on Learning Representations

    2024

  33. [33]

    Wenyu Tao, Xiaofen Xing, Yirong Chen, Linyi Huang, and Xiangmin Xu. 2025. TreeRAG: Unleashing the Power of Hierarchical Storage for Enhanced Knowledge Retrieval in Long Documents. InFindings of the Association for Computational Linguistics: ACL 2025. Association for Computational Linguistics, Vienna, Austria, 356–371. doi:10.18653/v1/2025.findings-acl.20

    work page doi:10.18653/v1/2025.findings-acl.20 2025

  34. [34]

    Qwen Team. 2025. Qwen3 Technical Report. arXiv:2505.09388 [cs.CL] https: //arxiv.org/abs/2505.09388

    Pith/arXiv arXiv 2025

  35. [35]

    Yixuan Tang and Yi Yang. 2024. MultiHop-RAG: Benchmarking Retrieval- Augmented Generation for Multi-Hop Queries. arXiv:2401.15391 [cs.CL] https: //arxiv.org/abs/2401.15391

    Pith/arXiv arXiv 2024

  36. [36]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is All You Need. InAdvances in Neural Information Processing Systems. 5998–6008

    2017

  37. [37]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2369–2380

    2018

  38. [38]

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal

  39. [39]

    Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou

  40. [40]

    Kunlun Zhu, Yifan Luo, Dingling Xu, Yukun Yan, Zhenghao Liu, Shi Yu, Ruobing Wang, Shuo Wang, Yishan Li, Nan Zhang, Xu Han, Zhiyuan Liu, and Maosong Sun

  41. [42]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629 [cs.CL] https://arxiv.org/abs/2210.03629

    Pith/arXiv arXiv 2023

  42. [46]

    RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (Eds.). Association for Computational Linguistics, Vienna, Austria, 8520–8544. doi:10.18653/v1...

    doi:10.18653/v1/2025.acl-long.418 2025

  43. [2023]

    InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics

    Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge- Intensive Multi-Step Questions. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics. 10014–10037

  44. [2024]

    InFindings of the As- sociation for Computational Linguistics: ACL 2024, Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.)

    M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation. InFindings of the As- sociation for Computational Linguistics: ACL 2024, Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 2318–2335. doi:10.18653/v1/2024.findings-acl.137

    doi:10.18653/v1/2024.findings-acl.137 2024

  45. [2025]

    Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models.arXiv preprint arXiv:2506.05176(2025)

    Pith/arXiv arXiv 2025

  46. [3520]

    doi:10.18653/v1/2025.acl-long.176

    doi:10.18653/v1/2025.acl-long.176 2025