Pith. sign in

REVIEW 3 major objections 7 minor 72 references

Weak-to-Strong GraphRAG: Aligning Weak Retrievers with Large Language Models for Graph-based Retrieval Augmented Generation

T0 review · 3 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ReG, a new method for graph-based retrieval-augmented generation, aligns weak retrievers with LLM reasoning by having the LLM filter candidate reasoning paths and reorganizing retrieved evidence into coherent chains, achieving…

desk verdict ReG is a credible graph-RAG method with strong experiments, but the abstract overreaches and the candidate-pool coverage assumption needs a direct audit before the claims are taken as ground truth. read the letter →

arxiv 2506.22518 v1 pith:UGVT3ZBL submitted 2025-06-26 cs.CL cs.AI

Deyu Zou , Yongqiang Chen , Mufei Li , Siqi Miao , Chenxi Liu , Bo Han , James Cheng , Pan Li This is my paper
classification cs.CLcs.AI
keywords graph-basedretrieval-augmentedgenerationknowledgegraphquestionansweringweaksupervisionrefinementLLMfeedbackevidencechainreorganizationretrieveralignmentdataefficiencyzero-shotgeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the weakest link in graph-based retrieval-augmented generation is not the retriever's architecture but the quality of the supervision it is trained on and the way retrieved knowledge is presented to the LLM. To fix this, ReG uses the LLM itself as a filter: from a candidate pool built from shortest paths and one-hop neighborhoods, the LLM selects the reasoning chains that are actually needed to answer a question, and those chains become the training signal for the retriever. At inference time, retrieved triples are reorganized into logically connected evidence chains via breadth-first expansion, matching the way LLMs prefer to read. The paper claims this yields state-of-the-art results on three knowledge-graph QA benchmarks, matches the best baselines with only 5% of the training data, and cuts reasoning-token use by up to 30% when paired with large reasoning models. If true, it means weak retrievers can be made strong simply by giving the LLM a voice in what counts as evidence.

What carries the argument

The candidate path pool P := Psp ∪ Pq ∪ Pa is the central object: it defines the universe of evidence the LLM is allowed to judge, so everything downstream inherits its coverage. The LLM-guided refinement step uses in-context learning with explanation-based demonstrations to select a subset bP+ of logically coherent paths, whose triples become the training labels bG+. At inference, a structure-aware reorganization module performs BFS-guided chain expansion over the retrieved triples—starting from query-anchored triples and extending only through entity-matched links—plus structural merging for multi-answer and multi-entity paths, producing the ordered evidence chains fed to the LLM. The compression step keeps the candidate pool at about 5% of its raw size, making the LLM calls tractable.

What would settle it

Build a set of KGQA questions whose correct answer requires a reasoning chain that is neither a shortest path between the query entity and the answer entity nor within one hop of either, and verify by direct inspection that these chains are absent from the candidate pool P. If ReG's retriever, trained on LLM-refined supervision from that pool, systematically fails these questions while a retriever given the full KG succeeds, the coverage assumption is violated and the method's guarantee collapses.

Watch

Extended reading notes

Core claim

ReG treats graph-based RAG as a black-box combinatorial search for a minimal sufficient subgraph, shows that directly solving this search is intractable, and instead obtains high-quality supervision by prompting an LLM to pick plausible reasoning paths from a candidate pool P = Psp ∪ Pq ∪ Pa (shortest paths, query-centric and answer-centric one-hop neighborhoods). The LLM-selected paths form the refined supervision signal bG+ that trains the retriever. A second component, structure-aware reorganization, turns the retrieved subgraph into ordered evidence chains by BFS-guided expansion from query-anchored triples, with merging rules for redundant multi-answer and multi-entity paths. The paper's central claim is that this two-step alignment—LLM-refined supervision during training and structure-aware presentation at inference—is what lets weak retrievers match or beat much stronger baselines across WebQSP, CWQ, and GrailQA.

Load-bearing premise

The candidate pool built from shortest paths plus one-hop neighborhoods around query and answer entities must contain every piece of evidence needed to answer the question; if an essential reasoning step lies outside this pool, the LLM refiner can never select it and the retriever will be blind to it.

Editorial extensions

If this is right

  • On WebQSP-sub and CWQ-sub, ReG with GPT-4o-mini or GPT-4o outperforms all baselines on all four reported metrics (Macro-F1, Micro-F1, Hit, Hit@1).
  • Retraining with only 5% of the training data yields better Macro-F1 and Hit than the SubgraphRAG baseline trained with 80%, showing supervision quality dominates supervision quantity.
  • With QwQ-32B and DeepSeek-R1 reasoning models, ReG reduces average reasoning tokens by up to 30% while improving several accuracy metrics, indicating that cleaner, better-organized evidence curbs overthinking.
  • LLM-refined supervision transfers across backbone LLMs and improves zero-shot OOD generalization on GrailQA's unseen schemas.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The coverage assumption deserves direct measurement: one could take a sample of questions, annotate the oracle subgraphs, and compute what fraction of oracle triples falls inside P. The paper cites prior work for shortest-path recall but does not measure this on the benchmarks it uses.
  • The two components are logically separable: LLM-refined supervision improves the retriever's precision, while structure-aware reorganization improves the reasoner's use of correct retrievals. A natural test is applying each component independently to other graph-RAG pipelines to see which contributes more in different regimes.
  • The token-cost result suggests a cheap probe for reasoning quality: organize the same retrieved triples in random order versus evidence-chain order and measure both accuracy and reasoning-token count on the LRMs; the paper's claim predicts a measurable drop in tokens with no loss in accuracy for the reorganized form.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes ReG, a graph-based retrieval-augmented generation framework for KGQA. ReG first constructs a candidate path pool from query-answer shortest paths plus one-hop query- and answer-centric neighborhoods, then uses an LLM to select high-quality reasoning chains from that pool; the selected chains serve as refined supervision for training a retriever at triple, entity, or path granularity. At inference time, ReG reorganizes the retrieved triples into BFS-expanded evidence chains before presenting them to the reasoning LLM. The authors formulate graph-based RAG as black-box combinatorial optimization, prove a query-complexity lower bound, and report experiments on WebQSP, CWQ, and GrailQA across multiple LLMs, claiming state-of-the-art accuracy, data efficiency at 5% training data, OOD transferability, and reduced reasoning token cost with large reasoning models.

Significance. If the empirical claims hold, ReG is a useful contribution to graph-based RAG: it directly addresses the weak-supervision problem for graph retrievers, shows that LLM-refined signals transfer across different refinement and reasoning LLMs, and demonstrates efficiency gains with reasoning-focused models. The paper's strengths are its clear problem decomposition, the coverage of three retrieval granularities, the inclusion of OOD and LRM evaluations, and an ablation that separates the supervision-refinement and reorganization components. However, the central coverage assumption of the candidate pool is never audited, and the state-of-the-art claim is overstated relative to the reported tables. These issues are load-bearing for the main conclusions and need to be resolved before the contribution can be fully assessed.

major comments (3)
  1. [Sec. 4.1; Eq. (33)-(34); Table 1] The method's core assumption that the candidate pool P := Psp ∪ Pq ∪ Pa covers the oracle subgraph bG* is never audited. Because Eq. (33) and (34) train every triple outside bG+ as a negative, an oracle triple missing from P is not merely unretrievable; the retriever is explicitly trained to suppress it. Table 1 reports that the complexity-control step compresses the pool to about 5% of its original size, yet no experiment checks whether this compression preserves the evidence needed for test queries. Please add an oracle-coverage audit (e.g., the fraction of test questions whose gold evidence triples from the dataset logical forms are contained in P and in the compressed pool) and ablate the pool construction (e.g., adding two-hop neighborhoods) so that the coverage assumption is empirically grounded.
  2. [Sec. 5.2; Tables 2 and 3] The statement in Section 5.2 that 'all 12 metrics outperforming existing baselines' is not supported by a single consistent configuration. In Table 2, ReG@Triple (GPT-4o) obtains Micro-F1 57.88 on WebQSP-Sub, which is lower than SubgraphRAG (GPT-4o) at 58.91, and in Table 3 ReG@Triple (Llama3.1-8B) obtains Macro-F1 69.91 on WebQSP, which is lower than SubgraphRAG (Llama3.1-8B) at 70.57. If the claim refers to the best ReG variant for each metric, this must be stated explicitly and the per-variant results reported; as written, the claim overstates the evidence.
  3. [Sec. 5.2; Figure 4] Figure 4a is used to support the abstract-level claim that ReG 'matches the state-of-the-art performance with 5% training data,' but the figure reports no error bars or significance tests, and Section 5.1 fixes a single seed. Please report the underlying numbers and variance over at least three seeds for the 5% and 80% training ratios, and specify the number of LLM refinement calls and the token cost so the reader can assess the claimed computational savings.
minor comments (7)
  1. [Def. 3.1, Eq. (2)] Equation (2) uses 't ∈ (bG ∩ bG*)' where 'τ' is meant; please fix the variable.
  2. [Sec. 4.1] The final candidate pool is written as 'P := Psp ∪ Pa ∪ Pa'; this should read P := Psp ∪ Pq ∪ Pa.
  3. [Appendix D.4.1, Algorithm 1] Algorithm 1 defines bGsrc as triples with h ∈ Eq, whereas Section 4.2 defines it as h ∈ Eq ∨ t ∈ Eq; the algorithm and the text should match.
  4. [Figure 1] Figure 1 contains truncated labels (e.g., 'viewf', 'Optical') that make the pipeline difficult to follow; a higher-resolution figure is needed.
  5. [Table 6] Table 6 should define the intersection and union operations over bG+ (paths versus triples) and report how the union handles conflicting selections.
  6. [Table 4] The 'w/o S(I) & S(II)' row is left empty for the Path level; please explain why this variant is not applicable.
  7. [Appendix D.2] The answer-merging step says one representative answer a* is selected because answers share the same reasoning path, but the selection criterion is unspecified; please state it and discuss cases where distinct answers require distinct reasoning paths.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: ReG is evaluated on external QA accuracy rather than on its own LLM preferences; the only self-citation concern is the P-coverage premise, which is an empirical assumption, not a by-construction equivalence.

full rationale

ReG's derivation chain is not circular. The retriever is trained on LLM-selected evidence chains (Eqs. 33-35) but evaluated on external QA metrics (Macro-F1, Micro-F1, Hit, Hit@1, average reasoning tokens), not on the LLM refiner's own selections. The 5%-data result, OOD generalization, and LRM token-reduction results are measured outcomes, not quantities fitted into the method and then renamed as predictions. The single potentially load-bearing external premise is that the candidate pool P covers the oracle subgraph, justified by the statement in Sec. 4.1: "Since shortest paths provide high-recall coverage (Li et al., 2024a), we construct P based on Psp." Li et al. (2024a) shares authors with this paper, so this is a self-citation. However, the cited claim is an empirically falsifiable property of the KGQA datasets and is checked indirectly by ReG's end-to-end improvements over SubgraphRAG and other baselines; it is not an equation-level identity such as P = bG*. The paper also explicitly acknowledges in Sec. H that "The LLM-refined supervision signals may be not exactly the oracle one," confirming that bG+ is not claimed to equal bG* by construction. The theoretical intractability result (Prop. 3.1) is independent of the method's validity. Overall, the central claim is self-contained against external benchmarks; the score reflects one minor self-citation that is not load-bearing to the main SOTA claim.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central empirical claim rests on standard ML training plus several domain assumptions: coverage of the candidate pool, LLM reliability as a path selector, LLM sensitivity to ordering, and a scalar-reward abstraction for the theoretical bound. The free parameters are experimental hyperparameters chosen by hand; none enter a derivation. No invented entities are introduced.

free parameters (5)
  • Retrieval budget K (top-K triples) = 500 for GPT-4o and GPT-4o-mini; 200 for LLaMA-3.1-8B
    Chosen per reasoner capacity; directly controls how much evidence the LLM sees at inference and affects the balance between coverage and noise.
  • Max chain expansion length L = unlimited for CWQ; 2 for WebQSP and GrailQA
    The BFS chain expansion stops at L; setting L=2 for non-CWQ means longer coherent chains are not formed, potentially dropping multi-hop evidence beyond two hops; L is dataset-specific.
  • Beam width for path-level retriever = 20 for GPT-4o and GPT-4o-mini; 10 for LLaMA-3.1-8B
    The ReG@Path planner (LLaMA-2-7B) grounds predicted relation paths via beam search; beam width trades recall for inference cost.
  • Answer merging in candidate pool = one representative answer a* per query
    When merging the candidate pool, the paper keeps only one representative answer from Aq, assuming correct answers share the same reasoning path; this is an ad hoc reduction that could discard answer-specific evidence for multi-answer queries.
  • Retriever training epochs = 80 for MLP and GNN; 8 for LLM planner with LoRA
    Fixed iteration counts with best validation recall checkpoint selection; standard practice, but no sensitivity analysis is reported.
assumptions (5)
  • domain assumption Shortest paths between query entities and answer entities provide high-recall coverage of the oracle subgraph bG*.
    Used in Section 4.1 to construct the candidate pool P from Psp; cited to Li et al. (2024a) but not measured in this paper; if recall is lower than assumed, LLM refinement cannot recover missing paths.
  • domain assumption LLMs are sensitive to the structure and ordering of input facts, so presenting logically ordered chains improves reasoning.
    Grounds the structure-aware reorganization module in Section 3.3; supported by cited empirical studies (Chen et al., 2024b; Guo et al., 2025c), not re-derived.
  • domain assumption The LLM used for supervision refinement produces selections that better approximate the oracle subgraph bG* than weak heuristic supervision bGw.
    Central mechanism of ReG described in Section 4.1; the paper provides empirical validation but no formal guarantee, and the 'Theoretical Discussion' asserts accuracy without proof.
  • ad hoc to paper The black-box LLM evaluator reward r(S,q) can be abstracted by Eq. 1-2 with at most O(N) possible reward values.
    This modeling assumption underpins the information-theoretic lower bound in Proposition 3.1; it abstracts the content of LLM feedback to a scalar reward, which is a simplification.
  • standard math Fano's inequality and standard information-theoretic bounds are valid.
    Used in the proof of Proposition 3.1 in Appendix B.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Weak-to-Strong GraphRAG: Aligning Weak Retrievers with Large Language Models for Graph-based Retrieval Augmented Generation." pith.science (2026). https://pith.science/paper/UGVT3ZBL

@misc{pith2026250622518,
  author       = {Pith},
  title        = {Pith review of: Weak-to-Strong GraphRAG: Aligning Weak Retrievers with Large Language Models for Graph-based Retrieval Augmented Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UGVT3ZBL}},
  note         = {Machine review of arXiv:2506.22518}
}
read the original abstract

Graph-based retrieval-augmented generation (RAG) enables large language models (LLMs) to ground responses with structured external knowledge from up-to-date knowledge graphs (KGs) and reduce hallucinations. However, LLMs often rely on a weak retriever in graph-based RAG: I) Due to the lack of ground truth, the retriever is often trained on weak supervision, which often introduces spurious signals to the LLMs. II) Due to the abstraction of graph data, the retrieved knowledge is often presented in unorganized forms. To mitigate the issue, we present Refined Graph-based RAG (ReG) to align weak retrievers to LLMs for graph-based RAG. Specifically, ReG incorporates LLM feedback to get rid of spurious signals and improve the quality of the supervision. Meanwhile, ReG introduces a structure-aware reorganization module to refactor the retrieval results into logically coherent evidence chains. Experiments on prominent benchmarks demonstrate that ReG significantly and consistently brings improvements across different LLM backbones by up to 10%. The improved supervision quality enables ReG to match the state-of-the-art performance with 5% training data and to transfer to out-of-distribution KGs. Notably, when adopted to reasoning-based LLMs, ReG reduces the reasoning token cost by up to 30% and improves the performance by up to 4%.

Figures

Figures reproduced from arXiv: 2506.22518 by the authors.

Figure 1
Figure 1. Overall framework of ReG. Given a KG G with a query-answer pair (q, a), ReG first constructs a candidate path pool P to cover diverse candidate reasoning paths. Then, ReG uses LLM to select high-quality Gb+ for training retrievers. During inference, retrieved items are reorganized into logic-consistent chains to better align with the needs for LLM reasoning. information compared to text-based retrieval, particularly… view at source ↗
Figure 2
Figure 2. Scaling trends of: (a) Psp size versus the number of q-a pairs, and (b) the number of generated reasoning chains versus retrieved triples. To bridge the gap between Gb∗ and Gbw in a cost-efficient way, we first construct a candidate path pool P, to cover diverse reasoning patterns, and then leverages LLMs to identify high-quality supervision signals from the candidates. Multi-Faceted Candidate Generation. Since shor… view at source ↗
Figure 3
Figure 3. Illustration of BFS-guided chain expansion. To bridge the representation gap between retrieval and reasoning (Sec. 3.3), ReG also transforms the retrieved triples Gb into a list of coherent evidence chains. Essentially, as the KG is originally organized in a logically coherent form, we align the retrieved results following the order in the KG. Chain Expansion. Specifically, we perform BFS-guided chain ex￾pansion ove… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Performance comparison (Macro-F1 and Hit) between [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Ablations across three retrieval levels over [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Detailed prompt for LLM-based refinement used in our experiments. [PITH_FULL_IMAGE:figures/full_fig_p022_6.png]
Figure 7
Figure 7. Figure 7: Detailed prompt for downstream QA used in our experiments. [PITH_FULL_IMAGE:figures/full_fig_p023_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

72 extracted references · 19 canonical work pages

  1. [1]

    A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity

    Bang, Y., Cahyawijaya, S., Lee, N., Dai, W., Su, D., Wilie, B., Lovenia, H., Ji, Z., Yu, T., Chung, W., et al. A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.04023, 2023

    arXiv 2023

  2. [2]

    Freebase: a collaboratively created graph database for structuring human knowledge

    Bollacker, K., Evans, C., Paritosh, P., Sturge, T., and Taylor, J. Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pp.\ 1247--1250, 2008

    work page 2008

  3. [3]

    B., Lespiau, J.-B., Damoc, B., Clark, A., et al

    Borgeaud, S., Mensch, A., Hoffmann, J., Cai, T., Rutherford, E., Millican, K., Van Den Driessche, G. B., Lespiau, J.-B., Damoc, B., Clark, A., et al. Improving language models by retrieving from trillions of tokens. In International conference on machine learning, pp.\ 2206--2240. PMLR, 2022

    2022

  4. [4]

    D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al

    Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 0 1877--1901, 2020

    1901

  5. [5]

    and Mugnier, M.-L

    Chein, M. and Mugnier, M.-L. Graph-based knowledge representation: computational foundations of conceptual graphs. Springer Science & Business Media, 2008

    work page 2008

  6. [6]

    Pathrag: Pruning graph-based retrieval augmented generation with relational paths

    Chen, B., Guo, Z., Yang, Z., Chen, Y., Chen, J., Liu, Z., Shi, C., and Yang, C. Pathrag: Pruning graph-based retrieval augmented generation with relational paths. arXiv preprint arXiv:2502.14902, 2025

    arXiv 2025

  7. [7]

    Benchmarking large language models in retrieval-augmented generation

    Chen, J., Lin, H., Han, X., and Sun, L. Benchmarking large language models in retrieval-augmented generation. In Proceedings of the AAAI Conference on Artificial Intelligence, pp.\ 17754--17762, 2024 a

    work page 2024

  8. [8]

    A., Wang, X., and Zhou, D

    Chen, X., Chi, R. A., Wang, X., and Zhou, D. Premise order matters in reasoning with large language models. arXiv preprint arXiv:2402.08939, 2024 b

    arXiv 2024

Show all 72 references
  1. [9]

    Do not think that much for 2+ 3=? on the overthinking of o1-like llms

    Chen, X., Xu, J., Liang, T., He, Z., Pang, J., Yu, D., Song, L., Liu, Q., Zhou, M., Zhang, Z., et al. Do not think that much for 2+ 3=? on the overthinking of o1-like llms. arXiv preprint arXiv:2412.21187, 2024 c

    2024 arXiv

  2. [10]

    Principal neighbourhood aggregation for graph nets

    Corso, G., Cavalleri, L., Beaini, D., Li \`o , P., and Veli c kovi \'c , P. Principal neighbourhood aggregation for graph nets. Advances in neural information processing systems, 33: 0 13260--13271, 2020

    2020

  3. [11]

    R., Eisenschlos, J

    Dhingra, B., Cole, J. R., Eisenschlos, J. M., Gillick, D., Eisenstein, J., and Cohen, W. W. Time-aware language models as temporal knowledge bases. Transactions of the Association for Computational Linguistics, 10: 0 257--273, 2022

    2022

  4. [12]

    O., and Larson, J

    Edge, D., Trinh, H., Cheng, N., Bradley, J., Chao, A., Mody, A., Truitt, S., Metropolitansky, D., Ness, R. O., and Larson, J. From local to global: A graph rag approach to query-focused summarization. arXiv preprint arXiv:2404.16130, 2024

    2024 arXiv

  5. [13]

    A survey on rag meeting llms: Towards retrieval-augmented large language models

    Fan, W., Ding, Y., Ning, L., Wang, S., Li, H., Yin, D., Chua, T.-S., and Li, Q. A survey on rag meeting llms: Towards retrieval-augmented large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.\ 6491--6501, 2024

    2024

  6. [14]

    Efficient reasoning models: A survey

    Feng, S., Fang, G., Ma, X., and Wang, X. Efficient reasoning models: A survey. arXiv preprint arXiv:2504.10903, 2025

    2025

  7. [15]

    Retrieval-augmented generation for large language models: A survey

    Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, H., and Wang, H. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997, 2, 2023

    2023 arXiv

  8. [16]

    Beyond iid: three levels of generalization for question answering on knowledge bases

    Gu, Y., Kase, S., Vanni, M., Sadler, B., Liang, P., Yan, X., and Su, Y. Beyond iid: three levels of generalization for question answering on knowledge bases. In Proceedings of the Web Conference 2021, pp.\ 3477--3488, 2021

    2021

  9. [17]

    Don't generate, discriminate: A proposal for grounding language models to real-world environments

    Gu, Y., Deng, X., and Su, Y. Don't generate, discriminate: A proposal for grounding language models to real-world environments. arXiv preprint arXiv:2212.09736, 2022

    2022 arXiv

  10. [18]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025 a

    2025 arXiv

  11. [19]

    Empowering graphrag with knowledge filtering and integration

    Guo, K., Shomer, H., Zeng, S., Han, H., Wang, Y., and Tang, J. Empowering graphrag with knowledge filtering and integration. arXiv preprint arXiv:2503.13804, 2025 b

    2025 arXiv

  12. [20]

    I., and Russell, S

    Guo, T., Zhu, H., Zhang, R., Jiao, J., Mei, S., Jordan, M. I., and Russell, S. How do llms perform two-hop reasoning in context? arXiv preprint arXiv:2502.13913, 2025 c

    2025 arXiv

  13. [21]

    Lightrag: Simple and fast retrieval-augmented generation

    Guo, Z., Xia, L., Yu, Y., Ao, T., and Huang, C. Lightrag: Simple and fast retrieval-augmented generation. arXiv preprint arXiv:2410.05779, 2024

    2024 arXiv

  14. [22]

    J., Shu, Y., Gu, Y., Yasunaga, M., and Su, Y

    Guti \'e rrez, B. J., Shu, Y., Gu, Y., Yasunaga, M., and Su, Y. Hipporag: Neurobiologically inspired long-term memory for large language models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  15. [23]

    J., Shu, Y., Qi, W., Zhou, S., and Su, Y

    Guti \'e rrez, B. J., Shu, Y., Qi, W., Zhou, S., and Su, Y. From rag to memory: Non-parametric continual learning for large language models. arXiv preprint arXiv:2502.14802, 2025

    2025 arXiv

  16. [24]

    A., Mukherjee, S., Tang, X., et al

    Han, H., Wang, Y., Shomer, H., Guo, K., Ding, J., Lei, Y., Halappanavar, M., Rossi, R. A., Mukherjee, S., Tang, X., et al. Retrieval-augmented generation with graphs (graphrag). arXiv preprint arXiv:2501.00309, 2024

    2024 arXiv

  17. [25]

    Gasket RAG : Systematic alignment of large language models with retrievers, 2025

    Han, W., Fang, M., Li, Y., and Pechenizkiy, M. Gasket RAG : Systematic alignment of large language models with retrievers, 2025. URL https://openreview.net/forum?id=TqLY7QoELU

    2025

  18. [26]

    G-retriever: Retrieval-augmented generation for textual graph understanding and question answering

    He, X., Tian, Y., Sun, Y., Chawla, N., Laurent, T., LeCun, Y., Bresson, X., and Hooi, B. G-retriever: Retrieval-augmented generation for textual graph understanding and question answering. Advances in Neural Information Processing Systems, 37: 0 132876--132907, 2024

    2024

  19. [27]

    Ruler: What’s the real context size of your long-context language models?, 2024

    Hsieh, C.-P., Sun, S., Kriman, S., Acharya, S., Rekesh, D., Jia, F., Zhang, Y., and Ginsburg, B. Ruler: What’s the real context size of your long-context language models?, 2024. URL https://arxiv. org/abs/2404.06654, 2024

    2024 arXiv

  20. [28]

    J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al

    Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al. Lora: Low-rank adaptation of large language models. ICLR, 1 0 (2): 0 3, 2022

    2022

  21. [29]

    A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions

    Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems, 43 0 (2): 0 1--55, 2025

    2025

  22. [30]

    Advancing transformer architecture in long-context large language models: A comprehensive survey

    Huang, Y., Xu, J., Lai, J., Jiang, Z., Chen, T., Li, Z., Yao, Y., Ma, X., Yang, L., Chen, H., et al. Advancing transformer architecture in long-context large language models: A comprehensive survey. arXiv preprint arXiv:2311.12351, 2023

    2023 arXiv

  23. [31]

    J., Madotto, A., and Fung, P

    Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., and Fung, P. Survey of hallucination in natural language generation. ACM computing surveys, 55 0 (12): 0 1--38, 2023

    2023

  24. [32]

    X., and Wen, J.-R

    Jiang, J., Zhou, K., Zhao, W. X., and Wen, J.-R. Unikgqa: Unified retrieval and reasoning for solving multi-hop question answering over knowledge graph. arXiv preprint arXiv:2212.00959, 2022

    2022 arXiv

  25. [33]

    X., and Wen, J.-R

    Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W. X., and Wen, J.-R. Structgpt: A general framework for large language model to reason over structured data. arXiv preprint arXiv:2305.09645, 2023

    2023 arXiv

  26. [34]

    Jin, B., Yoon, J., Han, J., and Arik, S. O. Long-context llms meet rag: Overcoming challenges for long inputs in rag. arXiv preprint arXiv:2410.05983, 2024

    2024 arXiv

  27. [35]

    A., Choi, Y., Inui, K., et al

    Kasai, J., Sakaguchi, K., Le Bras, R., Asai, A., Yu, X., Radev, D., Smith, N. A., Choi, Y., Inui, K., et al. Realtime qa: What's the answer right now? Advances in neural information processing systems, 36: 0 49025--49043, 2023

    2023

  28. [36]

    Simple is effective: The roles of graphs and large language models in knowledge-graph-based retrieval-augmented generation

    Li, M., Miao, S., and Li, P. Simple is effective: The roles of graphs and large language models in knowledge-graph-based retrieval-augmented generation. arXiv preprint arXiv:2410.20724, 2024 a

    2024 arXiv

  29. [37]

    Are chatgpt and gpt-4 general-purpose solvers for financial text analytics? a study on several typical tasks

    Li, X., Chan, S., Zhu, X., Pei, Y., Ma, Z., Liu, X., and Shah, S. Are chatgpt and gpt-4 general-purpose solvers for financial text analytics? a study on several typical tasks. arXiv preprint arXiv:2305.05862, 2023 a

    2023 arXiv

  30. [38]

    Rag-ddr: Optimizing retrieval-augmented generation using differentiable data rewards

    Li, X., Mei, S., Liu, Z., Yan, Y., Wang, S., Yu, S., Zeng, Z., Chen, H., Yu, G., Liu, Z., et al. Rag-ddr: Optimizing retrieval-augmented generation using differentiable data rewards. arXiv preprint arXiv:2410.13509, 2024 b

    2024 arXiv

  31. [39]

    Towards general text embeddings with multi-stage contrastive learning

    Li, Z., Zhang, X., Zhang, Y., Long, D., Xie, P., and Zhang, M. Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.03281, 2023 b

    2023 arXiv

  32. [40]

    Explore then determine: A gnn-llm synergy framework for reasoning over knowledge graph

    Liu, G., Zhang, Y., Li, Y., and Yao, Q. Explore then determine: A gnn-llm synergy framework for reasoning over knowledge graph. arXiv preprint arXiv:2406.01145, 2024 a

    2024 arXiv

  33. [41]

    A survey on hallucination in large vision-language models

    Liu, H., Xue, W., Chen, Y., Chen, D., Zhao, X., Wang, K., Hou, L., Li, R., and Peng, W. A survey on hallucination in large vision-language models. arXiv preprint arXiv:2402.00253, 2024 b

    2024 arXiv

  34. [42]

    A comprehensive survey on long context language modeling

    Liu, J., Zhu, D., Bai, Z., He, Y., Liao, H., Que, H., Wang, Z., Zhang, C., Zhang, G., Zhang, J., et al. A comprehensive survey on long context language modeling. arXiv preprint arXiv:2503.17407, 2025

    2025

  35. [43]

    F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., and Liang, P

    Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., and Liang, P. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12: 0 157--173, 2024 c

    2024

  36. [44]

    A., et al

    Luo, H., Guo, Y., Lin, Q., Wu, X., Mu, X., Liu, W., Song, M., Zhu, Y., Tuan, L. A., et al. Kbqa-o1: Agentic knowledge base question answering with monte carlo tree search. arXiv preprint arXiv:2501.18922, 2025 a

    2025 arXiv

  37. [45]

    Reasoning on graphs: Faithful and interpretable large language model reasoning

    Luo, L., Li, Y.-F., Haffari, G., and Pan, S. Reasoning on graphs: Faithful and interpretable large language model reasoning. arXiv preprint arXiv:2310.01061, 2023

    2023 arXiv

  38. [46]

    Gfm-rag: Graph foundation model for retrieval augmented generation

    Luo, L., Zhao, Z., Haffari, G., Phung, D., Gong, C., and Pan, S. Gfm-rag: Graph foundation model for retrieval augmented generation. arXiv preprint arXiv:2502.01113, 2025 b

    2025

  39. [47]

    and Karypis, G

    Mavromatis, C. and Karypis, G. Gnn-rag: Graph neural retrieval for large language model reasoning. arXiv preprint arXiv:2405.20139, 2024

    2024 arXiv

  40. [48]

    Mteb: Massive text embedding benchmark

    Muennighoff, N., Tazi, N., Magne, L., and Reimers, N. Mteb: Massive text embedding benchmark. arXiv preprint arXiv:2210.07316, 2022

    2022 arXiv

  41. [49]

    Openai o3-mini

    OpenAI. Openai o3-mini. https://openai.com/index/openai-o3-mini/, January 2025. [Online]

    2025

  42. [50]

    Graph retrieval-augmented generation: A survey

    Peng, B., Zhu, Y., Liu, Y., Bo, X., Shi, H., Hong, C., Zhang, Y., and Tang, S. Graph retrieval-augmented generation: A survey. arXiv preprint arXiv:2408.08921, 2024

    2024 arXiv

  43. [51]

    Raiaan, M. A. K., Mukta, M. S. H., Fatema, K., Fahad, N. M., Sakib, S., Mim, M. M. J., Ahmad, J., Ali, M. E., and Azam, S. A review on large language models: Architectures, applications, taxonomies, open issues and challenges. IEEE access, 12: 0 26839--26874, 2024

    2024

  44. [52]

    O'Reilly Media, Inc

    Robinson, I., Webber, J., and Eifrem, E. Graph databases: new opportunities for connected data. " O'Reilly Media, Inc.", 2015

    2015

  45. [53]

    Replug: Retrieval-augmented black-box language models

    Shi, W., Min, S., Yasunaga, M., Seo, M., James, R., Lewis, M., Zettlemoyer, L., and Yih, W.-t. Replug: Retrieval-augmented black-box language models. arXiv preprint arXiv:2301.12652, 2023

    2023 arXiv

  46. [54]

    Stop overthinking: A survey on efficient reasoning for large language models

    Sui, Y., Chuang, Y.-N., Wang, G., Zhang, J., Zhang, T., Yuan, J., Liu, H., Wen, A., Chen, H., Hu, X., et al. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503.16419, 2025

    2025 arXiv

  47. [55]

    M., Shum, H.-Y., and Guo, J

    Sun, J., Xu, C., Tang, L., Wang, S., Lin, C., Gong, Y., Ni, L. M., Shum, H.-Y., and Guo, J. Think-on-graph: Deep and responsible reasoning of large language model on knowledge graph. arXiv preprint arXiv:2307.07697, 2023

    2023 arXiv

  48. [56]

    and Berant, J

    Talmor, A. and Berant, J. The web as a knowledge-base for answering complex questions. arXiv preprint arXiv:1803.06643, 2018

    2018 arXiv

  49. [57]

    I., Burnell, R., Bai, L., Gulati, A., Tanzer, G., Vincent, D., Pan, Z., Wang, S., et al

    Team, G., Georgiev, P., Lei, V. I., Burnell, R., Bai, L., Gulati, A., Tanzer, G., Vincent, D., Pan, Z., Wang, S., et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530, 2024

    2024 arXiv

  50. [58]

    and Kr \"o tzsch, M

    Vrande c i \'c , D. and Kr \"o tzsch, M. Wikidata: a free collaborative knowledgebase. Communications of the ACM, 57 0 (10): 0 78--85, 2014

    2014

  51. [59]

    Knowledge-driven cot: Exploring faithful reasoning in llms for knowledge-intensive question answering

    Wang, K., Duan, F., Wang, S., Li, P., Xian, Y., Yin, C., Rong, W., and Xiong, Z. Knowledge-driven cot: Exploring faithful reasoning in llms for knowledge-intensive question answering. arXiv preprint arXiv:2308.13259, 2023 a

    2023 arXiv

  52. [60]

    R., and Neubig, G

    Wang, Z., Araki, J., Jiang, Z., Parvez, M. R., and Neubig, G. Learning to filter context for retrieval-augmented generation. arXiv preprint arXiv:2311.08377, 2023 b

    2023 arXiv

  53. [61]

    How easily do irrelevant inputs skew the responses of large language models? arXiv preprint arXiv:2404.03302, 2024

    Wu, S., Xie, J., Chen, J., Zhu, T., Zhang, K., and Xiao, Y. How easily do irrelevant inputs skew the responses of large language models? arXiv preprint arXiv:2404.03302, 2024

    2024 arXiv

  54. [62]

    Efficient streaming language models with attention sinks

    Xiao, G., Tian, Y., Chen, B., Han, S., and Lewis, M. Efficient streaming language models with attention sinks. arXiv preprint arXiv:2309.17453, 2023

    2023 arXiv

  55. [63]

    Interactive-kbqa: Multi-turn interactions for knowledge base question answering with large language models

    Xiong, G., Bao, J., and Zhao, W. Interactive-kbqa: Multi-turn interactions for knowledge base question answering with large language models. arXiv preprint arXiv:2402.15131, 2024

    2024 arXiv

  56. [64]

    Harnessing the power of llms in practice: A survey on chatgpt and beyond

    Yang, J., Jin, H., Tang, R., Han, X., Feng, Q., Jiang, H., Zhong, S., Yin, B., and Hu, X. Harnessing the power of llms in practice: A survey on chatgpt and beyond. ACM Transactions on Knowledge Discovery from Data, 18 0 (6): 0 1--32, 2024

    2024

  57. [65]

    Ape: Faster and longer context-augmented generation via adaptive parallel encoding

    Yang, X., Chen, T., and Chen, B. Ape: Faster and longer context-augmented generation via adaptive parallel encoding. arXiv preprint arXiv:2502.05431, 2025

    2025 arXiv

  58. [66]

    The value of semantic parse labeling for knowledge base question answering

    Yih, W.-t., Richardson, M., Meek, C., Chang, M.-W., and Suh, J. The value of semantic parse labeling for knowledge base question answering. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp.\ 201--206, 2016

    2016

  59. [67]

    Making retrieval-augmented language models robust to irrelevant context

    Yoran, O., Wolfson, T., Ram, O., and Berant, J. Making retrieval-augmented language models robust to irrelevant context. arXiv preprint arXiv:2310.01558, 2023

    2023 arXiv

  60. [68]

    Rankrag: Unifying context ranking with retrieval-augmented generation in llms

    Yu, Y., Ping, W., Liu, Z., Wang, B., You, J., Zhang, C., Shoeybi, M., and Catanzaro, B. Rankrag: Unifying context ranking with retrieval-augmented generation in llms. Advances in Neural Information Processing Systems, 37: 0 121156--121184, 2024

    2024

  61. [69]

    Subgraph retrieval enhanced model for multi-hop knowledge base question answering

    Zhang, J., Zhang, X., Yu, J., Tang, J., Tang, J., Li, C., and Chen, H. Subgraph retrieval enhanced model for multi-hop knowledge base question answering. arXiv preprint arXiv:2202.13296, 2022

    2022 arXiv

  62. [70]

    G., Jain, N., Shen, S., Zaharia, M., Stoica, I., and Gonzalez, J

    Zhang, T., Patil, S. G., Jain, N., Shen, S., Zaharia, M., Stoica, I., and Gonzalez, J. E. Raft: Adapting language model to domain specific rag. arXiv preprint arXiv:2403.10131, 2024

    2024 arXiv

  63. [71]

    Length extrapolation of transformers: A survey from the perspective of positional encoding

    Zhao, L., Feng, X., Feng, X., Zhong, W., Xu, D., Yang, Q., Liu, H., Qin, B., and Liu, T. Length extrapolation of transformers: A survey from the perspective of positional encoding. arXiv preprint arXiv:2312.17044, 2023

    2023 arXiv

  64. [72]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.