Pith. sign in

REVIEW 4 major objections 6 minor 42 references

A Query-Aware Multi-Path Knowledge Graph Fusion Approach for Enhancing Retrieval-Augmented Generation in Large Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A query-aware knowledge-graph fusion method for RAG reports the best scores on all five QA benchmarks it tests, including a 9.72-point gain over reranking on HotpotQA.

desk verdict A plausible engineering contribution with a real reproducibility hole: the fusion threshold tau is never defined, so the headline gains cannot be verified as written. read the letter →

arxiv 2507.16826 v1 pith:E2YSARJK submitted 2025-07-07 cs.IR cs.AIcs.CL

Qikai Wei , Huansheng Ning , Chunlong Han , Jianguo Ding This is my paper
classification cs.IRcs.AIcs.CL
keywords RetrievalAugmentedGenerationKnowledgeGraphQuery-AwareAttentionRewardModelSubgraphFusionQueryExpansionMulti-HopQuestionAnsweringLargeLanguageModels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes QMKGF, a retrieval-augmented generation method that builds a knowledge graph from the document collection and expands the user query with entities, relations, and triples drawn from three kinds of graph subgraphs: one-hop neighbors, multi-hop paths, and importance-ranked nodes. The central claim is that fusing these subgraphs under the guidance of a query-aware reward model produces more relevant retrieved passages and better answers than standard embedding retrieval, reranking, single-path graph methods, or unfiltered graph fusion. On HotpotQA the method reports a ROUGE-1 score of 64.98%, a gain of 9.72 percentage points over the reranking baseline, and it reports the best scores among the compared methods across all five datasets. The underlying insight is that connections between retrieved pieces of evidence carry information that isolated text chunks miss, and that a query-sensitive selection step can keep those connections while filtering noise.

What carries the argument

The load-bearing object is the multi-path subgraph construction and the query-aware attention reward model that scores and fuses the subgraphs. The reward model is a small trained head on a pretrained encoder that takes query and subgraph representations, applies multi-head attention with the query as the query vector, and outputs a scalar score; training data come from large-language-model judgments of each subgraph's KG richness, question relevance, and connectivity. Fusion uses the cosine similarity between the top subgraph and the query as a reference threshold and keeps triples from lower-scoring subgraphs whose similarity to the query meets that threshold, producing $G_{fusion}$. This mechanism converts graph structure into query-specific context before any document retrieval happens.

What would settle it

On HotpotQA, replace the reward model with a random subgraph selector and run the identical pipeline; if ROUGE-1 stays near 64.98%, the query-aware fusion is not responsible for the reported gain.

Watch

Extended reading notes

Core claim

QMKGF's central claim is that answer quality in RAG improves when retrieval is driven by a fused knowledge-graph subgraph rather than by isolated chunks or any single graph path. Entities and relations are extracted from the document collection with prompts to a large language model, producing a KG and an entity vector database. For a query, mapped entities anchor three subgraphs: top-$K$ one-hop neighbours, expansions through the two most relevant neighbours to second hop, and top-$K$ nodes by personalized PageRank. A query-aware attention reward model scores each subgraph's triples by semantic relevance to the query; the highest-scoring subgraph forms the backbone, and triples from the other subgraphs whose similarity to the query clears a threshold are added. The fused subgraph's entities, relations, and triples are concatenated with the original query to retrieve and rerank document chunks, which are then passed to the LLM. The paper claims this consistently outperforms all baselines on SQuAD, IIRC, Cultour, HotpotQA, and MuSiQue, with the largest gain on HotpotQA (64.98% ROUGE-1 versus 55.26% for the reranking baseline).

Load-bearing premise

The whole method assumes that the reward model, trained on large-language-model judgments of graph richness, question relevance, and connectivity, correctly identifies which subgraphs and triples improve the final generated answer; the paper does not test that correlation.

Editorial extensions

If this is right

  • Multi-hop reasoning benefits most: the reported gains on HotpotQA, IIRC, and MuSiQue are larger than on single-hop SQuAD, suggesting fused multi-path context helps cross-document inference.
  • Query expansion with KG triples improves retrieval recall: the fine-tuned embedding experiments and the 99.4% hit@10 on Cultour indicate that the expanded query retrieves a wider, more relevant document set.
  • Reward-model fusion beats both fusing all triples and fusing a fixed top-5, so query-aware filtering appears to remove noise rather than merely add more context.
  • Subgraph size matters: $K=10$ nodes outperforms $K=5$ and $K=20$, indicating a balance between semantic coverage and redundancy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the large-language-model judgments used to train the reward model are valid proxies for answer quality, the same query-aware fusion could be applied at sentence or paragraph level inside a document, not just on KG triples, to reduce noise in long-context retrieval.
  • A testable extension is to make the fusion threshold in Eq. (12) adaptive to the query instead of fixed, since the reported method does not specify how the threshold is set.
  • The approach implies the extracted KG's quality is the ceiling: on noisy or poorly extracted relations, the reward model's filtering becomes more influential, so robustness to extraction error is a natural stress test.
  • The method might reduce the context-window pressure in long-document QA by replacing many raw chunks with a compact, query-aligned subgraph; this is not tested in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes QMKGF, a retrieval-augmented generation framework that constructs a knowledge graph from unstructured documents via LLM-based entity and relation extraction, builds three types of query-anchored subgraphs (one-hop, multi-hop, and personalized-PageRank importance-based), scores these subgraphs with a query-aware attention reward model, fuses high-relevance triples from lower-scoring subgraphs into the highest-scoring one, and uses the fused subgraph to expand the query before reranking retrieved chunks. The method is evaluated on SQuAD, IIRC, Cultour, HotpotQA, and MuSiQue against several baselines. The central empirical claim is that QMKGF consistently outperforms all baselines across all datasets, with the headline result being a HotpotQA ROUGE-1 score of 64.98% versus 55.26% for BGE-Rerank. The paper also reports ablations on the attention mechanism, embedding fine-tuning, fusion strategy, subgraph size, reward-model backbone, and number of attention heads.

Significance. If the reported results are reproducible and statistically robust, the contribution is useful: the multi-path subgraph construction with reward-model-guided fusion is a nontrivial extension of graph-based RAG, and the evaluation covers five datasets with several non-tautological lexical metrics (ROUGE, BLEU, METEOR). The ablations in Tables 4 and 7 are directionally consistent with the claimed contributions. However, reproducibility is currently blocked by an undefined fusion threshold in Eq. (12) and by underspecified reward-model training data, and the absence of error bars or significance tests leaves the headline improvement of 9.72 points statistically ungrounded. These are load-bearing issues for the paper's central claim, but they appear fixable within the manuscript's scope.

major comments (4)
  1. [Section 3.3, Eqs. (10)-(12), and Algorithm line 16] The fusion rule is not fully specified: Eq. (10) defines r = cos(KGS_max, q), but Eq. (12) and Algorithm line 16 filter triples by sim((h,r,t), q) >= tau, and tau is never defined or related to r. The value of tau controls which triples from lower-scoring subgraphs enter G_fusion, and Table 7 shows that the fusion strategy materially changes results (RM fusion 64.98 vs. All fusion 64.07 ROUGE-1 on HotpotQA). As written, an independent implementation cannot decide which triples to retain, and the headline gain may depend on an undisclosed threshold. The authors must specify the value or selection procedure for tau, state whether it was tuned on a validation split, and report its chosen value.
  2. [Section 3.3, reward model training] The reward model is central to selecting KGS_max and to the fusion step, but the training data are described only as 'scores assigned by LLMs' in terms of KG richness, question relevance, and connectivity. The manuscript does not report the dataset size, the prompt template, the number of LLMs, the score scale, or any agreement/quality checks on these labels. This is a reproducibility gap for a load-bearing component. The authors should also provide the final number of attention heads h used in the RM, since Fig. 6 varies h but no single final value is stated in Section 4.3.
  3. [Section 5.6-5.7 and Tables 2-3] Hyperparameters K and the RM attention-head count appear to be selected on the test sets: Fig. 5 reports optimal K=10 and Fig. 6 reports performance for different head counts, apparently on the evaluation subsets. With no validation split, no error bars, and no significance tests in Tables 2-3, the claim that QMKGF 'consistently outperforms all baseline models' is not statistically supported. The authors should report multiple runs with standard deviations, significance tests where appropriate, and a hyperparameter-selection protocol that does not use the test labels.
  4. [Section 5.2 and Table 4] The ablation text states that removing personalized PageRank has a notable negative impact on English datasets, but Table 4 contains no row for 'w/o PageRank' — only w/o-attention and w/o-fintune. Either the missing ablation must be added or the claim removed, since the current text makes an assertion that the reported experiments do not support.
minor comments (6)
  1. [Section 3.1 and Section 3.2] The text refers to 'Fig. 1' for the prompt templates and the subgraph-construction process, but the actual figures are Fig. 3 and Fig. 2 respectively; these cross-references should be corrected.
  2. [Figure 7] The caption of Fig. 7 reads 'Effect of subgraph node count,' but Section 5.9 describes it as analyzing the effect of entity quantity in queries; the caption should match the content.
  3. [Abstract and Section 4.1] The dataset is called 'Culture' in the abstract and conclusion but 'Cultour' in Section 4.1 and Tables 2 and 5; one consistent name should be used throughout.
  4. [Section 5.2] The sentence 'the improvement is less pronounced compared to IIRC' is confusing because Table 4 reports only HotpotQA and Cultour; it should probably refer to Cultour.
  5. [Throughout] There are several typos and formatting issues, including 'docment' in Eq. (1), the stray '♪' in reference [26], and inconsistent capitalization of 'BGE-rerank' vs. 'bge-rerank'; these should be cleaned up.
  6. [General] The manuscript does not include a code or data availability statement, which would be valuable for a method whose reproducibility currently depends on several unspecified implementation details.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: QMKGF's gains are empirical and benchmarked; the underspecified fusion threshold is a reproducibility issue, not a circular reduction.

full rationale

The paper's central claim—that QMKGF improves ROUGE, BLEU, and METEOR on five benchmarks—is not derived from its inputs by construction. The reward model in Sec. 3.3 is trained on LLM-assigned quality scores, but the evaluation metrics are independent lexical-overlap measures against gold answers, so no fitted parameter is renamed as a prediction. The fusion threshold tau in Eq. (12) and Algorithm line 16 is undefined and not explicitly linked to r in Eq. (10); this is a reproducibility gap, not a circular reduction, since the final metric does not reduce to that threshold. Hyperparameter explorations (subgraph size K, head count) are performed on the evaluation sets, which raises methodological concerns about selection on test data, but this is not circularity. Self-citations such as [28] and [29] are prior-work baselines and related methods, not load-bearing justifications for QMKGF's superiority. No equation or algorithmic step in the paper is equivalent to the reported results by definition.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The framework leans on five unvalidated domain assumptions: LLM KG extraction fidelity, entity mapping via cosine similarity, PageRank as query-relevance signal, LLM scores as reward training signal, and query expansion effectiveness. The free parameter list shows the pipeline has many hand-set or test-tuned numbers, and the fusion threshold tau is undefined.

free parameters (6)
  • K (subgraph node count) = 10 (chosen from test-set experiments, Fig. 5)
    Controls the size of each multi-path subgraph; selected by comparing K=5,10,20 on the Cultour test set.
  • tau (fusion threshold) = Not defined; possibly r = cos(KGS_max, q)
    Determines which triples from lower-scoring subgraphs are retained in fusion (Eq. 12); the paper provides no value.
  • Attention heads h in reward model = Not fixed; 32/64 for HotpotQA, moderate elsewhere
    Head count is varied per dataset and the best performer is chosen (Fig. 6), so it is tuned on test data.
  • Temperature m in embedding contrastive loss = Not reported
    Scaling parameter in Eq. (1) for embedding fine-tuning; its value is never stated.
  • Number of one-hop seeds for multi-hop expansion = 2
    Section 3.2 selects the two most relevant one-hop neighbors to expand further; this choice is unexamined.
  • Top-k chunks after reranking = Not reported
    Number of reranked chunks fed to the LLM (Eq. 14, Eq. 15) is not specified, and it directly affects output quality.
assumptions (5)
  • domain assumption LLM prompt-based extraction faithfully produces entities and relations from document chunks (Section 3.1).
    Errors in the KG propagate to all subgraphs and to query expansion; the paper does not evaluate extraction accuracy.
  • domain assumption Cosine similarity in fine-tuned embedding space correctly matches query entities to KG entities (Eq. 2).
    The whole pipeline depends on correct entity mapping; no manual inspection or accuracy numbers are reported.
  • domain assumption Personalized PageRank importance scores are a valid proxy for query-relevant knowledge (Eq. 5).
    The importance-based subgraph is one of three competing paths; its usefulness is assumed rather than derived.
  • domain assumption LLM-assigned scores for KG richness, relevance, and connectivity are a valid training signal for the reward model (Section 3.3).
    The RM's selection and fusion decisions rely on this proxy; no evidence links LLM scores to answer quality.
  • domain assumption Concatenating the query with entities, relations, and triples improves vector-database retrieval (Section 3.4).
    Query expansion is never ablated by itself against plain query retrieval, so its marginal benefit is unmeasured.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Query-Aware Multi-Path Knowledge Graph Fusion Approach for Enhancing Retrieval-Augmented Generation in Large Language Models." pith.science (2026). https://pith.science/paper/E2YSARJK

@misc{pith2026250716826,
  author       = {Pith},
  title        = {Pith review of: A Query-Aware Multi-Path Knowledge Graph Fusion Approach for Enhancing Retrieval-Augmented Generation in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E2YSARJK}},
  note         = {Machine review of arXiv:2507.16826}
}
read the original abstract

Retrieval Augmented Generation (RAG) has gradually emerged as a promising paradigm for enhancing the accuracy and factual consistency of content generated by large language models (LLMs). However, existing RAG studies primarily focus on retrieving isolated segments using similarity-based matching methods, while overlooking the intrinsic connections between them. This limitation hampers performance in RAG tasks. To address this, we propose QMKGF, a Query-Aware Multi-Path Knowledge Graph Fusion Approach for Enhancing Retrieval Augmented Generation. First, we design prompt templates and employ general-purpose LLMs to extract entities and relations, thereby generating a knowledge graph (KG) efficiently. Based on the constructed KG, we introduce a multi-path subgraph construction strategy that incorporates one-hop relations, multi-hop relations, and importance-based relations, aiming to improve the semantic relevance between the retrieved documents and the user query. Subsequently, we designed a query-aware attention reward model that scores subgraph triples based on their semantic relevance to the query. Then, we select the highest score subgraph and enrich subgraph with additional triples from other subgraphs that are highly semantically relevant to the query. Finally, the entities, relations, and triples within the updated subgraph are utilised to expand the original query, thereby enhancing its semantic representation and improving the quality of LLMs' generation. We evaluate QMKGF on the SQuAD, IIRC, Culture, HotpotQA, and MuSiQue datasets. On the HotpotQA dataset, our method achieves a ROUGE-1 score of 64.98\%, surpassing the BGE-Rerank approach by 9.72 percentage points (from 55.26\% to 64.98\%). Experimental results demonstrate the effectiveness and superiority of the QMKGF approach.

Figures

Figures reproduced from arXiv: 2507.16826 by the authors.

Figure 1
Figure 1. Comparison between LLM-only and RAG-based paradigm. any external retrieval. The red line represents the RAG￾based approach, where a user query is used to retrieve relevant chunks from a document corpus, guiding the LLM to generate more informed responses. The blue line denotes the final response delivered to the user by the LLM. Existing RAG methods can be divided into document retrieval-based and graph structure-ba… view at source ↗
Figure 1
Figure 1. Subsequently, we refined the embedding representations to improve their capacity for semantic comprehension and entity correspondence. The training dataset was constructed in the format {"query": str, "pos": List[str], "neg": List[str]}, where "pos" refers to the relevant answers, and "neg" in￾cludes irrelevant samples randomly drawn from other doc￾uments. This corpus was employed to adapt the embedding model to the… view at source ↗
Figure 2
Figure 2. Framework of the proposed QMKGF. Given a text, identify all of these types of entities and all of the relationships between these entities. Please use {language} as the output language. Text: {text} -Steps￾1. Identify all entities. For each identified entity, extract the following information: entity_name: The name of the entity, in the same language as the input text. 2. From the entities identified in step 1, find… view at source ↗
Figures from the paper (5 more)
Figure 1
Figure 1. Figure 1: One-hop relations: Given the identified truth entity 𝑒𝑡 , we first construct its one-hop neighbor set 1 (𝑒𝑡 ), defined as { 𝑣 ∈ 𝑉 ∣ (𝑒𝑡 , 𝑣) ∈ 𝐸 } . Within this set, we compute the semantic similarity between 𝑒𝑡 and each neighboring entity 𝑒𝑖 ∈ 1 (𝑒𝑡 ), and select th…
Figure 4
Figure 4. Figure 4: Impact of fine-tuning embeddings (combined view) [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Effect of subgraph node count. the performance on ROUGE metrics remains relatively close, but BLEU and METEOR scores are notably lower. This suggests insufficient semantic coverage in the subgraph, which results in less informative and under-supported gen￾erated conten…
Figure 6
Figure 6. Figure 6: illustrates the impact of the number of atten￾tion heads in the multi-head attention mechanism on model performance, showing notable differences across the Hot￾potQA, Cultour, and SQuAD datasets. Experimental results demonstrate that increasing the number of attention …
Figure 7
Figure 7. Figure 7: Effect of subgraph node count. fusion, Top-5 triples fusion, and RM fusion—on the Hot￾potQA and MuSiQue datasets. All Fusion: All triples from each subgraph are retained without any filtering. Top-5 Triples Fusion: Only the top 5 triples from each subgraph are preserve…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

42 extracted references · 23 canonical work pages

  1. [1]

    Collaboration between intelligent agents and large language models: A novel approach for enhancing code generation capability

    Bai, X., Huang, S., Wei, C., Wang, R., 2025. Collaboration between intelligent agents and large language models: A novel approach for enhancing code generation capability. Expert Systems with Applica- tions 269, 126357. doi:https://doi.org/10.1016/j.eswa.2024.126357

    work page arXiv 2025

  2. [2]

    Rouge metric evaluation for text summarization techniques

    Barbella, M., Tortora, G., 2022. Rouge metric evaluation for text summarization techniques. Available at SSRN 4120317

    work page 2022

  3. [3]

    Chatlaw: Open- sourcelegallargelanguagemodelwithintegratedexternalknowledge bases

    Cui, J., Li, Z., Yan, Y., Chen, B., Yuan, L., 2023. Chatlaw: Open- sourcelegallargelanguagemodelwithintegratedexternalknowledge bases. arXiv preprint arXiv:2306.16092

    arXiv 2023

  4. [4]

    Devlin, J., Chang, M.W., Lee, K., Toutanova, K., 2019. Bert: Pre- training of deep bidirectional transformers for language understand- ing, in: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human lan- guagetechnologies,volume1(longandshortpapers),pp.4171–4186

    work page 2019

  5. [5]

    From local to global:Agraphragapproachtoquery-focusedsummarization

    Edge,D.,Trinh,H.,Cheng,N.,Bradley,J.,Chao,A.,Mody,A.,Truitt, S., Metropolitansky, D., Ness, R.O., Larson, J., 2024. From local to global:Agraphragapproachtoquery-focusedsummarization. arXiv preprint arXiv:2404.16130

    arXiv 2024

  6. [6]

    IIRC: A Dataset of Incomplete Information Reading Comprehension Questions

    Ferguson, J., Gardner, M., Hajishirzi, H., Khot, T., Dasigi, P., 2020. Iirc: A dataset of incomplete information reading comprehension questions. arXiv preprint arXiv:2011.07127

    work page Pith review arXiv 2020

  7. [7]

    Cuerag: Dynamic multi-output cue memory under h framework for retrieval- augmented generation

    Fu,Y.,Liu,D.,Zhang,B.,Jiang,Z.,Mei,H.,Guan,J.,2025. Cuerag: Dynamic multi-output cue memory under h framework for retrieval- augmented generation. Neurocomputing , 130235

    work page 2025

  8. [8]

    Retrieval-augmentedgenerationforlarge language models: A survey

    Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang,M.,Wang,H.,2023. Retrieval-augmentedgenerationforlarge language models: A survey. arXiv preprint arXiv:2312.10997

    arXiv 2023

Show all 42 references
  1. [9]

    Retrieval augmented language model pre-training, in: International conference on machine learning, PMLR

    Guu, K., Lee, K., Tung, Z., Pasupat, P., Chang, M., 2020. Retrieval augmented language model pre-training, in: International conference on machine learning, PMLR. pp. 3929–3938

    2020

  2. [10]

    Active retrieval augmented generation, in: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp

    Jiang,Z.,Xu,F.F.,Gao,L.,Sun,Z.,Liu,Q.,Dwivedi-Yu,J.,Yang,Y., Callan, J., Neubig, G., 2023. Active retrieval augmented generation, in: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 7969–7992

    2023

  3. [11]

    A two-stage proactive dialogue generator for efficient clinical information collec- tion using large language model

    Li, X., Hou, X., Ravi, N., Huang, Z., Gan, Y., 2025. A two-stage proactive dialogue generator for efficient clinical information collec- tion using large language model. Expert Systems with Applications , 127833

    2025

  4. [12]

    Graph learning in the era of llms: A survey from the perspective of data, models, and tasks

    Li,X.,Wu,Z.,Wu,J.,Cui,H.,Jia,J.,Li,R.H.,Wang,G.,2024. Graph learning in the era of llms: A survey from the perspective of data, models, and tasks. arXiv preprint arXiv:2412.12456

    2024 arXiv

  5. [13]

    Bge landmark embedding: A chunking-free embedding method for retrieval augmented long- context large language models

    Luo, K., Liu, Z., Xiao, S., Liu, K., 2024. Bge landmark embedding: A chunking-free embedding method for retrieval augmented long- context large language models. arXiv preprint arXiv:2402.11573

    2024 arXiv

  6. [14]

    Bcembedding: Bilingual and crosslingual embedding for rag.https://github.com/netease-youdao/BCEmbedding

    NetEase Youdao, I., 2023. Bcembedding: Bilingual and crosslingual embedding for rag.https://github.com/netease-youdao/BCEmbedding

    2023

  7. [15]

    Coregen: Contextualized code representation learning for commit message generation

    Nie, L.Y., Gao, C., Zhong, Z., Lam, W., Liu, Y., Xu, Z., 2021. Coregen: Contextualized code representation learning for commit message generation. Neurocomputing 459, 97–107

    2021

  8. [16]

    Training language models to follow instructions with human feedback

    Ouyang,L.,Wu,J.,Jiang,X.,Almeida,D.,Wainwright,C.,Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al., 2022. Training language models to follow instructions with human feedback. Ad- vances in neural information processing systems 35, 27730–27744

    2022

  9. [17]

    Assessing llms in mali- cious code deobfuscation of real-world malware campaigns

    Patsakis, C., Casino, F., Lykousas, N., 2024. Assessing llms in mali- cious code deobfuscation of real-world malware campaigns. Expert Systems with Applications 256, 124912. doi: https://doi.org/10. 1016/j.eswa.2024.124912

    2024

  10. [18]

    Check your facts and try again:Improvinglargelanguagemodelswithexternalknowledgeand automated feedback

    Peng, B., Galley, M., He, P., Cheng, H., Xie, Y., Hu, Y., Huang, Q., Liden, L., Yu, Z., Chen, W., et al., 2023. Check your facts and try again:Improvinglargelanguagemodelswithexternalknowledgeand automated feedback. arXiv preprint arXiv:2302.12813

    2023 arXiv

  11. [19]

    Squad:100,000+questionsformachinecompre- hension of text

    Rajpurkar,P.,2016. Squad:100,000+questionsformachinecompre- hension of text. arXiv preprint arXiv:1606.05250

    2016 arXiv

  12. [20]

    A structured review of the validity of bleu

    Reiter, E., 2018. A structured review of the validity of bleu. Compu- tational Linguistics 44, 393–401

    2018

  13. [21]

    The probabilistic relevance framework: Bm25 and beyond

    Robertson, S., Zaragoza, H., et al., 2009. The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends® in Infor- mation Retrieval 3, 333–389

    2009

  14. [22]

    Bleu, meteor, bertscore: Evaluation of metrics performance in assessing critical translation errors in sentiment-oriented text

    Saadany, H., Orasan, C., 2021. Bleu, meteor, bertscore: Evaluation of metrics performance in assessing critical translation errors in sentiment-oriented text. arXiv preprint arXiv:2109.14250

    2021 arXiv

  15. [23]

    Generate-then-ground in retrieval-augmented generation for multi- hop question answering

    Shi,Z.,Zhang,S.,Sun,W.,Gao,S.,Ren,P.,Chen,Z.,Ren,Z.,2024. Generate-then-ground in retrieval-augmented generation for multi- hop question answering. arXiv preprint arXiv:2406.14891

    2024 arXiv

  16. [24]

    Ernie: Enhanced representation through knowledge integration

    Sun,Y.,Wang,S.,Li,Y.,Feng,S.,Chen,X.,Zhang,H.,Tian,X.,Zhu, D., Tian, H., Wu, H., 2019. Ernie: Enhanced representation through knowledge integration. arXiv preprint arXiv:1904.09223

    2019 arXiv

  17. [25]

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al.,

  18. [26]

    Transactions of the Association for Computational Linguistics 10, 539–554

    Trivedi, H., Balasubramanian, N., Khot, T., Sabharwal, A., 2022.♪ musique: Multihop questions via single-hop question composition. Transactions of the Association for Computational Linguistics 10, 539–554

    2022

  19. [27]

    Query2doc:Queryexpansionwith large language models

    Wang,L.,Yang,N.,Wei,F.,2023. Query2doc:Queryexpansionwith large language models. arXiv preprint arXiv:2303.07678

    2023 arXiv

  20. [28]

    Qcg-rerank: Chunks graph rerank with query expansion in retrieval-augmented llms for tourism domain

    Wei, Q., Yang, M., Han, C., Wei, J., Zhang, M., Shi, F., Ning, H., 2024a. Qcg-rerank: Chunks graph rerank with query expansion in retrieval-augmented llms for tourism domain. arXiv preprint arXiv:2411.08724

    arXiv

  21. [29]

    Tourllm: Enhancing llms with tourism knowledge

    Wei, Q., Yang, M., Wang, J., Mao, W., Xu, J., Ning, H., 2024b. Tourllm: Enhancing llms with tourism knowledge. arXiv preprint arXiv:2407.12791

    arXiv

  22. [30]

    Effective bug detection in graph database engines: An llm-based approach

    Wu, J., Wu, Z., Li, R., Qin, H., Wang, G., 2024a. Effective bug detection in graph database engines: An llm-based approach. arXiv preprint arXiv:2402.00292 . Q. Wei et al.:Preprint submitted to Elsevier Page 12 of 13 Leveraging social media news

    arXiv

  23. [31]

    Medicalgraphrag:Towardssafemed- ical large language model via graph retrieval-augmented generation

    Wu,J.,Zhu,J.,Qi,Y.,2024b. Medicalgraphrag:Towardssafemed- ical large language model via graph retrieval-augmented generation. arXiv preprint arXiv:2408.04187

    arXiv

  24. [32]

    Tpke-qa: A gapless few-shot extractive question answering approach via task-aware post-training and knowledge enhancement

    Xiao, Q., Li, R., Yang, J., Chen, Y., Jiang, S., Wang, D., 2024. Tpke-qa: A gapless few-shot extractive question answering approach via task-aware post-training and knowledge enhancement. Expert Systems with Applications 254, 124475

    2024

  25. [33]

    C-pack:Packaged resources to advance general chinese embedding

    Xiao,S.,Liu,Z.,Zhang,P.,Muennighof,N.,2023. C-pack:Packaged resources to advance general chinese embedding. arXiv preprint arXiv:2309.07597

    2023 arXiv

  26. [34]

    Search- in-the-chain: Towards the accurate, credible and traceable content generation for complex knowledge-intensive tasks

    Xu, S., Pang, L., Shen, H., Cheng, X., Chua, T.s., 2023. Search- in-the-chain: Towards the accurate, credible and traceable content generation for complex knowledge-intensive tasks. arXiv preprint arXiv:2304.14732

    2023 arXiv

  27. [35]

    Staf-llm: A scalable and task-adaptive fine-tuning framework for large language models in medical domain

    Xu, T., Chen, L., Hu, Z., Li, B., 2025. Staf-llm: A scalable and task-adaptive fine-tuning framework for large language models in medical domain. Expert Systems with Applications 281, 127582. doi:https://doi.org/10.1016/j.eswa.2025.127582

    2025

  28. [36]

    Hotpotqa:Adatasetfordiverse,explainable multi-hop question answering

    Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W.W., Salakhutdinov, R.,Manning,C.D.,2018. Hotpotqa:Adatasetfordiverse,explainable multi-hop question answering. arXiv preprint arXiv:1809.09600

    2018 arXiv

  29. [37]

    Evalua- tionofretrieval-augmentedgeneration:Asurvey,in:CCFConference on Big Data, Springer

    Yu, H., Gan, A., Zhang, K., Tong, S., Liu, Q., Liu, Z., 2024. Evalua- tionofretrieval-augmentedgeneration:Asurvey,in:CCFConference on Big Data, Springer. pp. 102–120

    2024

  30. [38]

    Lgkgr:Aknowledge graphreasoningmodelusingllmsaugmentedgnns

    Zhang,Y.,Zheng,W.,Huang,J.,Xiao,G.,2025. Lgkgr:Aknowledge graphreasoningmodelusingllmsaugmentedgnns. Neurocomputing 635, 129919

    2025

  31. [40]

    Retrieval-augmentedgeneration for ai-generated content: A survey

    Zhao, P., Zhang, H., Yu, Q., Wang, Z., Geng, Y., Fu, F., Yang, L., Zhang,W.,Jiang,J.,Cui,B.,2024b. Retrieval-augmentedgeneration for ai-generated content: A survey. arXiv preprint arXiv:2402.19473

    arXiv

  32. [41]

    Knowl- edge graph-guided retrieval augmented generation

    Zhu, X., Xie, Y., Liu, Y., Li, Y., Hu, W., 2025a. Knowl- edge graph-guided retrieval augmented generation. arXiv preprint arXiv:2502.06864

    arXiv

  33. [42]

    Legn:A large language model-guided event graph network for intraoperative hypotension prediction

    Zhu,Z.,Zhao,Q.,Ge,Y.,Li,J.,Wang,S.,Yang,J.J.,2025b. Legn:A large language model-guided event graph network for intraoperative hypotension prediction. Expert Systems with Applications , 128677. Q. Wei et al.:Preprint submitted to Elsevier Page 13 of 13

  34. [2023]

    arXiv preprint arXiv:2302.13971

    Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.