{"id":"e61bc8ab-67df-4fad-b6bb-bce329c3a75e","arxiv_id":"2508.19855","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A schema-bounded agentic framework unifies graph construction and retrieval, reporting SOTA accuracy and up to 90.71% token-cost savings across six benchmarks, plus a new anonymized QA dataset.","lead":"This paper presents Youtu-GraphRAG, a graph retrieval-augmented generation system that uses a shared schema to guide both knowledge graph construction and query-time retrieval through agentic reasoning. Across six multi-hop QA benchmarks, the authors report higher accuracy and substantially lower token costs than existing GraphRAG baselines, and introduce an anonymized QA dataset to reduce knowledge leaking from pretrained models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 6 defines S_s in terms of ϕ while Eq. 5 defines ϕ in terms of S_s, and Eq. 7's merge criterion is an undefined expectation difference with no ε values—so the core community detection, and thus the knowledge tree underlying all results, is not reproducible from the paper.","rationale":"The most load-bearing condition for the central claim is that the proposed construction produces a coherent knowledge tree. The empirical results could be correct even if some equations are misprinted, and the code release is a real asset; that is why the concern is conditional, not a rejection. But the paper's novelty and 'theoretically-grounded' claim rest on this clustering. The reader's weakest assumption pointed at Eq. 7; I agree and add that Eq. 6 is even more directly circular. A code-level check can settle whether this is merely a notation error (in which case the manuscript needs a correction but the claim may stand) or whether the criterion is actually under-specified and threshold-tuned per dataset (in which case the SOTA claim is not supported). No other issue—baseline fairness, AnonyRAG task design, LLM-as-judge—is as immediately load-bearing, because those can be debated without invalidating the core method. This check is concrete and uses the authors' own released artifacts.","tokens_in":17288,"tokens_out":11710,"duration_ms":131532,"concrete_test":"Clone the linked repository (github.com/TencentCloudADP/Youtu-GraphRAG) and inspect the community-detection module. (1) Find the actual implementation of Eq. 6: is S_s computed as a cosine similarity between F_Θ(T_i) and a community centroid, not via ϕ? If yes, Eq. 6 is a typo, not a fatal flaw. (2) Find the implemented merge criterion: does it match Eq. 7, and if so, over what distribution is the expectation taken, and what ε is used per dataset? Check configs for µ, λ, ε values. (3) Re-run the pipeline on one benchmark (e.g., HotpotQA) with the paper's stated settings; if Table 1's 86.5/81.2 numbers are not reproduced within a small tolerance using only the code and no hidden per-dataset threshold tuning, the core indexing step is not a single well-defined algorithm and the SOTA claim is unsubstantiated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The dually-perceived community detection is the heart of the knowledge-tree construction, and every retrieval route (community filtering, DFS, etc.) consumes that tree. As written, the algorithm is not well-defined. Eq. 5 defines the affinity as ϕ(e_i, C_m) = S_r ⊕ λ S_s. Eq. 6 then defines S_s(e_i, C_m) = ϕ(F_Θ(T_i), Σ_{j∈C_m} F_Θ(T_j)), i.e., S_s is defined through ϕ, the very function being defined. Unless ϕ on the right is a placeholder for a different similarity (e.g., cosine), the system of equations is circular and cannot be computed. The operator ⊕ and coefficient λ are never specified, and no value for λ is reported anywhere. Eq. 7 then merges clusters when E[ϕ(e_i, C_a)] − E[ϕ(e_i, C_b)] < ε, but the expectation is not defined over any stated distribution, the 'dual-perception divergence' is a signed difference rather than a divergence, and ε (like µ in Eq. 3) is never given or described as tuned. The paper's own contribution statement calls this 'theoretically-grounded,' but no derivation or consistency argument is supplied. Because the four-level knowledge tree is built on these clusters, the SOTA accuracy and token-efficiency claims cannot be independently reconstructed from the text. The released code may resolve this, but the manuscript as written leaves the central construction ambiguous.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Youtu-GraphRAG, a schema-guided GraphRAG system that unifies graph construction and retrieval. A seed graph schema bounds an LLM extraction agent; a 'dually-perceived' community detection method combines structural and semantic signals to build a four-level hierarchical knowledge tree; an agentic retriever decomposes complex queries into schema-aligned sub-queries with iterative reflection; and a new AnonyRAG dataset with an 'Anonymity Reversion' task is introduced to mitigate LLM knowledge leakage. The paper reports state-of-the-art answer accuracy on six benchmarks (HotpotQA, 2WikiMultiHopQA, MuSiQue, G-Bench, AnonyRAG-CHS, AnonyRAG-ENG) under two LLM backbones, together with large reductions in graph-construction token costs.","tokens_in":17752,"tokens_out":4786,"duration_ms":55464,"significance":"If the reported results are reproducible, the paper makes a useful contribution: the vertical integration of schema-guided extraction, hierarchical community construction, and schema-aware retrieval is a plausible route to improving both accuracy and token efficiency in GraphRAG. The proposed dual-mode evaluation (reject/open) and the anonymous dataset address a real weakness in existing GraphRAG benchmarks, namely knowledge leakage from pretrained LLMs. The authors also release code and data, which is a strength. However, the central construction algorithm is under-specified in the manuscript, and the experimental reporting does not currently support the strength of the headline claims. The core idea is defensible, but the paper needs substantial revision before the claims can be independently verified.","major_comments":[{"comment":"The dual-perception community detection algorithm is not well-defined as written. Eq. (5) defines phi(e_i, C_m) = S_r ⊕_lambda S_s, and Eq. (6) then defines S_s(e_i, C_m) = phi(F_Theta(T_i), sum_j F_Theta(T_j)). Unless the right-hand phi in Eq. (6) is a placeholder for a different similarity (e.g., cosine), the definition is circular. The operator ⊕ and the weighting lambda are never specified, and no numeric value for lambda is reported. Eq. (7) merges clusters when E[phi(e_i,C_a)] - E[phi(e_i,C_b)] < epsilon, but the expectation is taken over no stated distribution, the expression is a signed difference rather than a divergence, and epsilon (like mu in Eq. (3)) is never given. Because every downstream retrieval route consumes the knowledge tree built on these communities, the core indexing step is not reproducible from the manuscript. The contribution text calls the method 'theoretical","section":"§3.2, Eqs. (5)–(7)"},{"comment":"The empirical evaluation lacks basic statistical reporting. No error bars, confidence intervals, or number of independent runs are given for any table. The ablation study in Table 3 appears to be a single run, and the differences between the full model and ablations (e.g., 19.8 points on 2Wiki for 'w/o Agent') could be within run-to-run variance given the observed prompt sensitivity noted in §4.1. In addition, the LLM-as-judge for response similarity is DeepSeek-V3-0324, the same model family used for graph construction, retrieval generation, and the base LLM in half the experiments; this creates a risk of self-preference bias. Please report variance across multiple seeds/runs, justify the choice of judge, and define the 'top-20 accuracy' metric precisely (is it response-level exact/semantic-match accuracy, retrieval recall, or something else?).","section":"§4.4, Tables 1–3"},{"comment":"The headline claims of 'up to 90.71% saving of token costs and 16.62% higher accuracy' are not traceable to any table or figure. Figure 5 and Figure 6 are image-only with no numeric token-consumption table, and the 16.62% accuracy gain does not correspond directly to any pairwise comparison in Table 1 or Table 2. The reader cannot verify which dataset, mode, and baseline produced these numbers. Please add a table of construction-time/token costs for all methods and datasets, and state exactly how the headline percentages are computed.","section":"Abstract and Figure 5/6"}],"minor_comments":[{"comment":"The abstract says 'six challenging benchmarks' while the Contributions list says 'five challenging benchmarks.' The experiments use six; please make the count consistent.","section":"Contributions and Abstract"},{"comment":"The anonymized datasets are referred to as 'AnonyRAG-CHS' and 'AnonyRAG-ENG' in the text but appear as 'Annoy-CHS' and 'Annoy-ENG' in the tables. Use a single consistent name.","section":"Tables 1–2 and §4.2"},{"comment":"After Eq. (6), the text says 'Ss denotes the Jaccard similarity matrix computed over the multiset of incident relation types,' but Ss as defined in Eq. (6) is a semantic cosine similarity; the Jaccard term is Sr. The notation appears swapped.","section":"Eq. (6) and surrounding text"},{"comment":"The schema-update rule uses the notation fLLM(d, S) ⊙ S, but neither the output type of fLLM nor the ⊙ operation is defined. Please clarify.","section":"Eq. (3)"},{"comment":"Several typos: 'Anonymous Revertion' in the Contributions section should be 'Anonymity Reversion'; 'HotptQA' in Table 3 should be 'HotpotQA'; 'vito role' in §4.6 should be 'vital role.'","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The central idea is interesting and the released code/data may resolve some of the ambiguity, but the manuscript as written is not self-contained in its algorithm description, and the experimental claims lack the reproducibility details expected for a systems paper. The authors should be asked to provide a complete specification of the community-detection procedure, report hyperparameters and variance, and make the headline efficiency/accuracy numbers directly derivable from tables. With these changes, the paper could be a solid contribution; without them, the SOTA claims cannot be independently assessed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read the Youtu-GraphRAG paper. Short version: it's a serious systems paper with consistent wins across six benchmarks and a genuinely useful new anonymized benchmark, but the core community-detection equations are under-specified to the point of non-reproducibility from the text, and the headline efficiency/accuracy numbers aren't directly traceable from the tables.\n\nWhat's actually new: the vertical unification of schema-bounded extraction, dual-perception clustering, and schema-guided agentic retrieval is a coherent design, and the AnonyRAG dataset (Water Margin, Dream of the Red Chamber, Moby-Dick, Middlemarch) is a real contribution that addresses knowledge leakage. The ablations show each component matters, with agent reasoning the largest factor on multi-hop. Code and data are linked.\n\nSoft spots, in order of severity. First, the math in Section 3.2: Eq. 5 defines phi with an undefined operator ⊕ and λ, and Eq. 6 defines S_s in terms of phi, which is circular if phi is the same function; likely they meant cosine similarity, but as written it's ambiguous. Eq. 7's merge criterion is an expectation difference with no distribution or ε value. µ and λ also never get numbers. The paper calls this \"theoretically-grounded\" but gives no derivation. If the released code resolves all this, fine, but the manuscript needs fixing. Second, the evaluation has no error bars, ablations are single-run, and the LLM judge is from the same family as the generator—though the reject-mode prompts do help. Third, the abstract claims \"up to 90.71% saving of token costs and 16.62% higher accuracy\" but I can't map those to particular cells in Tables 1-2 or Figures 5-6 without more work.\n\nThe central empirical claim—that schema-guided construction plus agentic retrieval moves the Pareto frontier—is plausible and backed by consistent results, so the weaknesses are repairable rather than fatal.\n\nRecommendation: send it to peer review. The reviewer should demand the authors clarify the notation, report all hyperparameters, add variance estimates, and provide a precise derivation of the token/accuracy gains. The AnonyRAG dataset alone makes it worth the referees' time.","headline":"Serious GraphRAG systems paper with a useful new benchmark, but the core clustering math is under-specified and the headline numbers aren't directly traceable.","tokens_in":18177,"tokens_out":3369,"would_cite":true,"duration_ms":33464,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A vertically unified agentic GraphRAG design ties graph construction to retrieval through one evolving schema, reporting up to 90.71% lower token costs and 16.62% higher accuracy across six benchmarks.","keywords":["GraphRAG","graph schema","community detection","multi-hop question answering","agentic retrieval","knowledge tree","knowledge leakage","anonymous benchmark"],"falsifier":"Re-run the full pipeline on all six benchmarks with one fixed value of the community-merging threshold and no per-dataset tuning. If top-20 accuracy under reject mode collapses on any dataset, the hierarchical knowledge tree is not domain-robust as claimed; this directly tests whether the merging rule defines a meaningful, transferable criterion.","tokens_in":17255,"feed_emoji":"🕸️","tokens_out":13602,"duration_ms":129232,"temperature":0.7,"pith_summary":"Youtu-GraphRAG sets out to prove that graph construction and graph retrieval in a GraphRAG system should be optimized as one connected process, not as separate components. It ties both stages to a single, evolving graph schema: the same schema bounds the extraction agent to relevant entity and relation types and later guides the retrieval agent in decomposing complex questions into parallel sub-queries. On six benchmarks the paper reports state-of-the-art top-20 accuracy while cutting graph-construction token costs by up to 90.71% relative to strong baselines, and it adds two anonymized datasets with an Anonymity Reversion task so that accuracy gains measure retrieved evidence rather than the model's memorized knowledge. If the claim holds, GraphRAG systems can become cheaper and more accurate at the same time, and moving to a new domain reduces to seeding a small schema rather than re-engineering the pipeline.","feed_headline":"One schema unifies GraphRAG, cutting token costs 90.7%","feed_subtitle":"Shared schema guides extraction and query decomposition, lifting multi-hop QA accuracy up to 16.62%.","key_machinery":"The load-bearing object is the graph schema S=(Se,Sr,Sattr), the set of targeted entity types, relation types, and attribute types that acts as the single interface between construction and retrieval. It bounds extraction to schema-compliant triples, is expanded adaptively via a confidence threshold, and is then read again by the retriever to decompose queries into sub-queries that match valid graph patterns. The second mechanism is dually-perceived community detection, a three-stage procedure that encodes each entity by aggregating embeddings of its one-hop triples, initializes clusters with K-means, and then iteratively fuses communities using a dual-perception score blending relational Ja","core_discovery":"The central claim is that a vertically unified agentic paradigm—rather than isolated improvements to graph construction or retrieval—is what drives both cost savings and reasoning gains in GraphRAG. The discovery is the shared graph schema itself, deployed twice. During construction it constrains the extraction agent to targeted entity types, relations, and attribute types and expands automatically with high-confidence patterns; during retrieval the same schema is interpreted by an agentic retriever to split a complex query into schema-aligned atomic sub-queries and to pick among entity, triple, community, and path-traversal retrieval routes. A second mechanism, dually-perceived community de","pith_inferences":["The schema could become a runtime control surface: editing entity or relation types at deployment time would change both extraction and retrieval behavior, giving practitioners a single knob for domain adaptation—an extension the paper does not test.","The Anonymity Reversion task could be applied to existing public benchmarks by re-anonymizing their entities, producing a low-cost way to re-baseline prior GraphRAG results and expose which systems relied on memorized knowledge.","The undefined merge criterion in Eq. (7) leaves room for a formal treatment: if the divergence is spelled out and the threshold derived rather than tuned, the community tree could be made provably coherent, turning the reported empirical gains into a transferable principle.","The schema-aligned decomposition suggests the same knowledge tree could support interactive, multi-turn query refinement without rebuilding the index, since re-planning sub-queries is cheaper than re-retrieving from scratch."],"forward_implications":["If the unified schema design is correct, graph construction and retrieval no longer trade off against each other: the same schema that slims extraction also tells the retriever where to look, so both accuracy and token cost can improve together.","The four-level knowledge tree gives one structure for both global and local questions, allowing top-down community filtering for broad queries and bottom-up triple and attribute retrieval for fact-level ones.","Schema expansion from document feedback means a new domain can be entered with a small seed schema and grown automatically, rather than requiring a hand-built ontology per corpus.","Because the anonymized evaluation forces reliance on retrieved context, the reported margins under reject mode indicate the retriever actually supplies complete evidence instead of letting the language model answer from memory.","An agentic retriever that decomposes queries along schema paths and reflects on its steps should scale to more compositional questions, since each sub-query is grounded in a valid graph pattern."],"supporting_citations":[{"why":"Defines the foundational GraphRAG construction pipeline (entity graph plus community detection and summarization) that the paper extends and must outperform.","marker":"[Edge et al., 2024]"},{"why":"Supplies the recursive tree-organized retrieval baseline that motivates the hierarchical knowledge tree and serves as a main hierarchical competitor.","marker":"[Sarthi et al., 2024]"},{"why":"Supplies a strong graph-memory retrieval baseline using personalized PageRank over an open information-extraction graph.","marker":"[Jimenez Gutierrez et al., 2024]"},{"why":"Supplies the strongest retrieval-side competitor with iterative retrieval and memory mechanisms, a direct target of the efficiency comparison.","marker":"[Gutiérrez et al., 2025]"},{"why":"Supplies an efficiency-focused flat graph retrieval baseline that represents the isolated-retrieval direction.","marker":"[Guo et al., 2024]"},{"why":"Supplies a hierarchical graph baseline aimed at construction and retrieval efficiency, used in the token-cost comparison.","marker":"[Zhao et al., 2025]"},{"why":"Supplies the domain-specific textbook-corpus benchmark used to test cross-domain transfer without task-specific tuning.","marker":"[Xiao et al., 2025]"},{"why":"Supplies a standard multi-hop question-answering evaluation set used for the open and reject accuracy comparisons.","marker":"[Yang et al., 2018]"},{"why":"Supplies a multi-hop question-composition dataset used for evaluation and as seed templates for the anonymized questions.","marker":"[Trivedi et al., 2022]"},{"why":"Supplies a multi-hop reasoning dataset with compositional steps, used as an evaluation set and as template source.","marker":"[Ho et al., 2020]"}],"fun_headline_variants":["Shared schema unifies GraphRAG, slashing tokens 90.7%","One schema guides build and query, boosting accuracy 16.6%","Youtu-GraphRAG: unified agents cut token cost by 90.7%","GraphRAG's vertical unity: same schema for build and retrieve","A single schema powers GraphRAG from extraction to retrieval"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The central claim depends on a community-merging rule in Eq. (7) that is never fully defined—the 'dual-perception divergence' it compares is never specified and its thresholds are never given values—so if that rule is ill-posed or tuned per dataset, the hierarchical knowledge tree that all downstream retrieval builds on has no guaranteed coherence.","fun_headline_variants_meta":{"raw":{"variants":["Shared schema unifies GraphRAG, slashing tokens 90.7%","One schema guides build and query, boosting accuracy 16.6%","Youtu-GraphRAG: unified agents cut token cost by 90.7%","GraphRAG's vertical unity: same schema for build and retrieve","A single schema powers GraphRAG from extraction to retrieval"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000171,"raw_usage":{"total_tokens":1149,"prompt_tokens":823,"completion_tokens":326,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":567,"completion_tokens_details":{"reasoning_tokens":238}},"tokens_in":567,"tokens_out":326,"duration_ms":3629,"temperature":1.0,"reasoning_tokens":238,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T15:22:43.712381+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the full pipeline on all six benchmarks with one fixed value of the community-merging threshold and no per-dataset tuning. If top-20 accuracy under reject mode collapses on any dataset, the hierarchical knowledge tree is not domain-robust as claimed; this directly tests whether the merging rule defines a meaningful, transferable criterion.","supporting_citations":[{"cited_title":"Hipporag: Neurobiolog- ically inspired long-term memory for large language models","cited_arxiv_id":null,"evidence_quote":"Supplies a strong graph-memory retrieval baseline using personalized PageRank over an open information-extraction graph."}],"review_version":1}