{"id":"0119d555-6a45-4e40-95e4-319b555512e2","arxiv_id":"2506.17951","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"GraphMPA combines an embedding-similarity hierarchical graph with mode-seeking preference optimization to improve RAG question answering on six datasets.","lead":"This paper introduces GraphMPA, a retrieval-augmented question answering system that builds a layered graph of text chunks and fine-tunes a language model to prefer reasoned answers. It reports consistent gains over several RAG baselines on six QA datasets, using small LLMs rather than expensive models for preference data.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (14) defines the mode-seeking loss through a reward-model-based distribution pr_phi that is never specified or trained; Appendix B additionally writes KL minimization as arg max, so the preference-alignment contribution is not supported as written and may not be what the experiments ran.","rationale":"The reader's weakest assumption and mine coincide: the mode-seeking loss is ill-defined. I read the paper's strongest claim as empirical effectiveness of GraphMPA; the table and ablations give some support to the graph-retrieval half, so I do not think the paper should be rejected outright. But the second half of the claimed contribution—mode-seeking preference alignment—is not verifiable from the manuscript. The training description is one line about trl; Eq. (14) references a reward model that does not appear elsewhere; Appendix B's arg max/arg min errors make the theory unreliable. These are internal-consistency problems, not merely disagreements with prior work. A revision that specifies pr_phi, the implemented loss, and code would settle it. Because the reader already assigned CONDITIONAL for essentially this reason, my stress test does not change the verdict.","tokens_in":19935,"tokens_out":6160,"duration_ms":76051,"concrete_test":"Obtain the released training code at https://github.com/tangquanwei/GraphMPA and inspect the actual loss call. Determine whether the training loop computes D_KL(pf_theta || pr_phi) with an explicit pr_phi (e.g., from a reward model), or whether it calls trl's DPOTrainer/DPO loss. If it is the latter, the MS loss in Eq. (14) was never implemented, and the mode-seeking claim should be revised or dropped. If no code or exact loss is provided, a written derivation of the loss from Eq. (14), including the definition of pr_phi and its gradient, would also settle the question.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central novelty, 'mode-seeking preference optimization', rests on Eq. (14): L_ms = E_{x~Dpref} E_{pi_sft(y1:K|x)} [D_KL(pf_theta(·|y1:K,x) || pr_phi(·|y1:K,x))]. pr_phi is called 'the empirical distribution based on the reward model', but the paper never trains or describes a reward model, never specifies how pr_phi is constructed, and never explains how gradients of this KL are computed. The experiments only say training uses the trl library with lr 1e-5, batch size 4, 2 epochs; no custom MS loss or reward model is mentioned. If the implemented loss is trl's standard DPO loss (or another built-in), then the reverse-KL/mode-seeking objective in Eq. (14) is not what the numbers measure. This matters because the abstract and contributions claim the mode-seeking strategy is the mechanism for better preference alignment. The graph component is better supported: the 'w/o Training' ablations in Table 3 still beat Basic RAG on most datasets. But the theoretical contribution is vacuous unless pr_phi and the training objective are pinned down. Appendix B compounds the problem: Eqs. (10)-(11) set both pi_ms and pi_dpo as arg max of a KL divergence, which should be arg min for either forward or reverse KL minimization; as written, the optimization direction is wrong.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GraphMPA, a retrieval-augmented question-answering framework built on a hierarchical document graph with community summarization, combined with a preference-alignment stage called mode-seeking preference optimization (MS). The graph is constructed by splitting documents, embedding them, connecting nodes by cosine similarity, detecting communities with the Leiden algorithm, and recursively summarizing communities into higher layers. The preference-alignment component builds a synthetic chosen/rejected dataset with small LLMs and claims to optimize a reverse-KL, mode-seeking loss rather than the forward-KL objective of standard DPO. Experiments on QASPER, QuALITY, RiddleSense, PubMedQA, MedQA, and MedMcQA across three 7B-8B backbones report consistent gains over Basic RAG, RAPTOR, LightGraphRAG, and Reward-RAG, along with ablations of the graph depth, retrieval top-k, similarity threshold, and training component.","tokens_in":20307,"tokens_out":5564,"duration_ms":60046,"significance":"If the empirical results hold, the graph-construction contribution is practically valuable: it avoids the expensive entity-relation extraction used by GraphRAG-style methods, uses only small LLMs for summarization, and shows gains on a diverse set of QA benchmarks, including long-document and medical QA. The paper also provides a fairly complete ablation structure that separates the contributions of retrieval, summarization, and training. However, the mode-seeking theory is currently not supported as written: the reward-model distribution in the central objective is never defined, the training implementation is not shown to match the stated loss, and the derivation in Appendix B contains optimization-direction errors. The paper is therefore significant but conditional on the authors clarifying and verifying the preference-alignment contribution.","major_comments":[{"comment":"The mode-seeking loss is not a well-defined training objective as written. The distribution pr_phi(·|y1:K,x) is called \"the empirical distribution based on the reward model,\" but no reward model is defined or trained anywhere in the paper, and Section 4.1 reports only that training uses the trl library with learning rate 1e-5, batch size 4, and 2 epochs. Consequently, the reader cannot tell whether the implemented loss is the reverse-KL objective in Eq. (14) or a standard DPO/IPO loss available in trl, and the claimed mode-seeking mechanism is not verified by the experiments. Please specify how pr_phi is constructed, how gradients of the KL term are computed, and provide a code-level description of the training loss; if the implemented loss is a standard preference-optimization loss, the mode-seeking claim should be revised or removed.","section":"Section 3.2, Eqs. (8) and (14)"},{"comment":"The derivation contains technical errors that undermine the distinction between mode-seeking and mean-seeking. Eqs. (10) and (11) define both pi_ms and pi_dpo as the arg max of a KL divergence; minimizing either forward or reverse KL should be written as arg min. In addition, Eq. (13) defines the forward KL D_KL(pi*||pi) with an expectation under y~pi(y|x), but the expectation for the forward KL should be under y~pi*(y|x). These errors must be corrected before the appendix can support Eq. (14).","section":"Appendix B, Eqs. (10), (11), and (13)"},{"comment":"All reported numbers appear to come from a single run, with no error bars, no number of seeds, and no significance tests. Several gaps are small, for example QuALITY with Qwen 7B (47.64 vs. 47.32 for Basic RAG) and MedMcQA with LLaMa 8B (64.28 for GraphMPA vs. 64.16 for both w/ DPO and w/o Training), so without variance estimates it is impossible to know whether the claimed improvement is reliable. Please report mean and standard deviation over at least three seeds and, where feasible, a paired significance test for the main comparisons.","section":"Section 4.2, Table 1"},{"comment":"The log-probability comparison on PubMedQA is a weak and indirect validation of \"better human preference alignment.\" The paper does not define a reward model, so the assertion that higher log probability means \"more in line with human preferences\" is not substantiated. Moreover, the observation that the MS distribution is \"concentrated in a small range\" is a variance statement, not evidence of mode-seeking behavior. A convincing validation would require either a held-out preference set with human judgments or a direct evaluation of the trained policy against a defined reward distribution.","section":"Section 4.3, Figure 6"},{"comment":"The procedure for constructing Dpref is underspecified and is load-bearing for the preference-alignment claim. The paper states that 20,000 samples are synthesized from TrainQAs for all datasets, but it does not say whether the \"answer\" in the chosen response is the gold answer or the small LLM's own answer, how the context subsets C1,C2,... are sampled, or how rejected responses are generated beyond \"only includes the answer.\" Since the MS loss in Eq. (14) assumes a well-defined preference distribution over chosen and rejected responses, the authors should provide the full prompt templates, filtering rules, and a precise description of the data-generation pipeline.","section":"Section 3.2, Preferred Dataset Preparation"}],"minor_comments":[{"comment":"The notation y1:K is used without definition; y1,...,yK should be defined as the sampled response set, and the statement that Dpref \"contains human-labeled preference data\" conflicts with the auto-synthesized construction described in the same section.","section":"Section 3.2, Eq. (8)"},{"comment":"The pseudocode has typos and inconsistencies: line 3 contains \"summay,\" and the edge-selection procedure in Algorithm 1 uses a full argsort while Algorithm 2 uses top-k selection; the two algorithms should be aligned and the function arguments (large, small, n_layers, tau) should be consistent.","section":"Algorithm 1"},{"comment":"Eq. (6) writes the retrieval step as a max over graph layers, but the surrounding text says the method selects the top-k most similar documents. Please clarify how layer-level similarity scores are aggregated into a document ranking.","section":"Section 3.2, Eq. (6)"},{"comment":"The figure has no axis labels or legend; the reader cannot tell which point cloud corresponds to MS, DPO, or SFT, and the quantitative claim about medians cannot be checked from the plot as rendered.","section":"Figure 6"},{"comment":"The sentence \"Qwen 7B surpasses all other models, excluding GPT-4\" is confusing because Table 1 shows that Mistral 8B outperforms Qwen 7B on four of the six datasets; please rephrase to describe the backbone comparisons accurately.","section":"Section 4.2"}],"recommendation":"major_revision","confidential_remarks":"I did not inspect the linked GitHub repository, so my assessment is based on the manuscript text alone. The empirical core of the graph framework appears defensible, but the preference-alignment section, which is claimed as a main contribution, is not currently reproducible or correctly derived. If the authors can clarify pr_phi and confirm the implemented training objective, the paper may be suitable for publication after a thorough revision. Otherwise, the mode-seeking claim should be reframed or removed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe graph part of this paper is the real contribution; the preference-alignment theory is not. If you take GraphMPA as a retrieval architecture, the empirical case is decent: an embedding-similarity hierarchical graph with community summarization, no dependence on large LLMs for entity extraction, evaluated on six QA datasets with three 7B-8B backbones. The ablation table shows the graph alone (w/o Training) beats Basic RAG on every dataset, sometimes by wide margins (e.g., RiddleSense 71.8 vs 60.2, MedMcQA 64.2 vs 50.4). That is a useful engineering result and supports the graph design.\n\nThe mode-seeking preference optimization, however, is not supported as written. Equation (14) defines the loss using pr_phi, an \"empirical distribution based on the reward model,\" but the paper never trains or specifies a reward model, never says how pr_phi is constructed, and the implementation section only mentions the trl library with lr 1e-5, batch size 4, and 2 epochs. Nothing indicates that a custom MS loss was used. So the abstract's claim that mode-seeking alignment drives the gains is unverified; the experiments as described are consistent with standard DPO. Appendix B compounds this with Eqs. (10)-(11) both written as arg max of a KL divergence, when minimizing either forward or reverse KL means arg min. That is a concrete technical error.\n\nThe other weakness is statistical: no error bars, no significance tests, and some baseline numbers (Reward-RAG) are taken from public reports rather than re-run. The single \"effectiveness of mode-seeking\" plot on PubMedQA is a self-reported log-probability check, not strong evidence.\n\nProportionately: the graph contribution holds up and is worth publishing; the preference-alignment contribution needs either a proper derivation and implementation details or removal from the claims. I'd send this to a serious referee, but the revision must pin down what objective was actually optimized and fix the appendix. If the training was indeed DPO, drop the mode-seeking framing.","headline":"The graph retrieval component is the real, well-supported contribution; the mode-seeking preference loss is not actually defined or verified, so the theoretical framing overclaims what the experiments show.","tokens_in":20794,"tokens_out":2424,"would_cite":true,"duration_ms":27761,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"GraphMPA combines a similarity-built hierarchical document graph with mode-seeking preference optimization to beat RAG baselines on six QA datasets.","keywords":["Retrieval-Augmented Generation","Question Answering","Hierarchical Document Graph","Community Summarization","Preference Alignment","Mode-Seeking Optimization","Direct Preference Optimization","Small Language Models"],"falsifier":"One concrete check is to inspect the training code and confirm the loss exactly matches Eq. (14) with a defined reward-model distribution; if the code uses a standard DPO loss such as $\\log\\sigma(\\beta(\\log\\pi_\\theta(y_w) - \\log\\pi_\\theta(y_l)))$, then the mode-seeking claim is not what the experiments test.","tokens_in":19770,"feed_emoji":"🧠","tokens_out":7052,"duration_ms":68523,"temperature":0.7,"pith_summary":"The paper claims that retrieval-augmented question answering can be improved by organizing external documents into a hierarchical graph built from embedding-similarity edges and community summaries, then fine-tuning the generator with a mode-seeking preference objective. The authors argue that entity-based graph retrieval misses sentences without entities, and that standard DPO-style alignment spreads probability mass across modes instead of concentrating on the human-preferred mode. On six QA datasets spanning general, riddle, and medical questions, GraphMPA beats Basic RAG, RAPTOR, LightGraphRAG, and Reward-RAG with LLaMA-8B, Qwen-7B, and Mistral-8B backbones. The framework aims to show that small-LLM data synthesis plus graph-aware retrieval can replace GPT-4-scale supervision.","feed_headline":"Graph retrieval plus preference tuning wins across six Q&A benchmarks","feed_subtitle":"Adding a similarity-based document graph and preference alignment improves answer accuracy for 7B-8B models.","key_machinery":"The central objects are (1) a hierarchical document graph whose nodes are small document chunks or community summaries, edges are cosine similarities between embeddings above a threshold $\\tau$, and layers are built by Leiden community detection followed by LLM summarization, and (2) the mode-seeking loss $L_{ms}(\\pi_\\theta) = \\mathbb{E}_{x\\sim D_{pref}} \\mathbb{E}_{\\pi_{sft}(y_{1:K}|x)} \\left[ D_{KL}(p_{f_\\theta}(\\cdot|y_{1:K},x) \\| p_{r_\\phi}(\\cdot|y_{1:K},x)) \\right]$, which aligns the model's empirical distribution with a reward-model-defined empirical distribution by minimizing reverse KL. Retrieval selects top-k documents across layers, and training data is synthesized by small LLMs generating reasoning, with chosen answers including reasoning and rejected answers being answer-only.","core_discovery":"GraphMPA's central claim is that two complementary mechanisms—a hierarchical document graph with community summarization based on generic embedding similarity, and mode-seeking preference optimization based on automatically synthesized small-LLM data—together improve RAG-based QA across six datasets. The graph replaces entity-relation extraction with dense-embedding cosine similarity and Leiden community detection, producing layers of summary nodes that supply both low-level and high-level context. The alignment step constructs chosen/rejected pairs where chosen answers include reasoning chains generated by small LLMs and rejected answers are answer-only, then optimizes a reverse-KL mode-seeking loss rather than DPO's forward-KL mean-seeking loss. Table 1 reports consistent gains over baselines for all three backbones, and ablations attribute the gain to retrieval, summarization, and training.","pith_inferences":["If the mode-seeking loss is implemented as described, it should apply to any preference-alignment task, such as summarization or dialogue, where concentrating on one preferred mode is desirable rather than covering all modes.","The graph construction recipe is input-domain agnostic, so the same similarity-plus-community-summarization approach could be tested on legal, scientific, or other long-document tasks.","A sharper test would isolate the graph contribution from the alignment contribution by comparing GraphMPA against a version with DPO, holding the data and graph fixed; the paper's ablation removes training or swaps DPO but does not fully separate the two mechanisms.","Because the paper never specifies a reward model behind $p_{r_\\phi}$, the practical implementation may reduce to a DPO variant; if so, the empirical gains would still support the graph and data-synthesis contributions, but not the mode-seeking mechanism."],"forward_implications":["Graph-based retrieval can be built with generic embedding models and small LLMs, avoiding costly entity-relation extraction by large models.","Retrieval over multiple graph layers supplies both low-level detail and high-level synthesized context, which the ablations show is necessary for the gains.","Mode-seeking preference alignment with small-LLM synthetic reasoning data can outperform DPO-style alignment in the tested QA settings.","The framework generalizes across LLaMA, Qwen, and Mistral instruction-tuned models in the 7B-8B range.","If the ablation results hold, each component—graph building, retrieval, and preference training—adds measurable value."],"supporting_citations":[{"why":"Defines DPO, the preference optimization baseline that GraphMPA extends and contrasts with mode-seeking loss.","marker":"Rafailov et al., 2024"},{"why":"Provides the mode-seeking and exact-optimization foundation that the MS loss builds on.","marker":"Ji et al., 2024"},{"why":"Supplies the Leiden community detection algorithm used to cluster graph nodes for summarization.","marker":"Traag et al., 2019"},{"why":"Supplies the BGE-M3 embeddings used to compute document similarity edges.","marker":"Chen et al., 2024"},{"why":"RAPTOR, a hierarchical retrieval-tree baseline that GraphMPA compares against.","marker":"Sarthi et al., 2024"},{"why":"LightGraphRAG, an entity-graph RAG baseline compared in the experiments.","marker":"Guo et al., 2024"},{"why":"Reward-RAG, a reward-driven RAG baseline compared in the experiments, including GPT-3.5 and GPT-4 results.","marker":"Nguyen et al., 2024"},{"why":"GraphRAG, the entity-based global graph approach that motivates the similarity-based graph design.","marker":"Edge et al., 2024"}],"fun_headline_variants":["GraphMPA: graph retrieval and mode-seeking alignment boost QA","Graph-based RAG plus preference tuning wins on six QA benchmarks","Hierarchical document graph with mode-seeking alignment improves QA","GraphMPA: similarity-based graphs plus reverse-KL alignment for QA","Mode-seeking preference optimization plus document graphs beats RAG baselines"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central assumption is that the training really optimizes the mode-seeking objective in Eq. (14) against a reward-model distribution; the paper never describes a reward model or verifies that the implemented loss equals this formula.","fun_headline_variants_meta":{"raw":{"variants":["GraphMPA: graph retrieval and mode-seeking alignment boost QA","Graph-based RAG plus preference tuning wins on six QA benchmarks","Hierarchical document graph with mode-seeking alignment improves QA","GraphMPA: similarity-based graphs plus reverse-KL alignment for QA","Mode-seeking preference optimization plus document graphs beats RAG baselines"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000322,"raw_usage":{"total_tokens":1745,"prompt_tokens":814,"completion_tokens":931,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":430,"completion_tokens_details":{"reasoning_tokens":845}},"tokens_in":430,"tokens_out":931,"duration_ms":8575,"temperature":1.0,"reasoning_tokens":845,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:23:27.792473+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"One concrete check is to inspect the training code and confirm the loss exactly matches Eq. (14) with a defined reward-model distribution; if the code uses a standard DPO loss such as $\\log\\sigma(\\beta(\\log\\pi_\\theta(y_w) - \\log\\pi_\\theta(y_l)))$, then the mode-seeking claim is not what the experiments test.","supporting_citations":[],"review_version":1}