Co-Evolving Graph and Text Memory for Training-Free Multi-Hop Question Answering

Hieu Man; Thien Huu Nguyen

REVIEW 3 major objections 5 minor 67 references

Co-E claims that a training-free multi-hop QA system can beat larger or trained baselines by maintaining two coupled memories — text snippets and a graph of triples — and synchronizing them after every reasoning step.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-07-31 23:52 UTC pith:TFL5MMMU

load-bearing objection A transparent system paper with a plausible new mechanism; the co-evolution story is not yet established because comparisons and ablations don't hold compute constant. the 3 major comments →

arxiv 2607.23278 v1 pith:TFL5MMMU submitted 2026-07-25 cs.CL cs.MA

Co-Evolving Graph and Text Memory for Training-Free Multi-Hop Question Answering

Hieu Man , Thien Huu Nguyen This is my paper

classification cs.CL cs.MA

keywords multi-hop question answeringretrieval-augmented generationknowledge-graph reasoningworking memorygraph-text synchronizationtraining-free inferenceMonte Carlo tree searchmemory consolidation

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that the hard part of multi-hop question answering is not retrieval or search but keeping textual evidence and relational structure in step with each other as reasoning unfolds. It proposes Co-E, a training-free system that maintains a paired text-and-graph working memory and, after each reasoning step, runs a synchronization cycle that consolidates text, adds text-derived triples to the graph, and writes surviving graph facts back into the text that conditions the next step. On six benchmarks, this bidirectional co-evolution — with an 8-billion-parameter open model and no fine-tuning — reaches 72.6 exact match on 2WikiMultiHopQA, 70.0 on Bamboogle, 52.5 on MuSiQue, and 74.9 Hits@1 on CWQ, outperforming comparable open systems and matching several larger or trained ones. If the claim holds, it gives a compute-and-memory recipe for improving reasoning without gradient updates.

Core claim

The central claim is that text and graph memory should co-evolve within each reasoning step, not be retrieved once or accumulated separately. The paper states this as the bidirectional Sync operator (T(t+1),G(t+1)) ← Sync(T(t),G(t),S(t),R(t),y(t)): textual consolidation tightens evidence; text-to-graph propagation extracts and merges new triples, expanding the relational frontier; graph-to-text propagation writes surviving triples back as natural-language statements; and re-consolidation reconciles the result. The authors argue this lets a bridge entity discovered in a passage seed the next graph or corpus query, while graph edges constrain what the generator is asked next. The evidence is t

What carries the argument

The four-operation bidirectional synchronization cycle that mediates text and graph memory. In one phrase: a within-step reconcile-and-propagate operator. It is load-bearing because it is the only component that couples the two memory stores: consolidation prevents memory from becoming an ever-growing noisy block; text-to-graph propagation lets newly discovered entities extend the graph frontier; graph-to-text propagation gives the generator direct, natural-language access to relational constraints; and re-consolidation reconciles both. The memory state M(t)=(T(t),G(t)) conditions the next sub-query, retrieval, and generation, which is why the paper calls the update co-evolution rather than

Load-bearing premise

The load-bearing premise is that the same 8-billion-parameter model's memory operations — consolidation, entity linking, relation extraction, triple pruning, and in search mode the verifier — are reliable enough to be trusted without any independent check; if they are systematically wrong, the claimed co-evolution is just extra calls to a fallible model.

What would settle it

Take the Bamboogle benchmark and replace each memory-synchronization component (consolidator, entity linker, relation extractor, triple pruner, verifier) with a deterministic or oracle version while keeping the rest fixed; if accuracy does not fall, co-evolution is not what is doing the work. Alternatively, give a retrieval-only baseline the same roughly 204 language-model calls per question for query rewriting and reranking without any memory sync; if it matches the 70.0 EM, the margin is compute budget, not the loop.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

If the central claim is right, an 8-billion-parameter open model without any training can outperform larger or trained systems on multi-hop benchmarks, which would lower the cost threshold for strong question answering.
Bidirectional coupling and active maintenance are necessary, not optional: the ablations show removing them costs 8.9–20.5 exact-match points, so the benefit is attributed to the co-evolution loop itself.
Search (MCTS) and sequential reasoning (CoT) both improve from the same memory sync, meaning the mechanism helps single-chain reasoning too, not just tree search.
Because the system still works when the benchmark-provided knowledge subgraph is replaced by full live knowledge-base retrieval (CWQ drops only about 1.5 points), the mechanism transfers beyond preprocessed settings.
The two memory modalities are each doing distinct work: removing either text or graph memory costs about 15 points, so neither alone reproduces the behavior.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The paper reports cost per question but does not run a compute-matched baseline: MCTS uses about 204 language-model calls per question versus about 58 for its own CoT and about 9 for a simple retrieval baseline, so a compute-matched comparison would be needed to fully isolate the co-evolution mechanism from the search budget.
A natural next test is to apply the same paired-memory synchronization to other multi-step agent tasks — tool use, planning, or code repair — since the mechanism itself is not QA-specific; the paper only evaluates QA.
The failure analysis suggests a concrete cheap extension: for the 45% of errors caused by retrieval gaps, cross-checking contested facts against a second independent corpus could recover a portion of the remaining error, at the latency cost the paper notes.
The paper's smaller-backbone run shows only about a 2–3 EM drop on Bamboogle, implying the mechanism's value is at least partly separable from model scale, so the same design could plausibly be pushed to even smaller or faster models.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

A transparent system paper with a plausible new mechanism; the co-evolution story is not yet established because comparisons and ablations don't hold compute constant.

read the letter

Quick take: this is a genuinely well-built agentic-RAG system with an interesting new mechanism, and the paper is unusually honest about its costs and limits. But the central claim — that bidirectional graph-text synchronization, not raw inference budget, drives the gains — is not supported by the current experiments. The stress-test note is on target.

What's new: Co-E maintains paired textual and graph working memory and syncs them in both directions with four operations: consolidate, text→graph, graph→text, re-consolidate. That's a real departure from SubQRAG's monotonic triple accumulation and from text-only memory agents. The system is training-free, uses a small 8B backbone, and ships code. The ablation table on Bamboogle is internally consistent: every piece matters, and removing active maintenance hurts most. The cost table (Table 10) is a model of transparency — 204 LLM calls per question for MCTS vs 9.3 for IR-CoT — and the failure analysis and limitations section are candid about retrieval gaps and shared-memory contamination.

Where it's soft: The main confound is exactly what the note says. MCTS spends ~22× the calls of IR-CoT; the ablations remove sync components but also reduce calls, so the EM drops could reflect a smaller reasoning budget, not the specific contribution of synchronization. There is no budget-matched baseline — neither a same-budget retrieval-only agent nor a same-budget unidirectional memory variant. CoT mode at 58 calls is still ~6× IR-CoT and shows similar directionality, which helps a little, but doesn't resolve it. Other issues: single-dataset ablations (Bamboogle only), no error bars or seeded runs, hand-set hyperparameters with no sensitivity analysis, and an LLM of the same family acting as verifier and judge. The KGQA numbers rely on benchmark-provided candidate subgraphs. These are all addressable, and none on their own kill the paper.

Who should read it: people working on agentic RAG, memory-augmented reasoning, or multi-hop QA evaluation. The cost analysis and failure taxonomy are worth a look regardless of the headline.

Verdict: Worth a serious referee. I'd ask for budget-matched baselines, multi-run statistics, and hyperparameter sensitivity before trusting the mechanism claim. That's a substantial revision, not a desk reject.

Referee Report

3 major / 5 minor

Summary. The paper proposes Co-E, a training-free multi-hop QA system built around a shared working memory with paired textual and graph stores. At each reasoning step, Co-E generates a sub-query, retrieves text snippets and KB triples, produces an intermediate answer, and applies a bidirectional synchronization operator (Eq. 2) that consolidates textual memory, extracts and merges open-vocabulary triples into graph memory, injects surviving graph facts back into textual memory, and re-consolidates. Two reasoning modes are presented: MCTS and CoT. The system is evaluated on two KGQA benchmarks (WebQSP, CWQ) and four text-QA benchmarks (HotpotQA, 2WikiMultiHopQA, MuSiQue, Bamboogle). The best configuration, Co-E (MCTS), achieves 85.5 Hits@1 on WebQSP, 74.9 on CWQ, 72.6 EM on 2WikiMultiHopQA, 70.0 on Bamboogle, and 52.5 on MuSiQue, outperforming open ≤8B training-free baselines. Ablations on Bamboogle (Table 3) attribute the gains to bidirectional synchronization, both memory modalities, and active maintenance. The paper also includes a cost analysis (Appendix C), a failure analysis (Appendix D), and full prompt templates (Appendix E).

Significance. If the central claim is accepted, Co-E is the strongest open ≤8B training-free system on several multi-hop QA benchmarks and provides a reusable memory-management mechanism that goes beyond static retrieval or monotonic memory accumulation. The paper has notable strengths: headline metrics are exact-match against gold labels (EM, Hits@1), so the main results are not self-referential; the codebase is released; per-role inference costs are reported transparently; and the failure analysis is honest about the system's inability to correct wrong corpus evidence. These features make the contribution easy to audit. However, the causal interpretation of the headline gains is currently weakened by a large inference-budget gap relative to baselines, by budget-confounded ablations, and by an unvalidated LLM-based search reward in MCTS mode. The framework is promising, but the evidence as presented does not yet establish that co-evolution, rather than additional compute, is the decisive factor.

major comments (3)

[§4.3, Appendix C, Table 10] The headline comparisons in Table 2 are not budget-matched. Appendix C reports Co-E (MCTS) uses 204.1 LLM calls and 27.0K output tokens per Bamboogle question, versus 9.3 calls and 1.2K tokens for IR-CoT, roughly a 22x gap. The EM gains in §4.3 are attributed to synchronized graph-text memory, but no baseline is given a comparable inference budget, and no Co-E variant with the same call/token budget but unidirectional or absent synchronization is reported. A retrieval-only agent with ~200 independent query–retrieve–answer steps, or a Co-E variant with a matched token budget but no memory updates, is necessary to separate 'co-evolution' from 'more compute.' Without such a control, the margin over IR-CoT, HopRAG, and other training-free baselines can be explained by compute alone.
[§4.4, Table 3] The ablations that motivate the central mechanism remove LLM calls, so the EM drops conflate mechanism removal with budget reduction. Removing bidirectional sync eliminates graph-to-text injection and re-consolidation; removing active maintenance eliminates consolidation, triple pruning, and related LLM calls; both variants necessarily use a smaller reasoning/retrieval budget. The largest drop, −20.5 EM for removing active maintenance in MCTS mode, may therefore reflect reduced compute rather than the specific value of maintenance. The table is also single-dataset (Bamboogle) with no confidence intervals. Please report per-variant LLM call and output-token counts, and add compute-matched ablations on at least one additional dataset, e.g., a variant with unidirectional or no memory updates but with extra retrieval/synthesis calls to match the full system's budget.
[§3.5, Appendix E.5.5] The MCTS reward is produced by a Qwen3-8B verifier, the same model family that generates the candidate answers, and no analysis of verifier reliability is provided. Because this reward selects branches and weights final candidates, the MCTS-vs-CoT gains could partly reflect self-consistency or stylistic biases rather than evidence quality. The provenance-audit step in E.4.1 mitigates contamination after the fact, but it does not validate the reward itself. Please include a verifier reliability study: compare verifier scores against gold labels on a sample, report calibration or agreement, and show examples where verifier-high-scoring candidates are nevertheless wrong. A concrete test such as replacing the verifier with an independent judge or with gold-label rewards would clarify how much of the MCTS improvement depends on the self-assessment signal.

minor comments (5)

[Table 8] Many hyperparameters are fixed without sensitivity analysis: MCTS PUCT exploration weight c, node priors, graph hops k, triple-pruner top-k, early-stopping thresholds, retrieval caps. At least one sensitivity experiment (e.g., varying c and k over a small range) would strengthen the claim that the mechanism is robust rather than tuned to these values.
[§B.1] For KGQA, entity linking uses a dictionary derived from the benchmark training set. Please clarify whether this dictionary contains entities or aliases that appear in the test questions or gold answers, and, if so, compare with a fully unsupervised linking baseline to rule out a benchmark-specific advantage.
[Figure 2] The Pass@k plot mixes tree expansions for MCTS with chain depth for CoT. The x-axis is therefore not directly comparable across the two curves. Please label the axes explicitly and consider a shared notion of 'reasoning step' or 'LLM call'.
[§6 and Appendix E.6] The supplementary Acc metric uses Qwen3-30B-A3B-Thinking as a judge, which is another member of the same model family as the backbone. It is presented as supplementary and offline, but the limitation should be stated where Acc is first reported.
[Throughout] Minor language issues: 'quering' in §2 should be 'querying'; 'Due to its significant smaller size' in §4.5 should be 'significantly smaller size'. The reference list contains many arXiv preprints and 2026-dated items; final publication should verify official versions where available.

Circularity Check

0 steps flagged

No significant circularity: headline EM/Hits@1 are exact-match against gold; no fitted parameter or self-citation chain makes the reported gains equivalent to Co-E's own memory operations.

full rationale

Co-E's central empirical claim is that the bidirectional Sync operator (Eq. 2) improves multi-hop QA. The headline metrics are exact-match against gold labels: 'For text-QA, we report Exact Match (EM)' and 'a prediction is correct if the top answer matches any gold alias after normalization' for KGQA, so correctness is not defined by the system's own memory states, verifier ratings, or LLM self-assessments. The method has no fitted parameters: Co-E is 'training-free' and every memory operation is a prompted LLM role, not an optimized parameter. The ablations in Table 3 remove components but are not budget-matched, and Appendix C concedes that MCTS uses 204.1 calls/question vs 9.3 for IR-CoT; that is a compute/validity confound, not a circular reduction. The MCTS verifier is the same LLM family as the generator, and the supplementary Acc judge is Qwen3-30B, but the judge is given the gold answer and is explicitly 'not part of the Co-E inference pipeline,' and the main tables do not depend on it. No load-bearing self-citation exists: the reference list contains no prior work by the present authors; MCTS node types and PUCT are attributed to external works (Qi et al., 2024; Silver et al., 2017). I found no equation or fitted value that is equivalent by construction to the reported predictions.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 2 invented entities

The central claim rests on no fitted mathematical constants, but on a set of hand-set hyperparameters (Table 8) and on domain assumptions about LLM component reliability, benchmark-provided subgraphs, and corpus/KB evidence coverage — several of which the paper itself partly undermines in its failure analysis and Limitations section.

free parameters (6)

MCTS PUCT exploration weight c = 2.5
PUCT constant in Eq. 3, hand-set; no sensitivity analysis reported (Table 8).
MCTS node priors P(s,a) = 0.60 / 0.50 / 0.45 / 0.40 / 0.30
Priors for sub-question/answer, self-correction, synthesis, question-rephrasing, and final-answer node types; chosen by hand, tuning not documented (Table 8).
Graph hops k = 2
SPARQL expansion depth for graph retrieval; fixed across all benchmarks (Table 8).
Triple pruner top-k = 128
Cap on candidate triples for graph retrieval and sync pruning (Table 8).
Early stopping thresholds = min iterations 5, high-confidence 0.9, convergence patience 5, semantic sufficiency 5
MCTS early-stopping and branch-confidence criteria (Table 8); no sensitivity analysis.
Retrieval caps (reranker/web/entity candidates) = 10 / 5 / 3
Reranker top-k 10, web search top-k 5, entity candidates per mention 3 (Table 8).

axioms (5)

domain assumption Qwen3-8B, prompted with JSON schemas and without training, can reliably perform memory consolidation, deduplication, contradiction retraction, NER/linking, relation extraction, and triple pruning as specified in Appendices E.2–E.4.
The entire synchronization benefit (ablations: −8.9 to −20.5 EM when sync components are removed, Table 3) rests on LLM component reliability; no independent evaluation of component-level accuracy is provided (§3.3).
domain assumption The benchmark-provided Freebase topic subgraphs for WebQSP/CWQ contain the answer-relevant relations; Co-E is restricted to them.
KGQA headline numbers (85.5/74.9) depend on preprocessed candidate graphs, and the Full-Wikidata rerun drops to 78.6/73.4 (§4.5, Table 4); the provenance of these candidate subgraphs is not described (Appendix B.1).
domain assumption The Wiki23 corpus plus live-Wikidata supplies the evidence needed for 2WikiMultiHopQA, HotpotQA, MuSiQue, and Bamboogle.
The failure analysis attributes 45% of Bamboogle errors to retrieval gaps where the corpus supports an incorrect value (§4.6, Table 11, Limitations) — the system cannot fix absent or wrong evidence.
standard math The PUCT selection formula (Eq. 3) is a valid exploration-exploitation rule for LLM MCTS.
Adopted from Silver et al. 2017; standard and uncontroversial.
domain assumption LLM-judged accuracy (Qwen3-30B) correlates with true correctness.
Supplementary Acc metric judged by a model from the same family as the reasoning backbone (Appendix E.6); no human-agreement measurement is reported.

invented entities (2)

Shared synchronized graph-text working memory M(t) independent evidence
purpose: Coupled textual (T) and graph (G) stores updated by the Sync operator after each reasoning step (Eqs. 1–2).
Software artifact, not a physical entity; its causal role is evidenced by ablations (removing sync or either store drops Bamboogle EM by 8.9–20.5 points, Table 3) and by the released code (footnote 1).
Open-vocabulary triples with natural-language relation labels independent evidence
purpose: Represent text-derived relations absent from KB schema in graph memory (§3.2).
Representational choice instantiated in the code and in the relation-extraction prompt (Appendix E.4.3).

pith-pipeline@v1.3.0-alltime-deepseek · 22770 in / 21549 out tokens · 194711 ms · 2026-07-31T23:52:16.743634+00:00 · methodology

0 comments

read the original abstract

Multi-hop question answering requires coordinating relational and textual evidence across reasoning steps, a combination neither a text corpus nor a knowledge graph can supply alone. Prior work often emphasizes only part of this loop: graph-augmented RAG retrieves from a pre-built or query-updated graph, KGQA systems search within topic-centered subgraphs, and memory-augmented agents maintain evolving memories without continuously reconciling graph memory with textual context. We propose Co-E, a training-free system built around synchronized bidirectional graph-text working memory. A synchronization cycle consolidates textual memory, extracts relational triples into graph memory, and injects graph facts back into the generation context. Because both memories are maintained, they shape subsequent retrieval and generation. Evaluated on six multi-hop QA benchmarks, Co-E improves over comparable training-free open-backbone baselines and is competitive with larger or trained systems.

Figures

Figures reproduced from arXiv: 2607.23278 by Hieu Man, Thien Huu Nguyen.

**Figure 2.** Figure 2: Pass@k on Bamboogle; k is the number of tree expansions for MCTS and chain depth for CoT. evidence into both memories. HotpotQA has the smallest margin: Co-E reaches 61.8 EM, ahead of SubQRAG and ProGraph-R1 but close to HopRAGGPT-4o. HotpotQA often has stronger lexical cues and shorter chains, making corpus-only or closedsource pipelines more competitive. 4.4 Ablation Study [PITH_FULL_IMAGE:figures/ful… view at source ↗

discussion (0)

Reference graph

Works this paper leans on

67 extracted references · 2 linked inside Pith

[1]

In CoT mode, this prompt also decides whether the question is already answerable from memory

to generate a focused sub-queryq(t) from the cur- rent memory M(t) and question q. In CoT mode, this prompt also decides whether the question is already answerable from memory. In MCTS mode, it serves as theSub-question/Answernode type. Request Generation Prompt You are an expert assistant for multi-hop question answering and reasoning decomposition. Deci...
[2]

Shuwen Xu, Yao Xu, Jiaxiang Liu, Chenhao Yuan, Wen- shuo Peng, Jun Zhao, and Kang Liu

Mcts-kbqa: Monte carlo tree search for knowledge base question answering.Preprint, arXiv:2502.13428. Shuwen Xu, Yao Xu, Jiaxiang Liu, Chenhao Yuan, Wen- shuo Peng, Jun Zhao, and Kang Liu. 2026. Graph- walker: Agentic knowledge graph question answer- ing via synthetic trajectory curriculum.Preprint, arXiv:2603.28533. Wujiang Xu, Zujie Liang, Kai Mei, Hang ...

Pith/arXiv arXiv 2026
[3]

Identify missing knowledge: only gaps that meaningfully advance reasoning toward the answer
[4]

Analyze the main question: identify core intent, key entities, constraints, and required reasoning steps
[5]

Check the context: if sufficient to answer, set`is_answerable`to true and stop
[6]

Fallback Queries: add 1-2 fallback queries using common aliases or alternative phrasings so that retrieval succeeds if the primary phrasing returns no results. ## Output Format Respond with a JSON object with exactly these keys: - queries: array of strings E.2.2 Extractor Filters each retrieved document into self-contained, question-relevant snippets that...
[7]

Oregon Trail/MECC (1971)

Generate subquestions: - For parallel gaps: each must be independently answerable. - For chained hops: generate in sequential order; use` Category % Representative example Retrieval gap 44.8 "Oregon Trail/MECC (1971)" is the longest-running franchise; corpus attributed it to Super Mario/Nintendo Incomplete chain 20.7 "Father of father of computer science"...

1971
[8]

Keep at most 3

Validate: remove subquestions answerable from context, redundant, or low-value. Keep at most 3. ## Scope Consistency Preserve the geographic or categorical scope of the main question. Do NOT silently narrow a global scope to a specific region without justification. ## Output Format Respond with a JSON object with exactly these keys: - is_answerable: boole...
[9]

Self-Contained: Each query understandable without original input
[10]

Atomic: One single fact per query
[11]

## Instructions

Essential & Non-Redundant: Every query necessary and unique. ## Instructions
[12]

Parse the Input: identify type (factual, comparative, causal, temporal), key entities, and required reasoning steps
[13]

Generate Strategic Queries: formulate queries that collectively cover all necessary information to answer the input
[14]

Ensure Self-Containment: each query must be independently answerable
[15]

Review for Completeness and Non- Redundancy
[16]

current",

Temporal Grounding: if the input contains "current", "now", or a present-tense superlative, add "as of [ current year]" to at least one query
[18]

Question Deconstruction: identify primary subject, key entities, and specific information sought
[19]

Be liberal and inclusive in this initial pass

Candidate Identification: identify and quote ALL passages potentially related to concepts in the question. Be liberal and inclusive in this initial pass
[20]

Relevance Evaluation: assess each quote against criteria (directly answering, contextual, supporting evidence, etc.)
[21]

Add context for clarity but preserve original meaning

Extraction: extract ALL relevant information. Add context for clarity but preserve original meaning. Each extracted item must be FULLY UNDERSTANDABLE on its own
[22]

Final evaluation: examine each item for self-containment and relevance; rewrite or remove as needed. ## Output Format Respond with a JSON object with exactly these keys: - relevant_information: array of strings E.3 Response Generation Used in stage 3 of each reasoning iteration (§ 3.1) to produce an intermediate answer from the current memory augmented wi...
[23]

Analyze the question: identify core intent, key entities, and specific information sought
[24]

Only use your own knowledge when context is absent or clearly incomplete, and explicitly state when doing so

Context priority: when context is provided, ground your answer exclusively in the context, do not introduce external facts. Only use your own knowledge when context is absent or clearly incomplete, and explicitly state when doing so
[25]

State any assumptions clearly

Synthesize a clear, well-reasoned answer. State any assumptions clearly. ## Output Format Respond with a JSON object with exactly these keys: - answer: string - concise_answer: string - reasoning: string - confidence_level: string E.4 Memory Updating E.4.1 Memory Consolidation Runs twice per synchronization cycle (§ 3.3): once after evidence retrieval (pr...
[26]

Question Analysis: identify primary subject, key entities, and information sought
[27]

No pronouns or references to external context

Memory Atomization: atomize memory into atomic, self-contained items. No pronouns or references to external context
[28]

If one item is completely contained in another, remove it

Deduplication: if two items have the same content, keep one. If one item is completely contained in another, remove it
[29]

Retrieval

Relevance Evaluation: keep items that contain ANY clue that could help answer the question. 4b. Provenance Audit: for each [System Prediction] item, check if any [Retrieval] item covers the same claim. If contradicted -> remove the [System Prediction] item. If supported -> upgrade provenance to "Retrieval". 4c. Hop Depth Filtering: items tagged [hop=N ] w...
[30]

Irrelevant Information Removal
[31]

If two [Retrieval] items conflict, keep both and note the conflict

Conflict Resolution: when a [Retrieval] item and a [System Prediction] item state conflicting specific facts, ALWAYS keep [Retrieval] and discard [System Prediction]. If two [Retrieval] items conflict, keep both and note the conflict
[32]

Refinement: ensure each item is self- contained and clear
[33]

System Prediction

Final check: verify every kept item is self-contained, non-redundant, and has correct provenance. ## Output Format Respond with a JSON object with exactly these keys: - consolidated_memory: array of objects; each object has: - content: string - provenance: string, exactly "System Prediction" or "Retrieval" - hop_depth: integer or null E.4.2 Named Entity R...
[34]

If text is a question, focus ONLY on entities which are main clues to answer the question
[35]

Define precise boundaries (include modifiers)
[36]

Handle ambiguity using context
[37]

Extract unique entities only once ( deduplicate by real-world identity). ## Output Format Respond with a JSON object with exactly these keys: - entities: array of objects; each object has: - id: string or null - name: string - description: string or null E.4.3 Relation Extraction Extracts open-vocabulary (subject, relation, object) triples from the consol...
[38]

Identify entity pairs with direct relationships
[39]

Break down complex relationships into simpler ones
[40]

Only extract explicitly stated or strongly implied relationships
[41]

Use clear, concise, active-voice relation types
[42]

## Canonical Relation Direction Always use the ACTIVE form

Ensure relations are self-contained and non-duplicated. ## Canonical Relation Direction Always use the ACTIVE form. Never use passive or inverse forms Rules, convert to active form when a predicate:
[43]

ends in "_of" -> invert: has_child, contains, has_capital
[44]

starts with "is_" -> invert to active form
[45]

Applied as the LLM stage of the two-stage pruner (cross-encoder reranker followed by this LLM filter)

ends in "_by" -> invert: precedes, directed, owns, succeeds ## Output Format Respond with a JSON object with exactly these keys: - relations: array of objects; each object has: - subject: string - subject_id: string or null - relation: string - object: string - object_id: string or null - context: string or null E.4.4 Triple Pruner Filters extracted and r...
[46]

What is the capital of France?

Direct relevance: both subject AND object are directly related to the question, and the relation connects them in a way that helps answer it. - Example Q: "What is the capital of France?" Keep: (France, capital, Paris) Drop: (France, borders, Germany)
[47]

Who is the spouse of the president of France?

Chain relevance: the triple forms part of a reasoning chain with another kept triple. One entity of this triple must match an entity in another relevant triple, and together they help answer the question. - Example Q: "Who is the spouse of the president of France?" Keep: (France, president, Macron) + ( Macron, spouse, Brigitte) ## Key Rule Do NOT keep a t...
[48]

Parse question requirements
[49]

Extract relevant facts from context
[50]

Evaluate proposed answer as: CORRECT / PARTIAL / INCORRECT / UNSUPPORTED
[51]

Generate refined answer. ## Output Format Respond with a JSON object with exactly these keys: - status: string, one of: correct, partial, incorrect, unsupported - refined_answer: string - confidence_level: string E.5.2 Question Rephrasing Node Question Rephrasing Prompt You are a Question Refiner that transforms unclear questions into precise, clear quest...
[52]

Clarity First: eliminate ambiguity and jargon
[53]

Preserve Intent: do not alter the core inquiry
[54]

Enhance Answerability: make specific and self-contained. ## Output Format Respond with a JSON object with exactly these keys: - rephrased_question: string E.5.3 Synthesis Node Synthesis Prompt You are a specialized AI for multi-step reasoning. Perform a single, focused reasoning step by analyzing context and producing a consolidated synthesis. ## Instructions
[55]

Analyze the main question objective
[56]

Review all information in context
[57]

If context is sufficient to directly answer, state this and formulate the definitive answer
[58]

Otherwise, synthesize new thoughts that advance reasoning: - Causal or temporal links - Core relationship identification - Progress summary - Contradiction identification - Hypothesis formulation ## Critical Constraints
[59]

No External Information: do NOT introduce facts not in the context
[60]

No New Questions: synthesize, do not query. ## Output Format Respond with a JSON object with exactly these keys: - is_answerable: boolean - step_conclusion: string - confidence_level: string E.5.4 Final Answer Synthesis Synthesizes the final answer from all candidate terminal nodes, weighted by their MCTS reward scores. Final Answer Synthesis Prompt You a...
[61]

High-score (>=0.5) + [Retrieval] corroboration
[62]

Convergent positive-score candidates
[63]

[Retrieval] facts regardless of candidate score
[64]

Logically sound reasoning consistent with graph triples
[65]

Revise if not

Majority agreement as last resort Phase III, Synthesis & Self-Critique: verify every factual claim is traceable to at least one reliable source. Revise if not. ## Output Format Respond with a JSON object with exactly these keys: - final_answer: string - concise_answer: string - reasoning: string - confidence_level: string, one of: high, medium, low, uncer...
[66]

- Otherwise, verify accuracy using your knowledge

Correctness (60%): Is the information factually accurate? - If correct_answer is provided and system_answer matches, award 10.0. - Otherwise, verify accuracy using your knowledge
[67]

## Output Format Respond with a JSON object with exactly these keys: - rating: number, float from 0.0 to 10.0 - reasoning: string

Helpfulness & Relevance (40%): Does it address the user's core need? Scoring Guidelines: - 9.0-10.0: Correct and comprehensive - 7.0-8.9: Mostly correct with minor issues - 5.0-6.9: Partially addresses or has accuracy concerns - 3.0-4.9: Significant correctness or relevance issues - 0.0-2.9: Incorrect or completely off- topic Uncertainty rule: if you are ...
[2025]

InFindings of the Association for Com- putational Linguistics: ACL 2025, pages 9288–9306, Vienna, Austria

ReKG-MCTS: Reinforcing LLM reasoning on knowledge graphs via training-free Monte Carlo tree search. InFindings of the Association for Com- putational Linguistics: ACL 2025, pages 9288–9306, Vienna, Austria. Association for Computational Lin- guistics. Jiashuo Sun, Chengjin Xu, Lumingyuan Tang, Saizhuo Wang, Chen Lin, Yeyun Gong, Lionel M. Ni, Heung- Yeung...

Pith/arXiv arXiv 2025

[1] [1]

In CoT mode, this prompt also decides whether the question is already answerable from memory

to generate a focused sub-queryq(t) from the cur- rent memory M(t) and question q. In CoT mode, this prompt also decides whether the question is already answerable from memory. In MCTS mode, it serves as theSub-question/Answernode type. Request Generation Prompt You are an expert assistant for multi-hop question answering and reasoning decomposition. Deci...

[2] [2]

Shuwen Xu, Yao Xu, Jiaxiang Liu, Chenhao Yuan, Wen- shuo Peng, Jun Zhao, and Kang Liu

Mcts-kbqa: Monte carlo tree search for knowledge base question answering.Preprint, arXiv:2502.13428. Shuwen Xu, Yao Xu, Jiaxiang Liu, Chenhao Yuan, Wen- shuo Peng, Jun Zhao, and Kang Liu. 2026. Graph- walker: Agentic knowledge graph question answer- ing via synthetic trajectory curriculum.Preprint, arXiv:2603.28533. Wujiang Xu, Zujie Liang, Kai Mei, Hang ...

Pith/arXiv arXiv 2026

[3] [3]

Identify missing knowledge: only gaps that meaningfully advance reasoning toward the answer

[4] [4]

Analyze the main question: identify core intent, key entities, constraints, and required reasoning steps

[5] [5]

Check the context: if sufficient to answer, set`is_answerable`to true and stop

[6] [6]

Fallback Queries: add 1-2 fallback queries using common aliases or alternative phrasings so that retrieval succeeds if the primary phrasing returns no results. ## Output Format Respond with a JSON object with exactly these keys: - queries: array of strings E.2.2 Extractor Filters each retrieved document into self-contained, question-relevant snippets that...

[7] [7]

Oregon Trail/MECC (1971)

Generate subquestions: - For parallel gaps: each must be independently answerable. - For chained hops: generate in sequential order; use` Category % Representative example Retrieval gap 44.8 "Oregon Trail/MECC (1971)" is the longest-running franchise; corpus attributed it to Super Mario/Nintendo Incomplete chain 20.7 "Father of father of computer science"...

1971

[8] [8]

Keep at most 3

Validate: remove subquestions answerable from context, redundant, or low-value. Keep at most 3. ## Scope Consistency Preserve the geographic or categorical scope of the main question. Do NOT silently narrow a global scope to a specific region without justification. ## Output Format Respond with a JSON object with exactly these keys: - is_answerable: boole...

[9] [9]

Self-Contained: Each query understandable without original input

[10] [10]

Atomic: One single fact per query

[11] [11]

## Instructions

Essential & Non-Redundant: Every query necessary and unique. ## Instructions

[12] [12]

Parse the Input: identify type (factual, comparative, causal, temporal), key entities, and required reasoning steps

[13] [13]

Generate Strategic Queries: formulate queries that collectively cover all necessary information to answer the input

[14] [14]

Ensure Self-Containment: each query must be independently answerable

[15] [15]

Review for Completeness and Non- Redundancy

[16] [16]

current",

Temporal Grounding: if the input contains "current", "now", or a present-tense superlative, add "as of [ current year]" to at least one query

[17] [18]

Question Deconstruction: identify primary subject, key entities, and specific information sought

[18] [19]

Be liberal and inclusive in this initial pass

Candidate Identification: identify and quote ALL passages potentially related to concepts in the question. Be liberal and inclusive in this initial pass

[19] [20]

Relevance Evaluation: assess each quote against criteria (directly answering, contextual, supporting evidence, etc.)

[20] [21]

Add context for clarity but preserve original meaning

Extraction: extract ALL relevant information. Add context for clarity but preserve original meaning. Each extracted item must be FULLY UNDERSTANDABLE on its own

[21] [22]

Final evaluation: examine each item for self-containment and relevance; rewrite or remove as needed. ## Output Format Respond with a JSON object with exactly these keys: - relevant_information: array of strings E.3 Response Generation Used in stage 3 of each reasoning iteration (§ 3.1) to produce an intermediate answer from the current memory augmented wi...

[22] [23]

Analyze the question: identify core intent, key entities, and specific information sought

[23] [24]

Only use your own knowledge when context is absent or clearly incomplete, and explicitly state when doing so

Context priority: when context is provided, ground your answer exclusively in the context, do not introduce external facts. Only use your own knowledge when context is absent or clearly incomplete, and explicitly state when doing so

[24] [25]

State any assumptions clearly

Synthesize a clear, well-reasoned answer. State any assumptions clearly. ## Output Format Respond with a JSON object with exactly these keys: - answer: string - concise_answer: string - reasoning: string - confidence_level: string E.4 Memory Updating E.4.1 Memory Consolidation Runs twice per synchronization cycle (§ 3.3): once after evidence retrieval (pr...

[25] [26]

Question Analysis: identify primary subject, key entities, and information sought

[26] [27]

No pronouns or references to external context

Memory Atomization: atomize memory into atomic, self-contained items. No pronouns or references to external context

[27] [28]

If one item is completely contained in another, remove it

Deduplication: if two items have the same content, keep one. If one item is completely contained in another, remove it

[28] [29]

Retrieval

Relevance Evaluation: keep items that contain ANY clue that could help answer the question. 4b. Provenance Audit: for each [System Prediction] item, check if any [Retrieval] item covers the same claim. If contradicted -> remove the [System Prediction] item. If supported -> upgrade provenance to "Retrieval". 4c. Hop Depth Filtering: items tagged [hop=N ] w...

[29] [30]

Irrelevant Information Removal

[30] [31]

If two [Retrieval] items conflict, keep both and note the conflict

Conflict Resolution: when a [Retrieval] item and a [System Prediction] item state conflicting specific facts, ALWAYS keep [Retrieval] and discard [System Prediction]. If two [Retrieval] items conflict, keep both and note the conflict

[31] [32]

Refinement: ensure each item is self- contained and clear

[32] [33]

System Prediction

Final check: verify every kept item is self-contained, non-redundant, and has correct provenance. ## Output Format Respond with a JSON object with exactly these keys: - consolidated_memory: array of objects; each object has: - content: string - provenance: string, exactly "System Prediction" or "Retrieval" - hop_depth: integer or null E.4.2 Named Entity R...

[33] [34]

If text is a question, focus ONLY on entities which are main clues to answer the question

[34] [35]

Define precise boundaries (include modifiers)

[35] [36]

Handle ambiguity using context

[36] [37]

Extract unique entities only once ( deduplicate by real-world identity). ## Output Format Respond with a JSON object with exactly these keys: - entities: array of objects; each object has: - id: string or null - name: string - description: string or null E.4.3 Relation Extraction Extracts open-vocabulary (subject, relation, object) triples from the consol...

[37] [38]

Identify entity pairs with direct relationships

[38] [39]

Break down complex relationships into simpler ones

[39] [40]

Only extract explicitly stated or strongly implied relationships

[40] [41]

Use clear, concise, active-voice relation types

[41] [42]

## Canonical Relation Direction Always use the ACTIVE form

Ensure relations are self-contained and non-duplicated. ## Canonical Relation Direction Always use the ACTIVE form. Never use passive or inverse forms Rules, convert to active form when a predicate:

[42] [43]

ends in "_of" -> invert: has_child, contains, has_capital

[43] [44]

starts with "is_" -> invert to active form

[44] [45]

Applied as the LLM stage of the two-stage pruner (cross-encoder reranker followed by this LLM filter)

ends in "_by" -> invert: precedes, directed, owns, succeeds ## Output Format Respond with a JSON object with exactly these keys: - relations: array of objects; each object has: - subject: string - subject_id: string or null - relation: string - object: string - object_id: string or null - context: string or null E.4.4 Triple Pruner Filters extracted and r...

[45] [46]

What is the capital of France?

Direct relevance: both subject AND object are directly related to the question, and the relation connects them in a way that helps answer it. - Example Q: "What is the capital of France?" Keep: (France, capital, Paris) Drop: (France, borders, Germany)

[46] [47]

Who is the spouse of the president of France?

Chain relevance: the triple forms part of a reasoning chain with another kept triple. One entity of this triple must match an entity in another relevant triple, and together they help answer the question. - Example Q: "Who is the spouse of the president of France?" Keep: (France, president, Macron) + ( Macron, spouse, Brigitte) ## Key Rule Do NOT keep a t...

[47] [48]

Parse question requirements

[48] [49]

Extract relevant facts from context

[49] [50]

Evaluate proposed answer as: CORRECT / PARTIAL / INCORRECT / UNSUPPORTED

[50] [51]

Generate refined answer. ## Output Format Respond with a JSON object with exactly these keys: - status: string, one of: correct, partial, incorrect, unsupported - refined_answer: string - confidence_level: string E.5.2 Question Rephrasing Node Question Rephrasing Prompt You are a Question Refiner that transforms unclear questions into precise, clear quest...

[51] [52]

Clarity First: eliminate ambiguity and jargon

[52] [53]

Preserve Intent: do not alter the core inquiry

[53] [54]

Enhance Answerability: make specific and self-contained. ## Output Format Respond with a JSON object with exactly these keys: - rephrased_question: string E.5.3 Synthesis Node Synthesis Prompt You are a specialized AI for multi-step reasoning. Perform a single, focused reasoning step by analyzing context and producing a consolidated synthesis. ## Instructions

[54] [55]

Analyze the main question objective

[55] [56]

Review all information in context

[56] [57]

If context is sufficient to directly answer, state this and formulate the definitive answer

[57] [58]

Otherwise, synthesize new thoughts that advance reasoning: - Causal or temporal links - Core relationship identification - Progress summary - Contradiction identification - Hypothesis formulation ## Critical Constraints

[58] [59]

No External Information: do NOT introduce facts not in the context

[59] [60]

No New Questions: synthesize, do not query. ## Output Format Respond with a JSON object with exactly these keys: - is_answerable: boolean - step_conclusion: string - confidence_level: string E.5.4 Final Answer Synthesis Synthesizes the final answer from all candidate terminal nodes, weighted by their MCTS reward scores. Final Answer Synthesis Prompt You a...

[60] [61]

High-score (>=0.5) + [Retrieval] corroboration

[61] [62]

Convergent positive-score candidates

[62] [63]

[Retrieval] facts regardless of candidate score

[63] [64]

Logically sound reasoning consistent with graph triples

[64] [65]

Revise if not

Majority agreement as last resort Phase III, Synthesis & Self-Critique: verify every factual claim is traceable to at least one reliable source. Revise if not. ## Output Format Respond with a JSON object with exactly these keys: - final_answer: string - concise_answer: string - reasoning: string - confidence_level: string, one of: high, medium, low, uncer...

[65] [66]

- Otherwise, verify accuracy using your knowledge

Correctness (60%): Is the information factually accurate? - If correct_answer is provided and system_answer matches, award 10.0. - Otherwise, verify accuracy using your knowledge

[66] [67]

## Output Format Respond with a JSON object with exactly these keys: - rating: number, float from 0.0 to 10.0 - reasoning: string

Helpfulness & Relevance (40%): Does it address the user's core need? Scoring Guidelines: - 9.0-10.0: Correct and comprehensive - 7.0-8.9: Mostly correct with minor issues - 5.0-6.9: Partially addresses or has accuracy concerns - 3.0-4.9: Significant correctness or relevance issues - 0.0-2.9: Incorrect or completely off- topic Uncertainty rule: if you are ...

[67] [2025]

InFindings of the Association for Com- putational Linguistics: ACL 2025, pages 9288–9306, Vienna, Austria

ReKG-MCTS: Reinforcing LLM reasoning on knowledge graphs via training-free Monte Carlo tree search. InFindings of the Association for Com- putational Linguistics: ACL 2025, pages 9288–9306, Vienna, Austria. Association for Computational Lin- guistics. Jiashuo Sun, Chengjin Xu, Lumingyuan Tang, Saizhuo Wang, Chen Lin, Yeyun Gong, Lionel M. Ni, Heung- Yeung...

Pith/arXiv arXiv 2025