Pith. sign in

REVIEW 5 major objections 5 minor 50 references

Harness-G: A Graph-Structured Harness for Search Agents

T0 review · 5 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that RL search agents fail because free-form query generation collapses retrieval diversity, and that replacing it with finite action selection over a graph-structured menu restores learning.

desk verdict A genuinely new interface idea for RL search agents, with a clean diagnostic and solid controlled ablations, but the headline numbers rest on unmatched baselines and an unquantified menu-coverage assumption. read the letter →

arxiv 2607.27652 v2 pith:574VEQUU submitted 2026-07-30 cs.CL

Yanning Hou , Haoyuan Chen , Sihang Zhou , Xiaoshu Chen , Xirui Liu , Duanyang Yuan , Lingyuan Meng , Siwei Wang
show 2 more authors
Quan Liu Jian Huang
This is my paper · ORCID
classification cs.CL
keywords reinforcementlearningretrieval-augmentedgenerationmulti-hopquestionansweringactionspacedesigncreditassignmentgraph-structuredretrievalpolicyoptimizationsearchagents
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper identifies a structural flaw in how reinforcement learning (RL) search agents interact with retrieval: letting the policy generate free-form natural-language queries creates 'retrieval-equivalence collapse,' where distinct query strings yield increasingly overlapping evidence sets, so rollout groups lose the contrast that group-relative optimization needs. To fix this, the authors propose Harness-G, which reframes the retrieval action as finite selection from a menu over a paragraph–sentence–entity graph: the policy selects an evidence sentence, looks up an entity, or chooses to answer, while the environment builds the menu and executes deterministic retrieval. On top of this interface, they introduce Structured Non-myopic Credit (SNC), which scores each chosen action against same-state alternatives using a frozen answer scorer and propagates downstream gains to earlier enabling actions. Across six QA benchmarks, Harness-G reports the highest average F1 at both evaluated model scales, beating the strongest baseline by 10.74 points at 1.5B and 3.98 points at 3B. The broader claim is that action-space design — not just reward design — is a primary lever for training search agents.

What carries the argument

The load-bearing mechanism is the paragraph–sentence–entity graph, built programmatically (no generative LLM) via sentence splitting, lightweight NER, and dense-embedding entity linking. At decision time the environment exposes a bounded action menu drawn from this graph: Select commits a visible sentence as evidence; Lookup follows an entity and deterministically constructs the retrieval query from the question plus committed evidence; Answer_With commits visible sentences and terminates; Answer ends the episode. Three properties carry the argument: finiteness (bounded menu size), verifiability (invalid actions are filtered before the policy sees them), and previewability (any candidate act

What would settle it

Construct a test set where the gold answer's key entity is never mentioned in any sentence that the graph links to the question's initial candidates; Harness-G's recall should drop to zero for those items, directly testing the coverage assumption.

Watch

Extended reading notes

Core claim

Harness-G's central discovery is that retrieval aliasing is a structural cause of poor RL search-agent training. When a policy expresses retrieval intent as free-form query strings, many strings map to the same retrieved evidence, so nominally diverse rollouts become utility-equivalent with respect to retrieval decisions; the paper measures query-form diversity staying high while retrieval-outcome diversity collapses below 10% by step 30. The paper argues this 'retrieval-equivalence collapse' is not fixed by denser credit signals, because those leave the many-to-one string-to-retrieval mapping intact. Harness-G instead redefines the action space: a programmatic graph exposes a finite, verifi

Load-bearing premise

The menu can only surface evidence that the programmatic graph and initial retrieval already placed on it; if the gold entity or evidence sentence is not among the menu targets, no policy action can retrieve it.

Editorial extensions

If this is right

  • If free-form query generation is the bottleneck, then existing and future search-agent RL pipelines should expect retrieval-equivalence collapse; any reward densification that leaves the free-text interface unchanged will leave a residual gap.
  • Finite action selection over graph-structured menus restores effective exploration: rollout groups preserve multiple retrieval-distinct outcomes, which gives group-relative optimizers real contrast even with outcome-only rewards.
  • The action menu makes same-state alternatives explicitly comparable, enabling credit assignment (SNC) that would be ill-posed over an open string space.
  • The gains are largest at smaller model scale (10.74 vs 3.98 points), suggesting that structured interfaces are especially important when policy capacity is limited.
  • The design is training-compatible with multiple RL algorithms (the paper reports stability under four standard algorithms) and transfers across backbones, so it's not tied to one model family.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own failure cases show trajectories that stop because the gold entity is not yet among the menu targets; this suggests a testable extension: measure coverage error of the graph and initial retrieval, and consider adaptive expansion when the gold evidence is not reachable.
  • The menu design separates 'information target' from 'retrieval mechanism'; this could generalize to other tool-using agents (e.g., code execution, database queries) where the same aliasing problem arises for free-form tool arguments.
  • Retrieval-equivalence collapse is a new diagnostic; it could be applied to other RL settings beyond QA, such as web navigation or scientific tool use, to detect when action diversity is illusory.
  • The frontier-relative credit idea might transfer to any environment with enumerable actions and deterministic transitions, not just retrieval; the precondition is previewability, which could be engineered in other domains.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper argues that the standard interface of RL search agents—generating free-form natural-language queries—is a structural bottleneck. It introduces Harness-G, which reformulates retrieval as finite action selection over a programmatically built paragraph–sentence–entity graph: the policy chooses Select/Lookup/Answer actions from a bounded, deduplicated, previewable menu, while the environment constructs queries and updates state. The paper also proposes Structured Non-myopic Credit (SNC), a two-term step credit that compares the selected action with same-state alternatives and propagates downstream gains along provenance edges. On six QA benchmarks with Qwen2.5-1.5B/3B, Harness-G is reported to outperform the strongest baseline Graph-R1 by 10.74 F1 at 1.5B and 3.98 at 3B. Controlled ablations support the menu and SNC contributions under a matched transition protocol.

Significance. The central claim is significant: if finite action selection over an evidence graph is genuinely more learnable than free-form query generation, then action-space design is a first-order lever for RL search agents, independent of reward-credit engineering. The paper’s matched menu-vs-free-query ablation (Table 2) is a well-designed isolation of the interface effect, and the SNC leave-one-out study (Table 3) gives credible evidence that both frontier-relative and enablement terms contribute. The paper also ships code, detailed hyperparameters, a full prompt stack, and a cost analysis that shows the graph is cheap to construct. However, the headline comparison and several auxiliary claims rest on evidence that is either cross-stack or lacks uncertainty quantification, and the menu-coverage limitation is acknowledged in failure cases but never measured. These gaps do not invalidate the core mechanism, but they do call for more careful empirical support before the strong claims in the abstract and conclusion can be accepted.

major comments (5)
  1. [Table 1 / Appendix C] The headline comparison against Graph-R1 uses published cross-stack numbers, not re-runs under the same protocol. Appendix C states this explicitly. The 10.74/3.98 F1 margins may therefore reflect differences in training recipe, graph construction, backbone, or evaluation rather than the interface alone. To support RQ1, the authors need matched Graph-R1 runs under the same datasets, backbone, budget, and evaluation, or at least confidence intervals confirming the margin is not within noise.
  2. [Appendix N.4 / Figures 19–20 / Appendix E] Menu coverage is never quantified. The authors’ own failure cases show gold entities (Obafemi Martins, Craig McCracken) absent from the typed targets, and the graph is built with SpaCy NER plus bge-large top-5 cosine≥0.80 neighbors under caps Ks=6, Ke=8. The matched ablation in Table 2 controls for coverage because both arms share the same target set, but the headline claim of a general interface advantage requires measuring how often gold evidence is reachable. Report coverage rates per dataset and F1 conditional on reachability; otherwise the reported margins are upper bounds on a coverage-favorable subset.
  3. [Appendix B / Table 1] Each held-out split contains only 128 questions, and no error bars, confidence intervals, or significance tests are reported. Given small per-dataset differences—e.g., Harness-G loses to Graph-R1 by 1.84 F1 on NQ at 3B—the aggregate 3.98 average could be within sampling noise. The authors should report bootstrap CIs, per-seed variance, or a paired significance test over the 128 questions, and should be cautious about per-dataset claims (e.g., the NQ comparison).
  4. [Appendix J / Table 10] The final action menu (removal of Rewrite_Query and Open_Context, merging Expand/Bridge into Lookup, adding Answer_With, internalized lookup) was selected using 2Wiki MultiHopQA results. This makes the 2Wiki result optimistically biased and weakens the cross-dataset generalization claim. The authors should either fix the menu a priori, validate the design on a held-out development set, or clearly present the screening as an exploratory analysis and avoid treating the 2Wiki gain as an unbiased estimate.
  5. [Appendix G / Eq. (13)] The retrieval-equivalence collapse phenomenon—central to the paper’s motivation—is defined by 'evidence-overlap clustering', but the clustering algorithm and threshold are not specified. Dquery, Dret, and Neq in Eqs. (14)–(16) rely on this undefined κ(E(τ)). Without the clustering procedure, Figure 1 cannot be reproduced, and the claimed collapse rate ('from ~86% to <10% by step 30') is not verifiable. The authors should specify the clustering method, similarity threshold, and the exact computation of κ.
minor comments (5)
  1. [Table 1, Table 8, Table 9] Several numeric entries are visually merged or broken, e.g., '36.8660.79' and '44.7537.50'. Please fix the table formatting so each metric is legible.
  2. [Eq. (3) / Algorithm 2] The frozen answerer g is described as using the gold answer y*; the paper calls this 'no external reward model', but the term 'frozen answerer' may confuse. Clarify in the main text that g is the initial policy checkpoint and that gold answers are used only for training, not at inference.
  3. [Appendix N.4] Three failure cases are shown without any failure-rate denominator. Reporting the total number of failed trajectories and the fraction with unreachable gold evidence would make the qualitative analysis more informative.
  4. [Appendix J] The screening sequence L0→L⋆ is useful, but F1 and EM values in Table 10 are from a single run. A note on variance (even a min/max over seeds) would help distinguish design choices from training noise.
  5. [Appendix G] The matched free-query arm uses a frozen resolver that maps generated strings to targets in the same latent set A_t. This is a reasonable isolation device, but it should be stated more prominently that this is not the standard free-query setting over an open corpus; the comparison isolates the interface, not the retriever.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; Harness-G's central comparisons are external and its credit signal is not a fitted prediction.

full rationale

The paper's load-bearing claims are empirical comparisons against external baselines (Table 1: Search-R1, IGPO, Graph-R1, etc.), not reductions of a prediction to a fitted parameter. The retrieval-equivalence-collapse diagnostic is defined by evidence-overlap clustering (Appendix G, Eqs. 12-15) and measured during Search-R1 training; the menu advantage is tested under a matched transition protocol (Appendix G) that holds feasible actions and transitions fixed. SNC is a training-time credit signal that uses a frozen answerer to score gold-answer probability (Eq. 3) and propagates gains along provenance edges (Eq. 6); it is not used to produce the evaluated F1 predictions. No equation in the paper equates a claimed outcome with its own input by construction. Self-citations (Hou et al. 2026a,b; Hou et al. 2025) appear only in related-work and motivation and do not carry the central claim. The unquantified menu-coverage limitation shown in Appendix N.4 (Figures 19-20, gold entity 'not yet among typed targets') is a genuine correctness risk, not circularity: the paper does not define the interface advantage in terms of coverage, and the limitation is disclosed as such.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the graph coverage assumption, the frozen-answerer proxy, and several hand-set SNC/menu hyperparameters. No new physical entities are postulated. Most free parameters are standard RL hyperparameters; the SNC-specific ones are listed.

free parameters (7)
  • SNC weight lambda = 0.2
    Balances outcome advantage and SNC step advantage in Eq. 7; chosen by hand (Table 7).
  • Propagation discount gamma = 1.0
    Default discount in Eq. 6; no tuning reported.
  • Frontier size Kf = 4
    Caps number of type-stratified alternatives previewed per step (Table 7).
  • Information-gain dead-zone = 1e-4
    Suppresses small p_t values judged to be scoring noise (Appendix E).
  • Visible-sentence cap Ks / Lookup cap Ke = 6 / 8
    Menu size caps that bound M_t; chosen by hand (Table 7).
  • Entity-linking similarity threshold = 0.80, top-5 neighbors
    Controls which entities are canonicalized/linked into the graph; chosen by hand (Appendix E).
  • SNC clamp c / scale floor s_min = 5.0 / 5e-4
    Numerical safeguards for credit scaling (Table 7).
assumptions (5)
  • standard math GRPO group-relative advantage normalization (Shao et al. 2024) is a valid training objective for this policy.
    Used as the optimization backbone; not derived in this paper.
  • domain assumption The frozen answerer's length-normalized teacher-forced gold-answer probability g(O) monotonically tracks evidence utility.
    Eq. 3 and Appendix I; if this proxy is miscalibrated, SNC step credit misallocates.
  • domain assumption The programmatic graph provides sufficient entity/sentence coverage for the six benchmarks.
    Appendix E graph construction; Figures 19-20 show failure when gold is not on menu.
  • domain assumption Final answers condition only on committed evidence; uncommitted visible text does not enter the generator.
    The Menu Environment: 'the final answer conditions only on committed evidence.'
  • ad hoc to paper Latest-producer provenance with equal split among producers is a sufficient model of enablement.
    Eq. 6 reverse topological propagation; no learned value model or counterfactual rollouts; equal split is a modeling choice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Harness-G: A Graph-Structured Harness for Search Agents." pith.science (2026). https://pith.science/paper/574VEQUU

@misc{pith2026260727652,
  author       = {Pith},
  title        = {Pith review of: Harness-G: A Graph-Structured Harness for Search Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/574VEQUU}},
  note         = {Machine review of arXiv:2607.27652}
}
read the original abstract

Reinforcement learning (RL) search agents commonly model retrieval as free-form natural-language query generation and optimize multi-turn interactions using final-answer rewards. Current studies mainly improve training with denser or more structured credit signals, but rarely examine whether retrieval is properly formulated at the policy-environment interface. We observe pronounced retrieval aliasing during Search-R1 training: rollouts for the same question continue to generate distinct query strings, yet their accumulated evidence sets increasingly overlap. We call this phenomenon retrieval-equivalence collapse; in this regime, trajectories approach utility equivalence with respect to retrieval decisions, leaving within-group returns with little effective retrieval contrast. To address this problem, we propose Harness-G, a graph-structured retrieval framework that redesigns this interface. It reformulates free-form query generation as finite action selection: the policy selects an evidence sentence or entity, or chooses to answer, while the environment constructs the menu, tracks retrieval state, and validates and executes each choice. This interface reduces linguistic aliasing and makes same-state alternatives directly comparable. Building on this interface, we introduce Structured Non-myopic Credit (SNC), which uses a frozen answer scorer to compare the selected action with its alternatives and assigns downstream gains to the earlier actions that enabled them. Across six QA benchmarks, Harness-G achieves the highest average F1 at both evaluated model scales, outperforming the strongest baseline, Graph-R1, by 10.74 points at 1.5B and 3.98 points at 3B.

Figures

Figures reproduced from arXiv: 2607.27652 by the authors.

Figure 1
Figure 1. Retrieval-equivalence collapse. (a) Query-form diversity remains high as retrieval-outcome diversity col￾lapses, producing illusory exploration. (b) Under matched transitions, the action menu preserves more retrieval-distinct outcomes per rollout group than free querying. vanishing within-group advantages (Wang et al. 2025a,b; Hou et al. 2026a). Existing methods primarily address these problems through process rewar… view at source ↗
Figure 2
Figure 2. Retrieval interface redesign. Left: free-query interface, where the policy generates a query string and receives searched documents. Right: Harness-G interface, where the policy selects a finite action ID from the menu (Select/Lookup/Answer) over a tripartite graph, and the environment constructs queries and updates retrieval state. actions are preferable from return differences among tra￾jectories sampled for the s… view at source ↗
Figure 3
Figure 3. Harness-G from graph construction to policy optimization. A paragraph–sentence–entity graph supports the online [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (18 more)
Figure 4
Figure 4. Figure 4: Training dynamics of Harness-G (Qwen2.5-3B): [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 7
Figure 7. Figure 7: Zero-advantage groups on 2Wiki (Qwen2.5-3B): [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: Interaction turns (a) and response length (b) during [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: F1 comparison across six datasets under O.O.D. [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 10
Figure 10. Figure 10: 2Wiki bridge retrieval: menu shows chosen vs. rejected targets at each hop. [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: 2Wiki comparison: both operands committed without [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]
Figure 1
Figure 1. Figure 1: free-query groups lose retrieval-distinct diversity [PITH_FULL_IMAGE:figures/full_fig_p020_1.png]
Figure 12
Figure 12. Figure 12: MuSiQue composition: two committed premises, no extra search. [PITH_FULL_IMAGE:figures/full_fig_p021_12.png]
Figure 13
Figure 13. Figure 13: 2Wiki birth-date comparison: both DOBs exposed as typed menu targets. [PITH_FULL_IMAGE:figures/full_fig_p022_13.png]
Figure 14
Figure 14. Figure 14: 2Wiki country comparison: two nationality commits, zero lookups. [PITH_FULL_IMAGE:figures/full_fig_p023_14.png]
Figure 15
Figure 15. Figure 15: Same 2Wiki bridge question: free-query failure vs. menu success. [PITH_FULL_IMAGE:figures/full_fig_p024_15.png]
Figure 16
Figure 16. Figure 16: SNC frontier term on the bridge Lookup: gain bars vs. same-state alternatives [PITH_FULL_IMAGE:figures/full_fig_p024_16.png]
Figure 17
Figure 17. Figure 17: SNC enablement term: downstream DOB gain flows back to the bridge [PITH_FULL_IMAGE:figures/full_fig_p025_17.png]
Figure 18
Figure 18. Figure 18: Same bridge trajectory under outcome-only, myopic, and SNC credit. [PITH_FULL_IMAGE:figures/full_fig_p025_18.png]
Figure 19
Figure 19. Figure 19: HotpotQA failure case: Answer_With on a birth-year-matched candidate; gold scorer is not yet among menu targets. Failure: wrong bridge hop HotpotQA · held-out · Menu+SNC, 3B · EM = 0 Question. Who was the animator behind the series that inspired Powerpuff Girls Z? (go…
Figure 20
Figure 20. Figure 20: HotpotQA failure case: anime-side bridge [PITH_FULL_IMAGE:figures/full_fig_p026_20.png]
Figure 21
Figure 21. Figure 21: HotpotQA failure case: gold name and Answer_With are available on the opening menu; the emitted answer string still mismatches CT [PITH_FULL_IMAGE:figures/full_fig_p027_21.png]
Figure 22
Figure 22. Figure 22: Prompt stack for the full Harness-G configuration. [PITH_FULL_IMAGE:figures/full_fig_p028_22.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 4 linked inside Pith

  1. [1]

    The Metamorphosis of Mr

    Opening menu (16 actions) Chosen: A1 Select S1 S1. The Metamorphosis of Mr. Samsa (1977) by Caroline Leaf. Exposes bridge entity. Not chosen (same state): S2–S5 unrelated directors (Kiarostami, Samanta, Ray, ...); Lookup only from distractor S0; premature Answer_With

    1977

  2. [2]

    InAdvancesinNeural Information Processing Systems

    HippoRAG: Neurobiologically Inspired Long-Term MemoryforLargeLanguageModels. InAdvancesinNeural Information Processing Systems. Gutiérrez, B. J.; Shu, Y.; Qi, W.; Zhou, S.; and Su, Y

  3. [3]

    Caroline Leaf bornAugust 12, 1946 in Seattle, Washington

    Answer evidence Returned by Lookup S6. Caroline Leaf bornAugust 12, 1946 in Seattle, Washington. Chosen: A4 Answer_With S6 Also visible: music / Cannes credits — left uncommitted

    1946

  4. [4]

    fetch something

    Committed set𝐶𝑇 Committed evidence ✓ S1: film→ director ✓ S6: director→ birth date Not in 𝐶𝑇 : distractor directors; Kafka; city/state Answer: August 12, 1946 EM = 1, F1 = 1.0 Menu properties. Finite typed actions; visited targets drop out; queries are built by the environment. Path: commit the director sentence, Lookup that entity, terminate on the DOB s...

    1946

  5. [5]

    Mallen, A.; Asai, A.; Zhong, V.; Das, R.; Khashabi, D.; and Hajishirzi,H.2023.WhenNottoTrustLanguageModels:In- vestigating Effectiveness of Parametric and Non-Parametric Memories

    Think-on-Graph 2.0: Deep and Faithful Large Lan- guage Model Reasoning with Knowledge-guided Retrieval Augmented Generation.arXiv preprint arXiv:2407.10805. Mallen, A.; Asai, A.; Zhong, V.; Das, R.; Khashabi, D.; and Hajishirzi,H.2023.WhenNottoTrustLanguageModels:In- vestigating Effectiveness of Parametric and Non-Parametric Memories. InProceedings of ACL...

    arXiv 2023

  6. [6]

    filter” or “dedup

    DeepSeekMath: Pushing the Limits of Mathemati- cal Reasoning in Open Language Models.arXiv preprint arXiv:2402.03300. Sheng, G.; Zhang, C.; Ye, Z.; Wu, X.; Zhang, W.; Zhang, R.; Peng, Y.; Lin, H.; and Wu, C. 2024. HybridFlow: A Flexible and Efficient RLHF Framework.arXiv preprint arXiv:2409.19256. Song,H.;Jiang,J.;Min,Y.;Chen,J.;Chen,Z.;Zhao,W.X.; Fang, L...

    arXiv 2024

  7. [7]

    Search- R1 (Jin et al

    and related adaptive RAG controllers further learn whenandwhatto retrieve under reflection or complexity- aware policies, but typically use prompting or supervised objectives rather than multi-turn group-relative RL. Search- R1 (Jin et al. 2025) made outcome-reward RL the default for multi-turn search: the policy emits free-form queries, a black-box retri...

    2025

  8. [8]

    retrievalasinteractivedecisionprocess

    studies process versus outcome rewards for agentic RAG;CriticSearch(Zhangetal.2026)retrospectivelyscores turns with a frozen critic; Tree-GRPO (Ji et al. 2025) cre- ates step-level contrast through tree expansion with addi- tional rollouts, whereas GiGPO (Feng et al. 2025) retroac- tively groups actions from repeated anchor states in al- ready collected t...

    2026

Show all 50 references
  1. [9]

    Only choose from the currentavailable_actions

    Do not invent action ids. Only choose from the currentavailable_actions. 10.Action ids can change at every step. Read the currentavailable_actions before choosing. 11.Do not write a free-form search query by itself. Always start with an avail- able action id. 12.After choosing...

  2. [10]

    Menu after committing S1 Chosen: A10 Lookup Caroline Leaf provenance: from S1 Rejected alternatives: A11 Lookup Franz Kafka’s A12 Lookup The Metamorphosis residual director Selects A13 Answer (premature) Target is an entity id, not a free string

  3. [13]

    The Frozen Child is a1921 Hungarian silent film

    Both years visible S1. The Frozen Child is a1921 Hungarian silent film. A1 Select S1 S0. Naadody is a1992 Indian Malayalam-language film. later via Answer_With Near-name distractor S2 Frozen (2010) — never selected. Other titles lack the required years

    2010

  4. [14]

    Select residual titles Answer without second year No Lookup issued Both operands already in view

    After committing S1 Menu still offers A5 Answer_With S0 ← chosen (second year) Lookup Béla Balogh, cast, ... Select residual titles Answer without second year No Lookup issued Both operands already in view

  5. [15]

    Committed operands Committed evidence ✓ S1: Frozen Child→ 1921 ✓ S0: Naadody→ 1992 1921 < 1992 Not committed S2 Frozen (2010); director / cast lookups

    1921

  6. [16]

    director of

    Terminate Comparison resolved from two typed commits. Environment never forces a free- form year query. Answer: The Frozen Child EM = 1, F1 = 1.0 Trace. Menu exposes both year-bearing sentences at step 1; the policy commits the two operands and stops. Near-name distractors rem...

  7. [17]

    A3 Select S3 first Bridge premise S0.For centuries, peoples in modern-day Nigeria traded overland with traders from North Africa

    Opening menu — two premises visible Answer premise S3.Muslim conquest ofNorth Africa began in themid-7th to early 8th cen- turies. A3 Select S3 first Bridge premise S0.For centuries, peoples in modern-day Nigeria traded overland with traders from North Africa. later: A5 Answer...

  8. [18]

    AfterSelectS3 Menu (subset) A5 Answer_With S0← chosen Lookup Arabia / North Africa / Sahara low-priority bridge lookups Select residual distractors Answer without bridge sentence No extra search step Join entityNorth Africaalready links S0 and S3

  9. [19]

    bridge, then attribute

    Join and commit Composition path Nigeria’s trade region ↓ North Africa ↓ Muslim conquest period Committed𝐶𝑇 ✓ S3: conquest period ✓ S0: Nigeria↔ North Africa Answer: mid-7th to early 8th centuries EM = 1, F1 = 1.0 Trace. Both premises are menu-visible; the policy commits the c...

    2010

  10. [20]

    Hans Wilhelm Gustav Winderstein (29 Oc- tober 1856– 23 June 1925), German conductor and composer

    Opening menu: both DOBs visible S0. Hans Wilhelm Gustav Winderstein (29 Oc- tober 1856– 23 June 1925), German conductor and composer. A0 Select S0 / also A6Answer_With S0 S1. Mladen Ličina (born 14 March 1991), Ser- bian football midfielder. A1 Select S1 / A7Answer_With S1 Nam...

    1925

  11. [21]

    No open-ended birth-date query

    Typed choices (no free string) Feasible path under the menu Commit / answer-with S0 (1856) Commit / answer-with S1 (1991) Optional: Lookup Winderstein / Ličina aliases (menu also lists Lüneburg, Hanover, ...) Comparison is 1856 < 1991 once both year- bearing sentences are type...

    1991

  12. [22]

    Toei Animation

    Committed set and answer Committed evidence ✓ S0: Winderstein→ 1856 ✓ S1: Ličina→ 1991 1856 < 1991 Answer: Mladen Ličina EM = 1, F1 = 1.0 Held-out rollout (index 14) recovers both DOB sen- tences from the typed menu and terminates on the later birth. Menu effect. Both birth da...

    1991

  13. [23]

    Early Opera Company is aBritish ensemble dedicated to baroque operas on period instru- ments

    Opening menu exposes both countries S2. Early Opera Company is aBritish ensemble dedicated to baroque operas on period instru- ments. A2 Select S2 S1. The Beggars’ Guild is anAmerican rock band from Georgia. later: Answer_With S1 Unrelated visible text S0 concerns The Beggar’s...

  14. [24]

    Policy issues no Lookup

    T wo commits, zero lookups Executed actions A2 Select S2 (British) A6 Answer_With S1 (American) Menu also offersLookup on Early Opera Company aliases, Dublin, ... Policy issues no Lookup. Both nationality predicates are already typed sentence targets

  15. [25]

    director of The Metamorphosis of Mr. Samsa

    Boolean close Committed𝐶𝑇 ✓ S2: Early Opera Co.→ British ✓ S1: Beggars’ Guild→ American British ≠ American Answer: No EM = 1, F1 = 1.0 Early termination: evidence is sufficient after two typed commits. Trace. The menu makes both country predicates selectable without rewriting ...

    1946

  16. [26]

    𝑔(𝑂𝑡) low on 𝑦∗

    Decision state 𝑧𝑡 Committed / observed S1: film→ Caroline Leaf DOB not yet in𝑂𝑡. 𝑔(𝑂𝑡) low on 𝑦∗

  17. [27]

    Read-only previews 𝑝𝑡(𝑎) = 𝑔(𝑂𝑡∪ ˜𝑈(𝑎))− 𝑔(𝑂𝑡) Adopted: Lookup Caroline Leaf, 𝑝𝑡 =+0.61 FrontierF𝑡 : Kafka+0.02; film title+0.03; other lookups≈ 0 ¯𝑝𝑡 = 0.02

  18. [28]

    Credit Frontier-relative 𝑟 fr 𝑡 = 𝑝𝑡(𝑎𝑡)− ¯𝑝𝑡 = 0.61− 0.02 =+0.59 Credits only actions that outgain same- state alternatives

  19. [29]

    Open string spaces cannot enumerate this baseline

    Preview gains 𝑝𝑡(𝑎) Lookup Caroline Leaf +0.61 LookupThe Metamorphosis +0.03 LookupFranz Kafka’s +0.02 Other Lookuptargets ≈ 0 ¯𝑝𝑡 averages non-adopted frontier ac- tions. Open string spaces cannot enumerate this baseline. SNC frontier term. Credit is relative to the same-stat...

    2010

  20. [30]

    Steven Davis, born1985 S1

    Opening menu (16 actions) Visible bios (birth-year cue): S0. Steven Davis, born1985 S1. Arjen Robben, born1984 S2. Vincent Kompany, born1986 Also: Kilbane, Hazard awards; no League Cup final scorer sentence. Gold on menu? No

  21. [31]

    scored in the 2010–11 final

    Policy choice Chosen: A7 Answer_With S1 Matches the born in 1984 attribute only. Not used (same state): A12–A15 Lookup Davis / Northern Irish / Premier League / Southampton — none is Martins. Zero bridge verification of “scored in the 2010–11 final.”

    1984

  22. [32]

    Prediction: Arjen Robben Gold: Obafemi Martins EM = 0, F1 = 0, turns = 2 Gold scorer not yet among typed targets on this item

    Outcome Committed𝐶𝑇 ={𝑆1} S1 entails DOB,not final scorer. Prediction: Arjen Robben Gold: Obafemi Martins EM = 0, F1 = 0, turns = 2 Gold scorer not yet among typed targets on this item. Reading. Actions stay finite and typed; the trajectory commits a birth-year-matched bio bef...

  23. [33]

    Powerpuff Girls Zis a Japanese anime directed by Megumu Ishiguro, based onThe Powerpuff Girls

    Opening menu S1. Powerpuff Girls Zis a Japanese anime directed by Megumu Ishiguro, based onThe Powerpuff Girls. Typed lookups: A13 Lookup Megumu Ishiguro A14 Lookup The Powerpuff Girls A12 Lookup Powerpuff Girls Z Better hop A14 is available

  24. [34]

    Anime-side bridge Chosen: A13 Lookup Megumu Ishiguro anime director, not American creator Rejected alternative: A14 Lookup The Powerpuff Girls (the inspired-by source series) Bridge targets the anime side

  25. [35]

    anime co-produced by Cartoon Network Japan / Aniplex; animated by Toei Animation

    Anime studio evidence Returned after hop S6. anime co-produced by Cartoon Network Japan / Aniplex; animated by Toei Animation. Chosen: Select S6 then Answer_With S1+S6 Studio of the adaptation, not animator of the source

  26. [36]

    Prediction: Toei Animation Gold: Craig McCracken EM = 0, F1 = 0, turns = 4 Gold never enters menu

    Outcome Committed 𝐶𝑇 ={𝑆1,𝑆 6} Entails anime studio / director, not Craig McCracken. Prediction: Toei Animation Gold: Craig McCracken EM = 0, F1 = 0, turns = 4 Gold never enters menu. Reading. Both hops are typed same-state alternatives (anime director vs. source series)—the s...

  27. [37]

    Back Roads (1981) stars Sally Field and Tommy Lee Jones

    Gold already typed S0. Back Roads (1981) stars Sally Field and Tommy Lee Jones. Same-state options: A6 Answer_With S0 A13 Lookup Tommy Lee Jones A12 Lookup Sally Field Gold on menu? Yes

    1981

  28. [38]

    He received four nominations

    Detour hops Step 1: A12 Lookup Sally Field co-star, not answer entity Step 2: A12 Lookup Tommy Lee Jones gold entity finally visited Also visible: pronoun-only Oscar sentences (“He received four nominations...”)

  29. [39]

    generate Chosen: A6 Answer_With S0 𝐶𝑇 contains the correct cast sen- tence naming Tommy Lee Jones

    Commit vs. generate Chosen: A6 Answer_With S0 𝐶𝑇 contains the correct cast sen- tence naming Tommy Lee Jones. Emitted answer string Tom Conti Differs from names in𝐶𝑇. Multi-constraint join (Oscar + co-star) is still hard at answer time

  30. [40]

    You are Qwen, created by Alibaba Cloud. You are a helpful assistant

    Outcome Gold was selectable on the opening menu. Multi-constraint join at answer time still hard for the 3B policy. EM = 0, F1 = 0, turns = 4 Contrast with F1/F2: gold string was explicit. Reading. The menu already exposes the gold as both an evidence sentence and aLookup targ...

  31. [41]

    query":

    First call <query>{"query": "INIT"}</query> to get the initial graph ob- servation

  32. [42]

    query":

    After each [HARNESS_G_OBS], choose exactly one available action id, for ex- ample <query>{"query": "A0"}</query>

  33. [43]

    Available actions areSELECT, LOOKUP, ANSWER_WITH, and ANSWER

  34. [44]

    SELECT means selecting a useful visible sentence as evidence

  35. [45]

    Just choose theLOOKUP action id; the retrieval query is built for you from the question and the evidence you have already selected

    LOOKUP means looking up an entity from the current observation to find missing information. Just choose theLOOKUP action id; the retrieval query is built for you from the question and the evidence you have already selected

  36. [46]

    ANSWER_WITH rule

    ANSWER means stop searching and provide the final answer. ANSWER_WITH rule. ANSWER_WITH means selecting a visible sentence as final evidence and stopping immediately. Use it only when that sentence alone is suf- ficient to answer the question

  37. [47]

    Multi-hop questions usually requireSELECT evidence, then LOOKUP the miss- ing entity, thenSELECT supporting evidence about that entity, thenANSWER

  38. [48]

    Do not answer before the selected evidence covers every hop required by the question

  39. [50]

    [n] {evidence sentence n} Provide the final answer in <answer>...</answer> using only the observed evidence

    {evidence sentence 1} ... [n] {evidence sentence n} Provide the final answer in <answer>...</answer> using only the observed evidence. Dense-retriever query prefix: Represent this sentence for searching relevant passages: Figure 22: Prompt stack for the full Harness-G configuration

  40. [2020]

    InProceedings of COLING

    Constructing a Multi-hop QA Dataset for Compre- hensive Evaluation of Reasoning Steps. InProceedings of COLING. Hou, Y.; Li, P.; Liu, Z.; Wang, Y.; Ruan, Y.; Qiu, J.; and Xu, K. 2026a. VisualAD: Language-Free Zero-Shot Anomaly Detection via Vision Transformer. InProceedings of...

    2025 arXiv

  41. [2024]

    InInternational Conference on Learning Representations

    Self-RAG: Learning to Retrieve, Generate, and Cri- tique through Self-Reflection. InInternational Conference on Learning Representations. Buck, C.; Bulian, J.; Ciaramita, M.; Gajewski, W.; Ges- mundo, A.; Houlsby, N.; andWang, W. 2018. Ask theRight Questions: Active Question R...

    2018

  42. [2025]

    Harutyunyan, A.; Dabney, W.; Mesnard, T.; Azar, M

    From RAG to Memory: Non-Parametric Contin- ual Learning for Large Language Models.arXiv preprint arXiv:2502.14802. Harutyunyan, A.; Dabney, W.; Mesnard, T.; Azar, M. G.; Piot, B.; Heess, N.; van Hasselt, H. P.; Wayne, G.; Singh, S.; Precup, D.; and Munos, R. 2019. Hindsight Cr...

    2019 arXiv

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.