HierSearch: A Hierarchical Enterprise Deep Search Framework Integrating Local and Web Searches

Jian Xie; Jiehan Cheng; Jiejun Tan; Ji-Rong Wen; Qiang Ju; Yan Yu; Zhicheng Dou

REVIEW 3 major objections 6 minor 1 cited by

HierSearch: A Hierarchical Enterprise Deep Search Framework Integrating Local and Web Searches

T0 review · 3 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read HierSearch claims that splitting deep search into local and web specialist agents trained with hierarchical reinforcement learning outperforms flat reinforcement learning and state-of-the-art deep search and multi-source RAG baselines acros

desk verdict The hierarchical RL architecture is a sensible contribution, but query-aware local corpora for most benchmarks undercut the headline results, so treat the numbers as provisional until rerun on independently built indices. read the letter →

arxiv 2508.08088 v1 pith:6OBCIYFC submitted 2025-08-11 cs.IR cs.AIcs.CL

Jiejun Tan , Zhicheng Dou , Yan Yu , Jiehan Cheng , Qiang Ju , Jian Xie , Ji-Rong Wen This is my paper

classification cs.IRcs.AIcs.CL

keywords deepsearchhierarchicalreinforcementlearningretrieval-augmentedgenerationmulti-knowledge-sourceenterpriseknowledgegraphGRPO

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes HierSearch, a framework for enterprise "deep search" where a reasoning model answers questions by searching both a private local corpus (text chunks plus a knowledge graph) and the live web. It claims that splitting the task into two specialist search agents — one local, one web — trained first by reinforcement learning, and then training a planner agent to coordinate them, learns more efficiently and answers more accurately than a single agent trained to operate all search tools at once. The paper reports that HierSearch outperforms flat-RL training and a range of deep search and multi-source RAG baselines across six QA benchmarks in general, finance, and medical domains, while keeping search costs comparable. A reasoning-aware knowledge refiner filters the evidence passed upward, preventing the planner from copying hallucinated reasoning or being distracted by irrelevant search results.

What carries the argument

Hierarchical reinforcement learning with GRPO: low-level local and web deep search agents are trained separately on a small set of tools (chunk search, graph search, adjacent passages; web search, browse URL), then a high-level planner agent is trained to call them as tools. The knowledge refiner filters trajectory evidence in two steps: first it keeps the top α% of evidence by embedding similarity to the agent's next thinking step, then it adds the top β% of remaining evidence by similarity to the agent's conclusion (concatenated with the other agent's conclusion when both were called). This prevents the planner from copying short-sighted or hallucinated reasoning and keeps the context conc

What would settle it

Re-evaluate HierSearch on general and medical benchmarks after building the local text-chunk corpus independently of the test questions (e.g., by randomly sampling Wikipedia and PubMed passages and removing any that contain known gold answers), then compare local search success rate and EM/F1; if the gap over flat RL and baseline methods narrows or vanishes, the reported advantage depends on the answer-aware corpus rather than on the hierarchy.

Watch

Extended reading notes

Core claim

HierSearch's central claim is that multi-knowledge-source deep search should be trained hierarchically: a local deep search agent and a web deep search agent each master the tools of one knowledge source, then a planner agent learns to call them selectively, combine their evidence, and produce a final answer. The framework is trained with GRPO (Group Relative Policy Optimization) and rule-based rewards, first on the low-level agents and then on the planner, and a knowledge refiner selects evidence from low-level trajectories by scoring each returned item's similarity to the agent's next reasoning step and to its final conclusion. In experiments on MuSiQue, OmniEval, BioASQ, NQ, HotpotQA, and

Load-bearing premise

The local corpus for the general and medical benchmarks was built by sampling passages that the authors knew were directly related to the test questions (plus hard negatives), so the local search index is answer-aware; if the corpus were built independently of those questions, local search success and the reported advantage could shrink.

Editorial extensions

If this is right

Enterprise private search systems can combine a secure local knowledge base with live web search without training a single monolithic agent, reducing the exploration burden on expensive web tools.
Separating agents by knowledge source makes RL training more sample-efficient: the training curves on MuSiQue and OmniEval show HierSearch learning faster and staying above flat RL through 300 steps.
The refiner's evidence selection implies that only the evidence that actually influenced a search agent's reasoning needs to be propagated, which shortens planner context and reduces error propagation.
Out-of-distribution evaluations on NQ, HotpotQA, and PubMedQA — datasets not in training — suggest the hierarchy generalizes to new domains and question types without retraining.
Because web search calls are the slow and expensive part, a hierarchy that decides when local evidence suffices can cut cost: HierSearch uses about 1.06 web searches per query on MuSiQue, fewer than several parallel-search baselines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The local corpus for the general and medical benchmarks is built by sampling passages that the authors knew were directly related to the test questions (plus hard negatives), so the local search index is answer-aware; if the corpus were built independently of the questions, local search success and the measured advantage could shrink.
The hierarchical decomposition and refiner could transfer beyond search: any long-horizon tool-use task where tools cluster into domains with different costs (e.g., database queries vs. web API calls) could use the same two-stage training to avoid sparse exploration in expensive tool regions.
The knowledge refiner's two-step selection is a form of trace summarization; it could be applied to interpretable agent logging, or combined with learned relevance models instead of embedding similarity, which might improve selection when reasoning text is not lexically similar to the evidence.
A testable extension would be to vary the α and β thresholds per domain to see if optimal refinement depends on corpus noise, since medical and financial corpora differ in how much irrelevant or contradictory evidence is retrieved.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's the short version: HierSearch is a plausible architecture—two low-level deep-search agents (local, web) plus a planner, trained hierarchically with GRPO, and a knowledge refiner that filters evidence—and the HRL-vs-flat-RL comparison is the right way to test the hierarchy claim. But the evaluation is undercut by a self-inflicted setup issue, so I wouldn't trust the absolute numbers yet.

What's actually new: the paper is the first, as far as the cited literature goes, to apply hierarchical RL to deep search. The decomposition is sensible: train the source-specific agents first, freeze them, then train the planner. The refiner addresses a real failure mode—planners copying low-level hallucinated conclusions—and the two-step embedding-similarity filter is a reasonable heuristic. The paper also ships code and data, which is more than most, and the efficiency analysis is useful.

The soft spot is in the Experiments, Implementation Details: for general and medical domains, the local corpus is built by sampling 'directly related passages for questions' plus BM25 hard negatives. That means for MuSiQue, NQ, HotpotQA, BioASQ, and PubMedQA, the index contains answer-bearing passages for every test question. Local search success of 94.25% on NQ (Table 3) is then essentially guaranteed, and the absolute scores, the ablation gains, and the generalization claims for NQ/HotpotQA/PubMedQA all ride on that. It also weakens the comparison with baselines—HierSearch's trained local agent could be leveraging the easy index more effectively than prompted baselines. The flat-RL comparison is still controlled (both use the same corpus), so the hierarchy claim survives in principle, but it only shows the hierarchy helps when local retrieval is artificially easy.

Second, there are no error bars or significance tests anywhere. The OmniEval gap of 1.07 EM is probably noise. And the refiner's alpha/beta hyperparameters are never reported, which is a reproducibility gap. These aren't fatal to the core idea, but they reset the confidence level.

Who's this for? Practitioners building agentic enterprise search. The paper deserves a serious referee and a major revision: rerun on an independently sampled corpus or at least justify why the query-aware sampling is fine, add variance, and report alpha/beta. If that holds, the hierarchy result would be worth publishing.

Referee Report

3 major / 6 minor

Summary. The paper proposes HierSearch, a hierarchical agentic deep search framework for enterprise scenarios that combine local and Web knowledge sources. It trains two low-level agents (local deep search and Web deep search) separately via GRPO, then a high-level planner agent that calls these agents as tools, and a reasoning-aware knowledge refiner that filters evidence before passing it to the planner. Experiments on six benchmarks (MuSiQue, OmniEval, BioASQ, NQ, HotpotQA, PubMedQA) report improvements over flat RL and several deep search / multi-source RAG baselines in EM, F1, and search/reasoning success rates. Code and datasets are released.

Significance. The hierarchical decomposition of search tools is a plausible and potentially useful answer to the large action-space and credit-assignment problems that arise in multi-source deep search. The controlled comparison between HRL and flat RL (Figure 3, Table 1) is a strength, as are the rule-based reward design and the public release of code. The knowledge refiner is a simple but sensible way to reduce irrelevant evidence and hallucination propagation. If the empirical results survive an independent-corpus evaluation, the work would be a solid contribution to agentic RAG and enterprise search. However, the current evidence is weakened by a query-aware local corpus construction and the absence of variance/statistical testing, so the empirical claims should be treated as provisional.

major comments (3)

[Experiments, Implementation Details] The local corpus for MuSiQue, NQ, HotpotQA, BioASQ, and PubMedQA is built from 'directly related passages for questions and hard negatives retrieved by BM25'. This means the test questions are used to select the corpus and knowledge graph, so every query is guaranteed to have a relevant passage in the index. This inflates absolute scores and the local-agent utility (e.g., Table 3 reports 94.25% local search success for HierSearch on NQ) and directly weakens the out-of-distribution generalization claim for NQ/HotpotQA/PubMedQA in Table 1. The flat-RL comparison also uses this favorable setup. Please re-run the evaluation with a corpus sampled independently of the test questions (e.g., a random sample of the full Wikipedia/PubMed dump or a time-based split) and report how the absolute and relative results change.
[Experiments, Main Results (Tables 1–3)] All reported EM/F1 values are single point estimates with no error bars, confidence intervals, or significance tests. Some differences are small (e.g., OmniEval EM: HierSearch 10.67 vs. HierSearch w/o HRL 7.73 and PrefRAG 9.60; MuSiQue F1: 62.83 vs. 57.19). Without variance estimates it is impossible to tell whether these gaps are meaningful, especially given the 400-sample test sets. Please run at least three seeds or use bootstrapping, and report means ± standard deviations together with paired significance tests (e.g., bootstrap or approximate randomization) for EM and F1.
[Methodology, Eq. (1) and Figure 3] The low-level agents are trained with the same F1-based reward computed against the golden answer, which couples their search behavior to the final answer format. The high local search success of HierSearch in Table 3 may therefore partly reflect the query-aware index rather than a genuinely learned retrieval policy. The paper should clarify whether the low-level agents receive any reward for retrieving evidence that contains the gold answer (e.g., a retrieval-aware reward) or at least report retrieval success during training on a corpus built without test-question passages. Without this, the contribution of HRL to the retrieval gap is difficult to isolate.

minor comments (6)

[Experiments] Main text says 373 samples are used for OmniEval, but Table 5 reports # Test Set = 375 for OmniEval. Please reconcile the numbers.
[Table 1 caption] The caption says 'The best and second best of each model are in bold and underlined', but the formatting is not visible in the text. Please clearly define which style denotes best and which denotes second best.
[Table 2] The percentage decreases (e.g., '23.25%↓') should be explained; it is unclear whether they are relative or absolute decreases. This affects readability of the ablation.
[Methodology, Knowledge Refiner] The hyperparameters α and β in the refiner are not given values. Provide the chosen values and, ideally, a sensitivity analysis.
[Figure 3] Figure 3 shows only the first 200 of 300 training steps. A note on whether the remaining 100 steps change the conclusions would help.
[References and Appendix] The paper uses the name 'ReCall' (Chen et al. 2025) but the reference list lists 'ReSearch'. Also, the appendix prompt in Figure 5 appears to be the planner agent prompt, not the local deep search agent prompt. Please correct these inconsistencies.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: HierSearch's training reward, evaluation metrics, and ablations all target external golden answers; self-citations are benchmarks, not load-bearing premises.

full rationale

The paper's central claim—that HierSearch beats flat RL and existing deep-search/RAG baselines—is an empirical result measured against held-out test sets from external benchmarks (NQ, HotpotQA, PubMedQA, BioASQ, MuSiQue, OmniEval) using EM/F1 against golden answers. The training reward (Eq. 1) is a rule-based F1 against the same golden answers, so the optimization target is not defined by the method itself. The knowledge refiner is an engineering heuristic that filters evidence by embedding similarity to an agent's thinking; it does not construct the final answer or the evaluation score. No load-bearing claim is justified by a self-citation: OmniEval is an author-created benchmark, but all methods are run on it under identical conditions and the main generalization evidence includes NQ/HotpotQA/PubMedQA, which are external and not in the training set. No uniqueness theorem or ansatz is imported from the authors' prior work. The only notable caveat is the local-corpus construction ('The sampling passages consist of directly related passages for questions and hard negatives retrieved by BM25'), which makes local retrieval artificially easy for five of six benchmarks; this is an evaluation-validity concern that could shrink the measured local-search advantage in a realistic independently-built enterprise corpus, but it is not a circular derivation because the method's outputs are still compared to external golden answers and no prediction is equivalent to its input by construction.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper introduces no physical or conceptual entities (no new particles, forces, or conserved quantities). The 'agents' and 'refiner' are software components. The main epistemic load is carried by the corpus construction assumption (test-derived passages in the local index) and by the unvalidated embedding-similarity proxy for evidence contribution.

free parameters (4)

alpha (first-step evidence selection proportion) = not reported
Top percentage of evidence selected in the first refining step by similarity to the next thinking process (Eq. 3). Chosen by hand and not disclosed.
beta (second-step evidence selection proportion) = not reported
Top percentage of remaining evidence selected in the second step by similarity to the agent's conclusion (Eq. 4). Not disclosed.
exploration reward coefficient = 0.1
Coefficient multiplying tool coverage t/T in the zero-F1 reward in Eq. (1). Hand-chosen.
per-call latency estimates = 43.99 ms/local, 2.30 s/web, 3.16 s/browse, 12.57 ms/token
Used in the efficiency analysis (Table 4) to compute latency instead of measuring end-to-end runtime.

assumptions (5)

ad hoc to paper The local retrieval corpus for general and medical benchmarks can be constructed from passages directly related to the test questions without invalidating the evaluation.
Stated in Implementation Details; introduces gold evidence into the index that would not exist in a fixed enterprise corpus.
domain assumption Embedding similarity between an evidence item and the agent's subsequent thinking measures the evidence's contribution to the final answer.
This is the premise of the knowledge refiner in Eq. (3); no validation is provided for this proxy.
domain assumption F1 score against the golden answer is an adequate reward for deep search agent training.
Used as the only correctness signal in Eq. (1); assumes golden answers cover the required knowledge.
domain assumption Flat RL is less data-efficient and gives poor mastery of web search tools in multi-source settings.
The motivating premise for HRL; supported only by the paper's preliminary experiments, whose details are not fully reported.
domain assumption Training on MuSiQue, OmniEval, and BioASQ generalizes to NQ, HotpotQA, and PubMedQA.
The out-of-domain test results in Table 1 rely on this transfer without an explicit distribution-shift analysis.

how reviews work

0 comments

Cite this review

Pith. "Pith review of HierSearch: A Hierarchical Enterprise Deep Search Framework Integrating Local and Web Searches." pith.science (2026). https://pith.science/paper/6OBCIYFC

@misc{pith2026250808088,
  author       = {Pith},
  title        = {Pith review of: HierSearch: A Hierarchical Enterprise Deep Search Framework Integrating Local and Web Searches},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6OBCIYFC}},
  note         = {Machine review of arXiv:2508.08088}
}

read the original abstract

Recently, large reasoning models have demonstrated strong mathematical and coding abilities, and deep search leverages their reasoning capabilities in challenging information retrieval tasks. Existing deep search works are generally limited to a single knowledge source, either local or the Web. However, enterprises often require private deep search systems that can leverage search tools over both local and the Web corpus. Simply training an agent equipped with multiple search tools using flat reinforcement learning (RL) is a straightforward idea, but it has problems such as low training data efficiency and poor mastery of complex tools. To address the above issue, we propose a hierarchical agentic deep search framework, HierSearch, trained with hierarchical RL. At the low level, a local deep search agent and a Web deep search agent are trained to retrieve evidence from their corresponding domains. At the high level, a planner agent coordinates low-level agents and provides the final answer. Moreover, to prevent direct answer copying and error propagation, we design a knowledge refiner that filters out hallucinations and irrelevant evidence returned by low-level agents. Experiments show that HierSearch achieves better performance compared to flat RL, and outperforms various deep search and multi-source retrieval-augmented generation baselines in six benchmarks across general, finance, and medical domains.

Figures

Figures reproduced from arXiv: 2508.08088 by the authors.

**Figure 1.** Illustration of the hierarchical agentic framework for HierSearch. We show exemplary trajectories of all low-level and [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

**Figure 2.** Illustration of the knowledge refining process from the local agent trajectory. The first step filters directly contributing [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Rewards on Validation Sets during Training. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (8 more)

**Figure 4.** Figure 4: The prompt for the flat deep search agent. [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]

**Figure 5.** Figure 5: The prompt for the local deep search agent. [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

**Figure 6.** Figure 6: The prompt for the Web deep search agent. [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

**Figure 7.** Figure 7: The prompt for the planner agent. Question: Who is the sibling of the author of Kapalkundala? Golden Answer: Sanjib Chandra Chattopadhyay <all search agent> Who is the sibling of the author of Kapalkundala </all search agent> <result> Local Chunk Corpus: Kapalkundala K…

**Figure 8.** Figure 8: Example trajectory for the planner agent. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]

**Figure 9.** Figure 9: Example trajectory for the local deep search agent. [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]

**Figure 10.** Figure 10: Example trajectory for the Web deep search agent. [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]

**Figure 11.** Figure 11: Example trajectory for the flat-RL-trained deep search agent. [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Fetch-then-Explore: Decoupling Selection from Extraction over a Persistent Workspace for Search Agents
cs.AI 2026-08 conditional novelty 6.0 of 10

Fetch-then-Explore, which stores fetched pages in a per-question workspace and extracts evidence on demand with grep/read, beats visit-and-read and browsing baselines on BrowseComp across three LLM backbones.

Reference graph

Works this paper leans on

4 extracted references · 4 linked inside Pith · cited by 1 Pith paper

[3]

In Globersons, A.; Mackey, L.; Belgrave, D.; Fan, A.; Paquet, U.; Tomczak, J

HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models. In Globersons, A.; Mackey, L.; Belgrave, D.; Fan, A.; Paquet, U.; Tomczak, J. M.; and Zhang, C., eds.,Advances in Neural Information Processing Systems 38: Annual Conference on Neural Infor- mation Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15...

arXiv 2024
[4]

In Long, G.; Blumestein, M.; Chang, Y .; Lewin-Eytan, L.; Huang, Z

HtmlRAG: HTML is Better Than Plain Text for Mod- eling Retrieved Knowledge in RAG Systems. In Long, G.; Blumestein, M.; Chang, Y .; Lewin-Eytan, L.; Huang, Z. H.; and Yom-Tov, E., eds.,Proceedings of the ACM on Web Con- ference 2025, WWW 2025, Sydney, NSW, Australia, 28 April 2025- 2 May 2025, 1733–1746. ACM. Team, Q. 2025. QwQ-32B: Embracing the Power of...

arXiv 2025
[2024]

InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vi- enna, Austria, May 7-11, 2024

Self-RAG: Learning to Retrieve, Generate, and Cri- tique through Self-Reflection. InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vi- enna, Austria, May 7-11, 2024. OpenReview.net. Chen, J.; Xiao, S.; Zhang, P.; Luo, K.; Lian, D.; and Liu, Z. 2024. BGE M3-Embedding: Multi-Lingual, Multi- Functionality, Multi-Granularity Text...

arXiv 2024
[2025]

Chowdhury, N.; Johnson, D.; Huang, V .; Steinhardt, J.; and Schwettmann, S

ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning. Chowdhury, N.; Johnson, D.; Huang, V .; Steinhardt, J.; and Schwettmann, S. 2025. Investigating truthfulness in a pre- release o3 model. Technical report, Transluce. Dayan, P.; and Hinton, G. E. 1992. Feudal Reinforcement Learning. In Hanson, S. J.; Cowan, J. D.; and Giles, C. L...

arXiv 2025

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [3]

In Globersons, A.; Mackey, L.; Belgrave, D.; Fan, A.; Paquet, U.; Tomczak, J

HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models. In Globersons, A.; Mackey, L.; Belgrave, D.; Fan, A.; Paquet, U.; Tomczak, J. M.; and Zhang, C., eds.,Advances in Neural Information Processing Systems 38: Annual Conference on Neural Infor- mation Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15...

arXiv 2024

[2] [4]

In Long, G.; Blumestein, M.; Chang, Y .; Lewin-Eytan, L.; Huang, Z

HtmlRAG: HTML is Better Than Plain Text for Mod- eling Retrieved Knowledge in RAG Systems. In Long, G.; Blumestein, M.; Chang, Y .; Lewin-Eytan, L.; Huang, Z. H.; and Yom-Tov, E., eds.,Proceedings of the ACM on Web Con- ference 2025, WWW 2025, Sydney, NSW, Australia, 28 April 2025- 2 May 2025, 1733–1746. ACM. Team, Q. 2025. QwQ-32B: Embracing the Power of...

arXiv 2025

[3] [2024]

InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vi- enna, Austria, May 7-11, 2024

Self-RAG: Learning to Retrieve, Generate, and Cri- tique through Self-Reflection. InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vi- enna, Austria, May 7-11, 2024. OpenReview.net. Chen, J.; Xiao, S.; Zhang, P.; Luo, K.; Lian, D.; and Liu, Z. 2024. BGE M3-Embedding: Multi-Lingual, Multi- Functionality, Multi-Granularity Text...

arXiv 2024

[4] [2025]

Chowdhury, N.; Johnson, D.; Huang, V .; Steinhardt, J.; and Schwettmann, S

ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning. Chowdhury, N.; Johnson, D.; Huang, V .; Steinhardt, J.; and Schwettmann, S. 2025. Investigating truthfulness in a pre- release o3 model. Technical report, Transluce. Dayan, P.; and Hinton, G. E. 1992. Feudal Reinforcement Learning. In Hanson, S. J.; Cowan, J. D.; and Giles, C. L...

arXiv 2025