Tree-Based Text Retrieval via Hierarchical Clustering in RAGFrameworks: Application on Taiwanese Regulations

Chia-Heng Yu; Yen-lung Tsai

REVIEW 4 major objections 5 minor 11 references

Tree-Based Text Retrieval via Hierarchical Clustering in RAGFrameworks: Application on Taiwanese Regulations

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Hierarchical clustering beats top-k retrieval in legal RAG.

desk verdict A plausible, incremental no-k RAG retrieval method backed by real (if narrow) expert-graded results; it deserves peer review but the mechanism needs stronger justification and the claimed efficiency gain is overstated. read the letter →

arxiv 2506.13607 v1 pith:G2TLGX7A submitted 2025-06-16 cs.IR

Chia-Heng Yu , Yen-Lung Tsai This is my paper · ORCID

classification cs.IR

keywords retrieval-augmentedgenerationhierarchicalclusteringtop-kretrievallegaltextqueryexpansionembeddingsimilaritydendrogramTaiwaneseregulations

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the standard top-k retrieval step in retrieval-augmented generation can be replaced by a hierarchical clustering tree: chunk embeddings are agglomeratively clustered under cosine distance, each non-leaf node stores the mean vector of its members, and a query returns the leaf texts of the subtree whose centroid is most similar to the query. On a corpus of Taiwanese regulations with expert-graded questions, the authors report that both this tree method and a variant that first extracts query facts score statistically significantly higher total expert ratings than the brute-force inner-product baseline, with one-tailed p-values of $0.027$ and $0.022$. If the claim holds, a RAG pipeline gains a retriever that needs no preset $k$ and returns an adaptively sized set of relevant provisions rather than a fixed-size list, which is a practical fix for a real deployment pain.

What carries the argument

The central object is a dendrogram produced by bottom-up (agglomerative) hierarchical clustering of the text chunk vectors $\{v_1,\ldots,v_N\}$ using the cosine distance $d(v,w)=1-\langle v,w\rangle/(\|v\|\|w\|)$ with single linkage, where each non-leaf node stores the mean of its children's vectors. At query time the algorithm checks every node's mean vector, picks the single closest node to the query vector, and returns all leaf texts under it, so the number of retrieved chunks adapts to the subtree's size without a preset $k$. Construction costs $O(N^2)$; retrieval costs $O(N)$.

What would settle it

Evaluate the method on a second statutory corpus, such as U.S. federal regulations, with expert-graded questions and a per-query tuned top-k baseline: the central claim is contradicted if the tree's total expert score no longer exceeds the baseline, or if some query's chosen subtree excludes all ground-truth provisions that the top-k retrieval would have included.

Watch

Extended reading notes

Core claim

The paper's central claim is that walking a dendrogram built by agglomerative hierarchical clustering with cosine distance, then outputting the leaf texts of the single subtree whose stored centroid is nearest to the query embedding, produces a retrieval set that improves expert-judged answer quality in a RAG pipeline relative to brute-force inner-product top-k retrieval. In the reported experiment on expert-graded Taiwanese legal questions, the Tree method and the Tree with query-extraction both had higher mean total scores than the Origin baseline, with paired one-tailed t-tests at $p=0.027$ and $p=0.022$ and Cohen's $d$ of $0.458$ and $0.484$. The paper presents the method as a way to eliminate the $k$ hyperparameter while keeping or improving accuracy and adding interpretability, because the selected subtree groups retrieved provisions under a semantically relevant node.

Load-bearing premise

The method assumes that cosine similarity in the embedding space of 200-character legal chunks forms clusters whose mean vectors reliably indicate legal relevance, so that the subtree whose centroid is nearest to the query contains the relevant provisions; if embeddings do not track legal topicality, the tree will select irrelevant subtrees and the measured gains will not generalize.

Editorial extensions

If this is right

RAG pipelines can drop the $k$ hyperparameter and use the tree as a drop-in replacement for the top-k search component, without retraining the retriever.
Retrieval becomes adaptive: a query whose relevant material sits in a dense cluster returns a small set of provisions, while a broad topic returns a larger subtree, instead of a fixed-size list.
The Tree plus query-extraction variant indicates that a short chain-of-thought reformulation of the legal question before retrieval sharpens the selected subtree's relevance.
Because the tree is built offline from the corpus embeddings and query time is linear in the number of chunks, the method is feasible for low-resource deployments that lack a dedicated approximate nearest-neighbor index.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the embedding space preserves legal topicality, the same tree could serve as a default retriever for other structured corpora such as contracts, patents, or administrative regulations, where k-free retrieval would avoid context-length failures for broad queries.
The comparison against a fixed top-k baseline leaves open how much of the gain comes from adapting the number of chunks rather than from the tree structure; a sharper test would tune $k$ per query and then compare against that tuned upper bound.
A natural extension is to replace single-best-subtree selection with a small mixture of the best few nodes, which could help queries whose relevant provisions fall in several distant clusters.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a useful, clearly described incremental contribution to RAG retrieval, not a breakthrough. It deserves a real referee, but the authors need to do more to make the mechanism credible and to stop overstating the complexity gain.

What the paper gets right: removing the k parameter is a genuine practical convenience, and the significance-tested results on an expert-graded Taiwanese legal corpus do support the claim that tree-based retrieval can beat brute-force top-k in this setting. The specific pipeline is new enough to report: BGE-M3 embeddings, single-linkage cosine hierarchical clustering, centroid vectors stored at internal nodes, and retrieval by nearest centroid, with code provided. The algorithm descriptions are legible even though the English prose elsewhere is heavily garbled, and Section 6 is unusually candid about untested domain transfer and LLM reliance.

Where I have real doubts: the stress-test concern is on target. Single linkage builds elongated chains; the arithmetic mean of a chain does not necessarily represent the cluster's dominant legal topic. The authors never show that centroid distance orders subtrees by legal relevance, so the mechanism is undersold. That said, the empirical claim does not collapse—the paper can survive with the weaker, honest claim that this procedure performs well on this corpus. A good referee should ask for a comparison against RAPTOR-style recursive summarization and against average-linkage or Ward variants to see whether the linkage choice actually matters.

The second soft spot is the time complexity claim. Algorithm 2 visits every node, so query time is O(N), same order as brute force over N in the embedding space. The contribution is 'no k', not speed, and the paper should stop implying otherwise.

Third, the evaluation is narrow: one small corpus, one baseline, one embedding model, expert grading without inter-rater reliability, and no public data. That is a scope limitation, not a fatal flaw. The paired comparisons with Cohen's d around 0.4 are at least real evidence, and the one-tailed p-values are not being oversold.

Net: I would send this to peer review. The core idea is simple and reproducible, the missing comparisons are exactly what a good referee can demand, and for a practitioner building legal RAG with limited compute, the paper is genuinely worth reading. For an IR researcher, it is a data point rather than a foundation, but an honest one.

Referee Report

4 major / 5 minor

Summary. The paper proposes replacing top-k retrieval in a retrieval-augmented generation (RAG) pipeline with a retrieval tree built by single-linkage hierarchical clustering over text chunks embedded with BGE-M3. At query time, Algorithm 2 computes the cosine distance between the query vector and every node's mean vector, selects the closest node, and returns all leaf texts under that node, thereby avoiding a predefined k. The authors compare this Tree method and a Tree+Query-Extraction variant against a brute-force inner-product baseline on Taiwanese legal questions, using expert grading and adjusted F1 scores, and report statistically significant improvements in total score. A code repository is provided.

Significance. If the claimed effect is robust, the paper offers a simple, adaptive retrieval mechanism that removes the need to tune k and could be useful for legal-domain RAG under limited resources. The paper has clear strengths: it ships reproducible code, reports statistical tests with effect sizes, and includes an explicit limitations section. However, the mechanism connecting the clustering construction to the retrieval decision is not established, and the empirical evidence rests on one small legal corpus, one baseline, and subjective expert grading without inter-rater reliability. The contribution is therefore plausible but not yet convincing at the level of a general retrieval method.

major comments (4)

[§3.2, Time Complexity] Algorithm 2 visits every node in the tree, computing a cosine distance at each node, so the search phase is O(N) in the number of leaf chunks. Brute-force inner-product top-k retrieval also requires O(N) distance computations (plus a small selection step). The paper's claim that the method is more efficient or more scalable than brute-force retrieval is therefore not supported by the stated complexity; the actual advantage is limited to avoiding the separate k-selection step, not avoiding a full scan of the collection.
[§3.1.2, Algorithm 1 and §3.2, Algorithm 2] The dendrogram is constructed with single linkage, where merges are decided by the minimum pairwise document distance between clusters, but the node vectors used for retrieval are the arithmetic means of all members in each cluster. Single linkage can produce elongated, chained clusters whose mean vector is not representative of the cluster's legal content and may be closer to unrelated subtrees. The paper provides neither a theoretical argument nor an experiment showing that, for BGE-M3 embeddings of 200-character legal chunks, centroid proximity of single-linkage clusters orders subtrees by legal relevance. The central retrieval mechanism is therefore not explained by the stated construction, and Section 6's limitations do not address this mismatch.
[§5, baseline comparison] The experimental comparison against 'brute-force inner product search' does not state the value of k used for the baseline or report a k-sensitivity analysis. Since the paper's motivation is that choosing k is difficult, the empirical claim that the tree method improves over top-k is only meaningful if the baseline is shown across a range of reasonable k values. Without that range, the observed improvements could be an artifact of a poorly chosen baseline k.
[§4.3 and §6, expert rating] The evaluation corpus appears to consist of a small number of expert-graded Taiwanese legal questions, but the paper does not report the number of queries, the number of raters, or any inter-rater reliability measure. The total score combines an expert's subjective 1–5 rating with an adjusted F1 score, and the limitations section acknowledges domain transfer and LLM reliance but not the single-rater subjectivity. The reported p-values are therefore difficult to interpret as evidence for a general retrieval advantage.

minor comments (5)

[Global] The submitted text contains extensive garbled or placeholder characters, which makes several sections, including the algorithms and the appendix prompts, very difficult to verify; the arXiv source should be cleaned before publication.
[Figure 5] The caption refers to 'Tree+Query' while the text and Table 2 use 'Tree+QE'; the terminology should be unified throughout.
[Table 2] The statistical tests are reported as one-tailed p-values; the choice of a one-tailed test should be justified, or two-tailed values should also be reported, because the direction of the effect is part of the claim.
[§3.2] Algorithm 2 does not specify tie-breaking when multiple nodes have the same minimum distance to the query, which can make the returned subtree nondeterministic at equal distances.
[§4.1] The chunk size of 200 characters with 40-character overlap is chosen without a sensitivity analysis; since chunking is a free parameter that directly affects retrieval quality, a short ablation would strengthen the method's practicality.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the retrieval comparison is empirical and self-contained, with no fitted constants or self-citation chain carrying the central claim.

full rationale

The paper's central claim is an empirical comparison of three retrieval methods (brute-force top-k baseline, hierarchical clustering tree, and tree with query extraction) on expert-graded Taiwanese legal questions. The derivation chain consists of (i) chunking legal texts, (ii) embedding chunks with the externally specified BGE-M3 model, (iii) building a single-linkage dendrogram over those embeddings, (iv) selecting the tree node whose stored centroid is closest to the query, and (v) returning that node's leaves as retrieved context. No parameter is fitted to the evaluation data: chunk size, overlap, the cosine distance, and the linkage rule are fixed before the experiments; the expert rating rubric and the 0.5/0.5 total-score weighting are evaluation conventions, not hidden inputs to the method. The statistical significance tests in Table 2 compare actual system outputs against the baseline, so the reported p-values are genuine outcomes rather than re-statements of assumptions. There are no self-citations used as evidence for the method's validity, no uniqueness theorem imported from the authors' prior work, and no ansatz smuggled in via citation. The single-linkage centroid representativeness concern is a legitimate robustness worry about whether the mechanism will generalize, but it is not circularity: the paper does not define the tree or the search rule in terms of the evaluation scores. The paper is self-contained against the empirical benchmark, and no prediction reduces by construction to its inputs.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim rests on embedding quality and clustering coherence, not on fitted constants. The only hand-chosen numbers are preprocessing and evaluation hyperparameters.

free parameters (4)

Chunk window size = 200 characters
Chosen by hand in Section 4.1 for preprocessing; affects granularity of retrieval units.
Chunk overlap = 50 characters
Chosen in Section 4.1; affects boundary coverage.
F_beta beta = 4
Beta value in the adjusted F1 metric in Section 4.3; weights recall heavily.
Total score weights = 0.5*Expert + 0.5*5*F1_adjusted
Weights chosen in Section 4.3 to combine expert and F1 scores.

assumptions (3)

domain assumption BGE-M3 embeddings capture semantic similarity for Taiwanese legal Chinese text.
The whole retrieval relies on cosine similarity between embeddings; Section 4.2 selects the model without legal-domain adaptation.
domain assumption Single-linkage hierarchical clustering with mean node vectors produces semantically coherent clusters.
Node vectors are computed as means of children (Algorithm 1), and single-linkage can produce chaining; Section 3.1.2.
domain assumption Expert ratings on a 1-5 scale are a reliable measure of answer quality.
Primary outcome is expert grading with no inter-rater reliability reported; Section 4.3.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Tree-Based Text Retrieval via Hierarchical Clustering in RAGFrameworks: Application on Taiwanese Regulations." pith.science (2026). https://pith.science/paper/G2TLGX7A

@misc{pith2026250613607,
  author       = {Pith},
  title        = {Pith review of: Tree-Based Text Retrieval via Hierarchical Clustering in RAGFrameworks: Application on Taiwanese Regulations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G2TLGX7A}},
  note         = {Machine review of arXiv:2506.13607}
}

read the original abstract

Traditional Retrieval-Augmented Generation (RAG) systems employ brute-force inner product search to retrieve the top-k most similar documents, then combined with the user query and passed to a language model. This allows the model to access external knowledge and reduce hallucinations. However, selecting an appropriate k value remains a significant challenge in practical applications: a small k may fail to retrieve sufficient information, while a large k can introduce excessive and irrelevant content. To address this, we propose a hierarchical clustering-based retrieval method that eliminates the need to predefine k. Our approach maintains the accuracy and relevance of system responses while adaptively selecting semantically relevant content. In the experiment stage, we applied our method to a Taiwanese legal dataset with expert-graded queries. The results show that our approach achieves superior performance in expert evaluations and maintains high precision while eliminating the need to predefine k, demonstrating improved accuracy and interpretability in legal text retrieval tasks. Our framework is simple to implement and easily integrates with existing RAG pipelines, making it a practical solution for real-world applications under limited resources.

Figures

Figures reproduced from arXiv: 2506.13607 by the authors.

**Figure 2.** Hierarchical Clustering using Single-Linkage and Cosine Similarity. For each text [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** How to search in the retrieval tree. Given two queries vector [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Average Expert Score, F1 Score, and Total Score for the three retrieval methods. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

**Figure 5.** Figure 5: Box plot of the score differences between the Tree method and the Tree+Query [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

11 extracted references · 6 canonical work pages

[1]

u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt\

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K\" u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt\" a schel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive nlp tasks. In Proceedings of the 34th International Conference on Neural Information Processing Systems ...

work page 2020
[2]

Attention is All you Need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is All you Need . In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 30. Curran Associates, Inc., 2017

work page 2017
[3]

A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions

Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Trans. Inf. Syst. , 43(2), January 2025

work page 2025
[4]

Dense passage retrieval for open-domain question answering

Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages 6769--6781, Onli...

work page 2020
[5]

Large language models are effective text rankers with pairwise ranking prompting

Zhen Qin, Rolf Jagerman, Kai Hui, Honglei Zhuang, Junru Wu, Le Yan, Jiaming Shen, Tianqi Liu, Jialu Liu, Donald Metzler, Xuanhui Wang, and Michael Bendersky. Large language models are effective text rankers with pairwise ranking prompting. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Findings of the Association for Computational Linguistics: N...

work page 2024
[6]

Answering complex open-domain questions with multi-hop dense retrieval, 2020

Wenhan Xiong, Xiang Lorraine Li, Srini Iyer, Jingfei Du, Patrick Lewis, William Yang Wang, Yashar Mehdad, Wen tau Yih, Sebastian Riedel, Douwe Kiela, and Barlas Oğuz. Answering complex open-domain questions with multi-hop dense retrieval, 2020. arXiv preprint arXiv:2009.12756

arXiv 2020
[7]

Parth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna, Anna Goldie, and Christopher D. Manning. Raptor: Recursive abstractive processing for tree-organized retrieval, 2024. arXiv preprint arXiv:2401.18059

arXiv 2024
[8]

From local to global: A graph rag approach to query-focused summarization, 2024

Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson. From local to global: A graph rag approach to query-focused summarization, 2024. arXiv preprint arXiv:2404.16130

arXiv 2024

Show all 11 references

[9]

Chi, Quoc V

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the 36th International Conference on Neural Information Processing Sys...

2022
[10]

Multilingual e5 text embeddings: A technical report, 2024

Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. Multilingual e5 text embeddings: A technical report, 2024. arXiv preprint arXiv:2402.05672

2024 arXiv
[11]

Gpt-4o-mini models, 2025

OpenAI. Gpt-4o-mini models, 2025. Available at: https://platform.openai.com/docs/models/gpt-4o-mini (accessed: 2025-04-12)

2025

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt\

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K\" u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt\" a schel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive nlp tasks. In Proceedings of the 34th International Conference on Neural Information Processing Systems ...

work page 2020

[2] [2]

Attention is All you Need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is All you Need . In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 30. Curran Associates, Inc., 2017

work page 2017

[3] [3]

A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions

Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Trans. Inf. Syst. , 43(2), January 2025

work page 2025

[4] [4]

Dense passage retrieval for open-domain question answering

Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages 6769--6781, Onli...

work page 2020

[5] [5]

Large language models are effective text rankers with pairwise ranking prompting

Zhen Qin, Rolf Jagerman, Kai Hui, Honglei Zhuang, Junru Wu, Le Yan, Jiaming Shen, Tianqi Liu, Jialu Liu, Donald Metzler, Xuanhui Wang, and Michael Bendersky. Large language models are effective text rankers with pairwise ranking prompting. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Findings of the Association for Computational Linguistics: N...

work page 2024

[6] [6]

Answering complex open-domain questions with multi-hop dense retrieval, 2020

Wenhan Xiong, Xiang Lorraine Li, Srini Iyer, Jingfei Du, Patrick Lewis, William Yang Wang, Yashar Mehdad, Wen tau Yih, Sebastian Riedel, Douwe Kiela, and Barlas Oğuz. Answering complex open-domain questions with multi-hop dense retrieval, 2020. arXiv preprint arXiv:2009.12756

arXiv 2020

[7] [7]

Parth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna, Anna Goldie, and Christopher D. Manning. Raptor: Recursive abstractive processing for tree-organized retrieval, 2024. arXiv preprint arXiv:2401.18059

arXiv 2024

[8] [8]

From local to global: A graph rag approach to query-focused summarization, 2024

Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson. From local to global: A graph rag approach to query-focused summarization, 2024. arXiv preprint arXiv:2404.16130

arXiv 2024