{"id":"66b6ce62-c644-4f2f-9927-70ad29726e0d","arxiv_id":"2506.13607","paper_version":1,"verdict":"CONDITIONAL","confidence":"LOW","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A hierarchical clustering tree with cosine-similarity centroids can replace top-k retrieval in RAG and improve expert-rated legal answers without tuning k.","lead":"This paper presents a retrieval method that builds a hierarchical clustering tree from legal text embeddings and selects the subtree nearest to a query, eliminating the need to choose a top-k value in retrieval-augmented generation. The authors report statistically significant improvements in expert-rated answer quality over brute-force top-k search on a Taiwanese regulatory corpus.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Centroid-based subtree selection is built on single-linkage clusters whose centroids need not be representative; the search mechanism lacks support.","rationale":"Reader already conditional low confidence. I agree with the general worry that embedding/cluster geometry is the weak point, but I want to sharpen it: the mismatch between single-linkage construction and centroid-based querying is internal to the method and testable. Single linkage merges clusters by closest pair, not by centroid closeness, so the stored mean is not the quantity that governed the merge. This can make nearest-centroid selection pick a large chained cluster whose leaves are not topically coherent. The proposed experiment isolates this factor by changing only the linkage rule; it also provides a direct diagnostic. If the result survives, the concern is resolved and conditional acceptance is fine; if not, the central claim needs revision. No ad hominem; the paper is transparent about data and limitations, and the statistical tests are reported, so the issue is one of mechanism, not integrity.","tokens_in":33412,"tokens_out":5306,"duration_ms":58569,"concrete_test":"Re-run the full pipeline on the same expert-graded queries using UPGMA/Ward (centroid/mean linkage) instead of single linkage, keeping Algorithm 2, prompts, embeddings, and evaluation fixed; if the Tree and Tree+QE gains over brute-force top-k in Table 2 vanish or reverse, the claimed benefit depends on the single-linkage/centroid mismatch. Also report the average cosine distance from leaves to their node centroid versus the average leaf-leaf distance for nodes selected by Algorithm 2, to quantify how unrepresentative the centroids are.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's core mechanism is Algorithm 2 (§3.2): among all tree nodes, pick the one whose mean vector v_node is closest to the query, then return its leaves. For this to retrieve relevant law, the mean of each cluster must be a faithful summary of the legal content of its leaves. But Algorithm 1 (§3.1.2) constructs the dendrogram with single linkage, merging clusters by min_{a in C_i, b in C_j} d(a,b), and only afterwards stores the arithmetic mean as the node vector. Single linkage is known to form elongated, chained clusters: a chain of pairwise-similar documents can be merged even though many members are far from the cluster centroid and similar to documents outside the cluster. Nothing in §3 shows that for BGE-M3 embeddings of 200-character legal chunks, centroids of single-linkage clusters order subtrees by legal relevance. The reported advantage over brute-force top-k is therefore not actually explained by the stated tree mechanism; it could be an artifact of the corpus, the embedding geometry, or the prompt/evaluation. Section 6's own limitations do not address this mismatch, only domain transfer and LLM reliance.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes replacing top-k retrieval in a retrieval-augmented generation (RAG) pipeline with a retrieval tree built by single-linkage hierarchical clustering over text chunks embedded with BGE-M3. At query time, Algorithm 2 computes the cosine distance between the query vector and every node's mean vector, selects the closest node, and returns all leaf texts under that node, thereby avoiding a predefined k. The authors compare this Tree method and a Tree+Query-Extraction variant against a brute-force inner-product baseline on Taiwanese legal questions, using expert grading and adjusted F1 scores, and report statistically significant improvements in total score. A code repository is provided.","tokens_in":33607,"tokens_out":4733,"duration_ms":54272,"significance":"If the claimed effect is robust, the paper offers a simple, adaptive retrieval mechanism that removes the need to tune k and could be useful for legal-domain RAG under limited resources. The paper has clear strengths: it ships reproducible code, reports statistical tests with effect sizes, and includes an explicit limitations section. However, the mechanism connecting the clustering construction to the retrieval decision is not established, and the empirical evidence rests on one small legal corpus, one baseline, and subjective expert grading without inter-rater reliability. The contribution is therefore plausible but not yet convincing at the level of a general retrieval method.","major_comments":[{"comment":"Algorithm 2 visits every node in the tree, computing a cosine distance at each node, so the search phase is O(N) in the number of leaf chunks. Brute-force inner-product top-k retrieval also requires O(N) distance computations (plus a small selection step). The paper's claim that the method is more efficient or more scalable than brute-force retrieval is therefore not supported by the stated complexity; the actual advantage is limited to avoiding the separate k-selection step, not avoiding a full scan of the collection.","section":"§3.2, Time Complexity"},{"comment":"The dendrogram is constructed with single linkage, where merges are decided by the minimum pairwise document distance between clusters, but the node vectors used for retrieval are the arithmetic means of all members in each cluster. Single linkage can produce elongated, chained clusters whose mean vector is not representative of the cluster's legal content and may be closer to unrelated subtrees. The paper provides neither a theoretical argument nor an experiment showing that, for BGE-M3 embeddings of 200-character legal chunks, centroid proximity of single-linkage clusters orders subtrees by legal relevance. The central retrieval mechanism is therefore not explained by the stated construction, and Section 6's limitations do not address this mismatch.","section":"§3.1.2, Algorithm 1 and §3.2, Algorithm 2"},{"comment":"The experimental comparison against 'brute-force inner product search' does not state the value of k used for the baseline or report a k-sensitivity analysis. Since the paper's motivation is that choosing k is difficult, the empirical claim that the tree method improves over top-k is only meaningful if the baseline is shown across a range of reasonable k values. Without that range, the observed improvements could be an artifact of a poorly chosen baseline k.","section":"§5, baseline comparison"},{"comment":"The evaluation corpus appears to consist of a small number of expert-graded Taiwanese legal questions, but the paper does not report the number of queries, the number of raters, or any inter-rater reliability measure. The total score combines an expert's subjective 1–5 rating with an adjusted F1 score, and the limitations section acknowledges domain transfer and LLM reliance but not the single-rater subjectivity. The reported p-values are therefore difficult to interpret as evidence for a general retrieval advantage.","section":"§4.3 and §6, expert rating"}],"minor_comments":[{"comment":"The submitted text contains extensive garbled or placeholder characters, which makes several sections, including the algorithms and the appendix prompts, very difficult to verify; the arXiv source should be cleaned before publication.","section":"Global"},{"comment":"The caption refers to 'Tree+Query' while the text and Table 2 use 'Tree+QE'; the terminology should be unified throughout.","section":"Figure 5"},{"comment":"The statistical tests are reported as one-tailed p-values; the choice of a one-tailed test should be justified, or two-tailed values should also be reported, because the direction of the effect is part of the claim.","section":"Table 2"},{"comment":"Algorithm 2 does not specify tie-breaking when multiple nodes have the same minimum distance to the query, which can make the returned subtree nondeterministic at equal distances.","section":"§3.2"},{"comment":"The chunk size of 200 characters with 40-character overlap is chosen without a sensitivity analysis; since chunking is a free parameter that directly affects retrieval quality, a short ablation would strengthen the method's practicality.","section":"§4.1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's central idea is appealing, but the retrieval mechanism hinges on centroid vectors of single-linkage clusters, and that connection is currently unsupported. I would ask for either a direct evaluation of whether the selected subtree contains the relevant chunks, or a comparison with an alternative linkage/centroid representation, before endorsing the method. The evaluation also needs to report the baseline k, a k-sensitivity curve, and inter-rater reliability. These are fixable within the manuscript's scope, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a useful, clearly described incremental contribution to RAG retrieval, not a breakthrough. It deserves a real referee, but the authors need to do more to make the mechanism credible and to stop overstating the complexity gain.\n\nWhat the paper gets right: removing the k parameter is a genuine practical convenience, and the significance-tested results on an expert-graded Taiwanese legal corpus do support the claim that tree-based retrieval can beat brute-force top-k in this setting. The specific pipeline is new enough to report: BGE-M3 embeddings, single-linkage cosine hierarchical clustering, centroid vectors stored at internal nodes, and retrieval by nearest centroid, with code provided. The algorithm descriptions are legible even though the English prose elsewhere is heavily garbled, and Section 6 is unusually candid about untested domain transfer and LLM reliance.\n\nWhere I have real doubts: the stress-test concern is on target. Single linkage builds elongated chains; the arithmetic mean of a chain does not necessarily represent the cluster's dominant legal topic. The authors never show that centroid distance orders subtrees by legal relevance, so the mechanism is undersold. That said, the empirical claim does not collapse—the paper can survive with the weaker, honest claim that this procedure performs well on this corpus. A good referee should ask for a comparison against RAPTOR-style recursive summarization and against average-linkage or Ward variants to see whether the linkage choice actually matters.\n\nThe second soft spot is the time complexity claim. Algorithm 2 visits every node, so query time is O(N), same order as brute force over N in the embedding space. The contribution is 'no k', not speed, and the paper should stop implying otherwise.\n\nThird, the evaluation is narrow: one small corpus, one baseline, one embedding model, expert grading without inter-rater reliability, and no public data. That is a scope limitation, not a fatal flaw. The paired comparisons with Cohen's d around 0.4 are at least real evidence, and the one-tailed p-values are not being oversold.\n\nNet: I would send this to peer review. The core idea is simple and reproducible, the missing comparisons are exactly what a good referee can demand, and for a practitioner building legal RAG with limited compute, the paper is genuinely worth reading. For an IR researcher, it is a data point rather than a foundation, but an honest one.","headline":"A plausible, incremental no-k RAG retrieval method backed by real (if narrow) expert-graded results; it deserves peer review but the mechanism needs stronger justification and the claimed efficiency gain is overstated.","tokens_in":34105,"tokens_out":1718,"would_cite":false,"duration_ms":21047,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Hierarchical clustering beats top-k retrieval in legal RAG.","keywords":["retrieval-augmented generation","hierarchical clustering","top-k retrieval","legal text retrieval","query expansion","embedding similarity","dendrogram","Taiwanese regulations"],"falsifier":"Evaluate the method on a second statutory corpus, such as U.S. federal regulations, with expert-graded questions and a per-query tuned top-k baseline: the central claim is contradicted if the tree's total expert score no longer exceeds the baseline, or if some query's chosen subtree excludes all ground-truth provisions that the top-k retrieval would have included.","tokens_in":33224,"feed_emoji":"🌳","tokens_out":12440,"duration_ms":113814,"temperature":0.7,"pith_summary":"The paper claims that the standard top-k retrieval step in retrieval-augmented generation can be replaced by a hierarchical clustering tree: chunk embeddings are agglomeratively clustered under cosine distance, each non-leaf node stores the mean vector of its members, and a query returns the leaf texts of the subtree whose centroid is most similar to the query. On a corpus of Taiwanese regulations with expert-graded questions, the authors report that both this tree method and a variant that first extracts query facts score statistically significantly higher total expert ratings than the brute-force inner-product baseline, with one-tailed p-values of $0.027$ and $0.022$. If the claim holds, a RAG pipeline gains a retriever that needs no preset $k$ and returns an adaptively sized set of relevant provisions rather than a fixed-size list, which is a practical fix for a real deployment pain.","feed_headline":"Tree retrieval beats top-k in legal RAG, no k needed","feed_subtitle":"On expert-graded Taiwanese regulations, the clustering tree scored higher than the inner-product baseline.","key_machinery":"The central object is a dendrogram produced by bottom-up (agglomerative) hierarchical clustering of the text chunk vectors $\\{v_1,\\ldots,v_N\\}$ using the cosine distance $d(v,w)=1-\\langle v,w\\rangle/(\\|v\\|\\|w\\|)$ with single linkage, where each non-leaf node stores the mean of its children's vectors. At query time the algorithm checks every node's mean vector, picks the single closest node to the query vector, and returns all leaf texts under it, so the number of retrieved chunks adapts to the subtree's size without a preset $k$. Construction costs $O(N^2)$; retrieval costs $O(N)$.","core_discovery":"The paper's central claim is that walking a dendrogram built by agglomerative hierarchical clustering with cosine distance, then outputting the leaf texts of the single subtree whose stored centroid is nearest to the query embedding, produces a retrieval set that improves expert-judged answer quality in a RAG pipeline relative to brute-force inner-product top-k retrieval. In the reported experiment on expert-graded Taiwanese legal questions, the Tree method and the Tree with query-extraction both had higher mean total scores than the Origin baseline, with paired one-tailed t-tests at $p=0.027$ and $p=0.022$ and Cohen's $d$ of $0.458$ and $0.484$. The paper presents the method as a way to eliminate the $k$ hyperparameter while keeping or improving accuracy and adding interpretability, because the selected subtree groups retrieved provisions under a semantically relevant node.","pith_inferences":["If the embedding space preserves legal topicality, the same tree could serve as a default retriever for other structured corpora such as contracts, patents, or administrative regulations, where k-free retrieval would avoid context-length failures for broad queries.","The comparison against a fixed top-k baseline leaves open how much of the gain comes from adapting the number of chunks rather than from the tree structure; a sharper test would tune $k$ per query and then compare against that tuned upper bound.","A natural extension is to replace single-best-subtree selection with a small mixture of the best few nodes, which could help queries whose relevant provisions fall in several distant clusters."],"forward_implications":["RAG pipelines can drop the $k$ hyperparameter and use the tree as a drop-in replacement for the top-k search component, without retraining the retriever.","Retrieval becomes adaptive: a query whose relevant material sits in a dense cluster returns a small set of provisions, while a broad topic returns a larger subtree, instead of a fixed-size list.","The Tree plus query-extraction variant indicates that a short chain-of-thought reformulation of the legal question before retrieval sharpens the selected subtree's relevance.","Because the tree is built offline from the corpus embeddings and query time is linear in the number of chunks, the method is feasible for low-resource deployments that lack a dedicated approximate nearest-neighbor index."],"supporting_citations":[{"why":"Defines retrieval-augmented generation and the top-k retrieval step whose k-selection problem this paper removes.","marker":"[1]"},{"why":"Chain-of-thought prompting, used by the Tree+QE variant to extract query facts before retrieval.","marker":"[9]"},{"why":"Provides the BGE-M3 embeddings that define the vector space and cosine-similarity geometry used in clustering and retrieval.","marker":"[10]"},{"why":"GPT-4o-mini generates the expert-rated answers from the retrieved context, completing the evaluation pipeline.","marker":"[11]"}],"fun_headline_variants":["Dendrogram walk beats top-k in legal RAG","No k needed: tree retrieval tops legal RAG","Hierarchical tree retrieval outperforms top-k for law RAG","Tree-based RAG retrieval: expert scores up, k gone","Agglomerative tree retrieval eliminates k, boosts legal answers"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that cosine similarity in the embedding space of 200-character legal chunks forms clusters whose mean vectors reliably indicate legal relevance, so that the subtree whose centroid is nearest to the query contains the relevant provisions; if embeddings do not track legal topicality, the tree will select irrelevant subtrees and the measured gains will not generalize.","fun_headline_variants_meta":{"raw":{"variants":["Dendrogram walk beats top-k in legal RAG","No k needed: tree retrieval tops legal RAG","Hierarchical tree retrieval outperforms top-k for law RAG","Tree-based RAG retrieval: expert scores up, k gone","Agglomerative tree retrieval eliminates k, boosts legal answers"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000179,"raw_usage":{"total_tokens":1281,"prompt_tokens":908,"completion_tokens":373,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":524,"completion_tokens_details":{"reasoning_tokens":290}},"tokens_in":524,"tokens_out":373,"duration_ms":4067,"temperature":1.0,"reasoning_tokens":290,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:58:00.207995+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate the method on a second statutory corpus, such as U.S. federal regulations, with expert-graded questions and a per-query tuned top-k baseline: the central claim is contradicted if the tree's total expert score no longer exceeds the baseline, or if some query's chosen subtree excludes all ground-truth provisions that the top-k retrieval would have included.","supporting_citations":[{"cited_title":"u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt\\","cited_arxiv_id":null,"evidence_quote":"Defines retrieval-augmented generation and the top-k retrieval step whose k-selection problem this paper removes."},{"cited_title":"Gpt-4o-mini models, 2025","cited_arxiv_id":null,"evidence_quote":"GPT-4o-mini generates the expert-rated answers from the retrieved context, completing the evaluation pipeline."}],"review_version":2}