{"id":"674580fc-cf4e-4ed1-b487-675023412498","arxiv_id":"2506.00344","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Latent Semantic Clustering uses a generator LLM's hidden states to group semantically equivalent outputs, removing the need for external NLI or embedding models in test-time scaling.","lead":"The authors propose Latent Semantic Clustering (LSC), a method that clusters an LLM's multiple answers by using the model's own internal hidden states instead of an external language model. It aims to make test-time scaling techniques like uncertainty estimation and tree-of-thought reasoning faster and at least as accurate.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The automatic cluster count k relies on an unsupported eigenvalue-threshold heuristic for continuous adjacency matrices, with τ tuned per task on 100 validation examples; the undisclosed hidden layer index l can shift F1 by ~0.3, so reported gains may reflect hyperparameter fitting rather than LSC…","rationale":"The paper's core idea is attractive and plausibly useful: use the generator's own contextual hidden states instead of an external NLI model. The experiments span multiple LLMs and tasks, and Table 3 provides a direct clustering-quality comparison, though the labels come from a same-family 70B model and no code or data are released. The reader's CONDITIONAL verdict is appropriate, because the most load-bearing weakness is not the idea itself but the cluster-count mechanism plus the undisclosed layer choice. The method's only nonstandard algorithmic component is reading k off eigenvalues of a continuous Laplacian; this has no theorem support and is tuned per task. Because k feeds directly into spectral clustering and all downstream metrics, an incorrect k invalidates the reported results. The missing layer index compounds the issue: Figure 5 demonstrates large F1 variation across layers, so without knowing l the reader cannot tell whether LSC is robust or cherry-picked. A controlled test that fixes τ and uses oracle k would directly adjudicate whether the gains are intrinsic to hidden-state clustering. I do not see an internal contradiction that would force REJECT; the concern is testable. If the controlled test shows the results survive fixed τ and oracle k, the central claim stands. Hence the verdict remains CONDITIONAL, unchanged from the reader's assessment, with the added requirement that the authors disclose l and show τ sensitivity.","tokens_in":17243,"tokens_out":11195,"duration_ms":111695,"concrete_test":"Run a controlled sweep on Tables 1, 2, and A3: (a) use a single fixed τ across all datasets and models, chosen by a principled rule such as the eigengap or a fixed quantile of the eigenvalue distribution, instead of per-task validation tuning; (b) use the oracle cluster count k derived from the Llama3-70B pairwise labels; and (c) for each model, report AUROC and accuracy as a function of layer index l. If SE-LSC, DSE-LSC, or SExp-LSC no longer match or exceed the NLI-based baselines under (a) or (b), the central claim is a hyperparameter artifact. Plotting (c) will show whether the undisclosed l is selected from the favorable end of Figure 5.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3 determines k = |{λ_i < τ}| from the normalized Laplacian of the cosine-similarity matrix A. The cited Theorem 1 (Von Luxburg) guarantees zero-eigenvalue multiplicity equals the number of connected components only for binary (or nonnegative) adjacency matrices; for continuous A with positive weights the graph is a single connected component, so λ1 = 0 and no theorem supports treating small positive eigenvalues as semantic clusters. The paper acknowledges this but asserts the heuristic without derivation, and Appendix B shows τ is tuned per task on 100 validation examples. Since k is fed directly into spectral clustering, every downstream result (SE-LSC, DSE-LSC, SExp-LSC) is conditional on this tuned k. In addition, the layer index l in Eq. (4) is never reported; Figure 5 shows F1 varies from about 0.6 to 0.9 across layers, so the undisclosed l can dominate the comparison. The results therefore do not yet establish that hidden-state clustering itself provides the reported gains; they may reflect per-task, per-model fitting of (τ, l) on a small validation set.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Latent Semantic Clustering (LSC), a method that uses the hidden states of the generator LLM itself, rather than external NLI or sentence-embedding models, to build a semantic similarity graph over sampled outputs and cluster them via spectral clustering. The number of clusters is determined automatically by thresholding the small eigenvalues of the normalized Laplacian. LSC is evaluated in two test-time-computation settings: uncertainty quantification (SE-LSC and DSE-LSC replace the NLI-based clustering in Semantic Entropy and Degree-based Semantic Entropy) and multi-step reasoning (SExp-LSC replaces the NLI-based semantic exploration in SExp). The experiments report that LSC matches or improves AUROC/AUARC relative to baselines in most settings and reduces LLM inference counts in reasoning, e.g., by 22.58% on ARC with equal accuracy. Additional analyses compare clustering quality and latency against NLI and embedding models.","tokens_in":17439,"tokens_out":4572,"duration_ms":46525,"significance":"If the results hold, LSC is a significant practical advance: it removes the need for external models in semantic clustering for test-time scaling, which is a major computational bottleneck in methods like semantic entropy and semantic tree search. The paper provides broad empirical coverage (two LLMs, three QA datasets for uncertainty, two reasoning benchmarks, and a clustering-quality study) and the efficiency gains are concrete (e.g., 22.58% fewer LLM inferences on ARC). The central idea of using the generator's own context-conditioned hidden states is well motivated and aligns with prior evidence that intermediate LLM layers carry semantic information. However, the significance is currently conditional on resolving two load-bearing issues: the theoretical and practical validity of the eigenvalue-thresholding heuristic for choosing the number of clusters, and the complete reporting of hyperparameters (layer index l and threshold τ) that can dominate the comparison. With those addressed, the paper could be a solid contribution to efficient test-time scaling.","major_comments":[{"comment":"The claim that the distribution of small eigenvalues of the normalized Laplacian can estimate the number of semantically distinct meanings is not justified. The cited Theorem 1 (Von Luxburg, 2007) equates the multiplicity of the zero eigenvalue with the number of connected components only for binary adjacency matrices; for the continuous cosine-similarity matrix A in Eq. (6), the graph is generically a single connected component, so λ1=0 has multiplicity one and the small positive eigenvalues have no proven correspondence to clusters. Since k = |{λ_i < τ}| is fed directly into spectral clustering, every downstream result in Tables 1–3 and the reasoning experiments is conditional on this heuristic. The paper should either supply a theoretical argument (e.g., a perturbation or near-block-structure analysis) or demonstrate empirically that the eigengap or thresholded count reliably recovers the true number of semantic clusters on held-out data, with sensitivity to τ reported.","section":"Section 3, Eq. (7) and the paragraph after Theorem 1"},{"comment":"The clustering threshold τ is selected per task on a validation set of only 100 examples, but the paper does not report the selected τ values, the search grid, or the criterion used for selection (clustering F1 vs. downstream AUROC/accuracy). If τ is chosen to maximize the final metric, the reported gains of SE-LSC/DSE-LSC and SExp-LSC may reflect per-task tuning of a sensitive hyperparameter rather than an inherent advantage of hidden-state clustering. The authors should report τ for each dataset/model, show the sensitivity of the main results to τ (e.g., a sweep), and ideally use a fixed τ across tasks or a principled automatic selection rule.","section":"Appendix B and Table A1"},{"comment":"The layer index l used to extract hidden states is never specified for any experiment. Figure 5 shows that clustering F1 varies from about 0.6 to 0.9 depending on the chosen layer, so the unreported l can dominate the comparison. If l was selected on the validation set per model/dataset, that is an additional per-task hyperparameter; if it was fixed arbitrarily, the reader cannot reproduce the results. The paper must report the exact l used for each model (and dataset, if it varies) and justify the choice, or show that the main conclusions are robust across layers.","section":"Section 3, Eq. (4), and Figure 5"},{"comment":"The claim that LSC 'maintains or exceeds the performance of existing methods' is not uniformly supported by the data. On Mistral-7B-Instruct with TriviaQA, SE-LSC and DSE-LSC achieve lower AUROC than the corresponding SE and DSE baselines (0.8234 vs. 0.8338 and 0.8161 vs. 0.8245). Moreover, no variance or significance measures are reported anywhere; the observed differences (e.g., +0.03 AUROC on BioASQ for Llama3-8B) may be within noise. The authors should add error bars or statistical tests over multiple seeds, and temper the abstract's phrasing to reflect the settings where LSC is not clearly superior.","section":"Tables 1 and A3"}],"minor_comments":[{"comment":"The result labeled 'Theorem 1' is a direct citation of Von Luxburg (2007); it should be presented as a known result from the literature rather than as a contribution of this paper. Also, the statement should clarify that it applies to binary or nonnegative adjacency matrices, as the following paragraph acknowledges.","section":"Section 3"},{"comment":"There is a typo: 'demostrated' should be 'demonstrated'. Please also proofread for other spelling errors.","section":"Section 5"},{"comment":"The LSC row reports memory usage as '≈0 GB'. Please specify how this was measured (e.g., hidden-state tensors stored during generation) and clarify the comparison to the 4.6–20 GB memory of decoder-only LLM baselines.","section":"Table 3"},{"comment":"The paper does not state the number of generated sequences N or the branching factor d used in the reasoning experiments beyond the MCTS settings in Table A2. Since the computational overhead of spectral clustering is O(N^3), please state N explicitly for the reader to assess the claimed 'virtually no additional computation'.","section":"Section 4.2 and Table 2"},{"comment":"Table A1 lists 'threshold for clustering' as a hyperparameter but does not give its value or selection criterion; please provide a concrete description of the validation procedure, including which metric is optimized and the range of τ searched.","section":"Appendix B"}],"recommendation":"major_revision","confidential_remarks":"The paper presents a useful and well-motivated idea, and the empirical evidence is generally supportive, but the two major issues—the unsupported eigenvalue heuristic for k and the undisclosed layer index l—must be resolved before the results can be taken at face value. The per-task tuning of τ on a tiny validation set raises a risk that the reported improvements are partly an artifact of hyperparameter fitting. I suggest the editor send the paper back for a careful revision that reports all hyperparameters, adds sensitivity analyses, and either justifies or replaces the automatic k-selection step. If the authors can address these points, the paper would likely be a solid accept."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe short version: this is a worthwhile empirical paper that deserves a serious referee, but the advertised \"consistently\" highest AUROC is not true, and the paper owes the reader more detail on two hyperparameters that could be load-bearing. The core idea—use the generator's own hidden states, not an external NLI or embedding model, to cluster sampled outputs—is simple, plausible, and well-tested.\n\nWhat's new: prior work used hidden states for uncertainty (Han et al., Kossen et al.) but usually with trained probes or for specific tasks. LSC's combination of last-token intermediate-layer extraction, cosine adjacency, spectral clustering, and eigenvalue-threshold k estimation applied to both UQ and tree-search reasoning is a clean package. The efficiency gains are real: Table 3 shows LSC with roughly zero memory and 0.0069s latency against NLI at 1.5GB/0.13s and even LLM embeddings at 4.6GB/0.08s. The UQ results are mostly positive, and the reasoning experiments show LSC matching or improving SExp accuracy while cutting LLM inferences by up to 22.6% on ARC.\n\nSoft spots, in order of size. First, the eigenvalue-threshold heuristic. The paper cites Von Luxburg's Theorem 1, which applies to binary adjacency; for continuous weights the graph is connected, so zero-multiplicity is 1. The paper says the small eigenvalue distribution \"can still be used\" but gives no argument. That's a real gap, though it doesn't break the method—the eigengap heuristic is common. But the paper should either justify it or frame k as an empirical choice. Second, the hidden layer index l used in the main experiments is never reported. Figure 5 shows F1 ranges roughly 0.6 to 0.9 across layers, so this is not a small detail. The threshold τ is tuned per task on 100 validation examples (Appendix B); that's disclosed and modest, but combined with the undisclosed l, the reader can't tell how much of the gain is from fitting. Third, the paper claims \"consistently highest AUROC\" but Table A3 shows on Mistral-7B TriviaQA both SE-LSC and DSE-LSC underperform their NLI counterparts (0.8234 vs 0.8338, 0.8161 vs 0.8245). It's a handful of points but the sentence should be softened. No error bars and no code are also weaknesses, but not fatal.\n\nThe stress-test worry that results could reflect hyperparameter fitting rather than LSC itself is overstated—the pattern holds across two LLMs and multiple datasets—but the paper doesn't yet rule it out. The central claim does hold up for efficiency and, for the most part, effectiveness: you can drop the external model and not lose much.\n\nWho this is for: anyone doing test-time scaling or semantic uncertainty who wants to cut compute. Deserves a serious referee. My recommendation: send it to review, but expect the reviewers to ask for the layer index, error bars, and either a justification of the k heuristic or a sensitivity analysis. If the authors show robustness to τ and l, this is a solid contribution.","headline":"Worthwhile empirical paper that deserves peer review: LSC's core idea of clustering on generator hidden states is practical and mostly holds up, but the cluster-count heuristic is under-justified and the undisclosed layer index could be load-bearing.","tokens_in":17997,"tokens_out":3386,"would_cite":true,"duration_ms":32820,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proposes Latent Semantic Clustering, which groups LLM outputs by meaning using the generator's own hidden states and removes the need for external NLI and embedding models in test-time scaling.","keywords":["latent semantic clustering","test-time computation scaling","spectral clustering","semantic entropy","uncertainty quantification","hidden states","LLM reasoning","natural language inference"],"falsifier":"On a prompt set with known semantic-equivalence labels, compute the cosine-similarity Laplacian from a chosen intermediate layer, count eigenvalues below the paper's threshold, and compare that cluster count with the true number of semantic meanings; any prompt where the count is wrong shows the eigenvalue heuristic, not the labels, is what decides the clusters.","tokens_in":16992,"feed_emoji":"🧠","tokens_out":10318,"duration_ms":92519,"temperature":0.7,"pith_summary":"The paper proposes Latent Semantic Clustering (LSC), a way to group LLM-generated outputs by meaning that uses the hidden states of the generator model itself instead of a separate natural-language-inference or embedding model. The central claim is that the cosine similarity between internal representations of generated sequences already carries enough context-dependent semantic information to drive clustering, so test-time scaling methods do not need to pay for an external model. On question-answering uncertainty benchmarks, replacing the NLI step in semantic entropy and its variants with LSC improves or matches AUROC, the standard measure of how well uncertainty scores rank wrong answers below correct ones, while removing the NLI inference. In tree-search reasoning, LSC reduces the number of LLM calls by up to 22.58 percent on ARC without losing accuracy. If the claim holds, semantic clustering becomes nearly free at test time, because the representations are already being computed during generation.","feed_headline":"LLM hidden states can cluster their own outputs, skipping NLI models","feed_subtitle":"Clustering from the generator's own representations matches or beats external models and cuts test-time compute.","key_machinery":"The load-bearing object is the symmetric normalized Laplacian $L = I - D^{-1/2} A D^{-1/2}$ of the cosine-similarity adjacency matrix $A$ built from the generator's hidden states. Spectral clustering uses the eigenvectors of $L$, and the number of semantic clusters $k$ is estimated by counting eigenvalues below a threshold $\\tau$. In the binary-adjacency case, a standard spectral-clustering theorem guarantees that the multiplicity of the zero eigenvalue equals the number of connected components; the paper extends this heuristic to continuous similarities, where the small eigenvalues are interpreted as indicating semantically distinct meanings. Hidden states are extracted from an intermediate Transformer layer at the last generated token, giving representations that carry the context because they were computed while the model was reading that context.","core_discovery":"The central discovery is that semantic relations among generated sequences can be read off from the generator's own latent states. For each sequence the paper takes the hidden state at a fixed intermediate layer of the last generated token, forms an $N \\times N$ adjacency matrix $A$ from pairwise cosine similarities $a_{m,n} = h_m \\cdot h_n / (\\|h_m\\| \\|h_n\\|)$, and applies spectral clustering to the normalized Laplacian $L = I - D^{-1/2} A D^{-1/2}$. The number of clusters $k$ is chosen by counting eigenvalues of $L$ below a per-task threshold $\\tau$. The paper reports that clusters obtained this way are as good as or better than clusters from external NLI models, improving uncertainty estimates on QA benchmarks and reducing redundant exploration in tree-search reasoning.","pith_inferences":["Extension beyond the paper: if the eigenvalue-gap heuristic proves stable, the per-task threshold could in principle become a fixed model-level constant, since normalized-Laplacian eigenvalues lie in a bounded range; the paper only tunes it per task.","The paper's own Limitations section notes that LSC uses a single intermediate layer and the last token, and that it requires white-box access to the generator; averaging over tokens or layers and probing black-box models are natural next steps.","If the hidden-state adjacency carries the same information as NLI judgments on harder open-ended generation, the same matrix could also be used to aggregate rewards or rank reasoning paths in other test-time scaling schemes, not only the uncertainty and tree-search settings evaluated here.","An unstated consequence is that LSC's marginal cost approaches zero when the hidden states are already being produced for generation, so semantic clustering could become a default component of test-time scaling rather than a separate model call."],"forward_implications":["SE-LSC and DSE-LSC match or improve AUROC on BioASQ, SQuAD, and TriviaQA while removing the NLI inference step from semantic-entropy uncertainty estimation.","SExp-LSC reduces LLM inferences by up to 22.58 percent on ARC with no loss in accuracy, because context-conditioned hidden states merge redundant reasoning paths more accurately.","The hidden-state cosine-similarity adjacency can replace the NLI-based adjacency inside soft spectral methods (KLE, Deg, EigV, ECC), so the benefit extends beyond hard clustering.","Larger generators show larger gains: on BioASQ, Llama3-70B improves more over NLI baselines than Llama3.2-1B does.","Because the hidden states are already computed during generation, LSC adds only similarity and eigendecomposition work and no additional model forward pass."],"supporting_citations":[{"why":"Defines bi-directional entailment clustering and semantic entropy, the NLI-based framework that LSC replaces for uncertainty quantification.","marker":"(Kuhn et al., 2023)"},{"why":"KLE is a soft spectral-clustering baseline whose NLI-based adjacency matrix LSC substitutes with hidden-state cosine similarity.","marker":"(Nikitin et al., 2024)"},{"why":"Deg, EigV, and ECC provide the soft spectral-clustering uncertainty baselines used in the comparison.","marker":"(Lin et al., 2024)"},{"why":"Semantic Exploration is the tree-search reasoning baseline whose NLI-based clustering SExp-LSC replaces.","marker":"(Lee et al., 2025)"},{"why":"Supplies the theorem connecting zero-eigenvalue multiplicity to connected components, the theoretical basis for estimating the cluster count.","marker":"(Von Luxburg, 2007)"},{"why":"Sentence-BERT provides the embedding-based clustering baseline compared on quality, latency, and memory.","marker":"(Reimers and Gurevych, 2019)"},{"why":"Tree-of-Thoughts is a reasoning baseline that expands redundant reasoning paths, which LSC prunes.","marker":"(Yao et al., 2023)"}],"fun_headline_variants":["LLMs cluster their own outputs, no external models needed","Latent semantic clustering cuts test-time compute","Hidden-state clustering speeds up LLM reasoning","Self-clustering outputs: faster test-time scaling","LLM's own hidden states replace NLI models for clustering"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything downstream depends on a visible gap in the small eigenvalues of the hidden-state similarity graph appearing exactly where the true number of distinct meanings lies, and the threshold that locates that gap is tuned per task on just 100 examples.","fun_headline_variants_meta":{"raw":{"variants":["LLMs cluster their own outputs, no external models needed","Latent semantic clustering cuts test-time compute","Hidden-state clustering speeds up LLM reasoning","Self-clustering outputs: faster test-time scaling","LLM's own hidden states replace NLI models for clustering"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000182,"raw_usage":{"total_tokens":1273,"prompt_tokens":870,"completion_tokens":403,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":486,"completion_tokens_details":{"reasoning_tokens":329}},"tokens_in":486,"tokens_out":403,"duration_ms":4603,"temperature":1.0,"reasoning_tokens":329,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:06:51.820406+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a prompt set with known semantic-equivalence labels, compute the cosine-similarity Laplacian from a chosen intermediate layer, count eigenvalues below the paper's threshold, and compare that cluster count with the true number of semantic meanings; any prompt where the count is wrong shows the eigenvalue heuristic, not the labels, is what decides the clusters.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"KLE is a soft spectral-clustering baseline whose NLI-based adjacency matrix LSC substitutes with hidden-state cosine similarity."}],"review_version":1}