Pith. sign in

REVIEW 3 major objections 5 minor 39 references

Two-stage clustering makes LLM-output sharing safe by enforcing, for every sample, a minimum embedding similarity and exact attribute match to its representative — and does so in near-linear time at scale.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 11:57 UTC pith:UCO4WXXC

load-bearing objection The guardrail theorem is airtight and the set-cover framing is a genuinely useful way to think about representative-based LLM inference at scale; the bigger caveats are the unenforced balance assumption behind the 'linear in n' claim and the under-documented empirical and deployment evidence. the 3 major comments →

arxiv 2607.19704 v1 pith:UCO4WXXC submitted 2026-07-22 cs.LG stat.ML

Efficient Clustering with Provable Guardrails for LLM Inference at Scale

Longshaokan Wang , Wai Tsang Keung , Punit Ghodasara , Roman Wang , Ali Dashti , Francesc Moreno-Noguer This is my paper
classification cs.LG stat.ML
keywords clusteringLLM inferenceguardrailsset covermini-batch k-meansembedding similarityattribute matchingscalable clustering
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that LLM inference cost can be cut dramatically by clustering inputs and computing outputs once per cluster, without giving up per-sample quality control. Its central claim is that a two-stage algorithm — mini-batch K-Means for initial partitioning, then greedy set-cover representative selection inside each partition — enforces two guardrails exactly: every point is assigned to a representative whose embedding similarity is at least a user-set threshold α and whose categorical attributes match exactly. Because the quadratic pairwise work is confined to small initial clusters, the method runs in O(nd + n²d/K) time and O(nd + n²/K²) memory, linear in n when K grows with n. If correct, this gives practitioners a way to serve millions of users with a fraction of the LLM calls while being able to certify, per sample, that the representative is measurably close.

Core claim

The discovery is that the problem of safe input clustering for LLM inference can be reframed as a set-cover problem and solved exactly with a greedy heuristic. After embedding each input, the algorithm defines a match relation: two points match if their cosine similarity is at least α and their categorical attributes are equal. Stage 1 partitions the data with mini-batch K-Means; Stage 2, within each partition, repeatedly picks the point whose α-ball covers the most uncovered points and makes it a representative. This is the standard greedy set-cover heuristic, so the output satisfies the guardrail by construction — every point lands with a representative inside its α-ball — and the number o

What carries the argument

The load-bearing object is the match relation ~_α with its α-balls B_α(i) = {j : similarity(i,j) ≥ α and attributes equal}. The match relation is reflexive and symmetric but not transitive, which makes the clustering problem genuinely combinatorial. Stage 2 is exactly the greedy set-cover heuristic on the family of α-balls restricted to each initial cluster: pick the ball covering the most remaining points, assign those points to its center, and repeat. The greedy choice also produces the intentionally skewed cluster-size distribution used for tail trimming. The complexity statement O(nd + n²d/K) time and O(nd + n²/K²) memory follows from confining pairwise computation to initial clusters of

Load-bearing premise

The load-bearing premise is empirical: embedding cosine similarity must faithfully track whether a good LLM output for a representative is acceptable for a member — the paper measures this once on one internal use case (correlations around 0.78–0.80), and if that link fails in a deployment domain, the exact guardrail guarantees input nearness but not output quality.

What would settle it

Sample member–representative pairs across similarity buckets in a new domain (say, medical or safety-critical advice) and score whether the representative's LLM output is acceptable for the member. If acceptability does not rise with embedding similarity — or the correlation falls far below the reported ≈0.8 — the practical guarantee collapses even though the assignment constraint still holds exactly.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Every sample's inherited LLM output now comes with a per-sample certificate: its representative is at least α-similar in embedding space and matches on all required attributes, not just close on average.
  • Datasets in the tens of millions become feasible: with K=Θ(n) the total runtime is linear in n, while agglomerative and spectral baselines need O(n²) memory and become intractable.
  • At matched cluster counts, the method is 10–1000× faster than standard clustering baselines, and those baselines violate the similarity threshold for 3–21% of samples while this method never does.
  • The right-skewed cluster-size distribution enables a second reduction: keeping only the top 4% of clusters can cover 90% of customers, adding roughly a 25× further cut.
  • Deployed at 38M customers with α=0.77 and household-attribute matching, the pipeline cut downstream LLM cost and latency by about 50× while measured relevance dropped only 0.7%.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The exactness of the guardrail concerns input embeddings, not LLM outputs; the safety and quality story therefore inherits the correlation between embedding similarity and output acceptability, which the paper validates once (Pearson ≈0.78, Spearman ≈0.80). In a domain with weaker correlation, the same algorithm would still satisfy the assignment constraint but would not justify output sharing.
  • The same two-stage set-cover scheme transfers to any expensive per-item function — code generation, image captioning, retrieval augmentation — provided an embedding and a similarity function approximate output equivalence; the α-threshold then becomes a tunable quality–cost dial.
  • Because the set-cover bound is per initial cluster and the global bound depends on partition quality, better initial partitions or hierarchical refinement could improve data reduction beyond the reported numbers; this is an optimization the paper leaves open.
  • A testable extension: replace cosine similarity with a learned agreement predictor trained on LLM output pairs, which could tighten the guardrail from similar inputs to similar outputs and reduce the number of clusters needed.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper addresses LLM-inference scaling by clustering input personas and invoking the LLM only on cluster representatives. It proposes Algorithm 1: Mini-batch K-Means initial clusters (Stage 1), then greedy set-cover representative selection within each initial cluster using a match relation that requires cosine similarity ≥ α and exact attribute equality (Stage 2). The guardrail property f_sim(E_i,E_j)≥α and A_i=A_j is proved exactly in Appendix D, with a (1+ln|C_k|) approximation bound on representative count per cluster. The paper claims O(nd+n^2d/K) time and O(nd+n^2/K^2) memory, linear in n when K=Θ(n), and reports 10–1000× speedups on three 100K datasets plus a 38M-customer production deployment with ~50× compute reduction.

Significance. The central algorithmic insight—viewing representative selection as greedy set cover over α-balls—is clean, and the guardrail guarantee is genuinely unconditional: Appendix D proves it directly from the match relation, with no fitted quantity used to define the target. The set-cover approximation bound is standard but applicable. If the complexity claim were rigorous, the method would be practically important. However, the paper currently overstates the scalability guarantee: linear-in-n depends on an unguaranteed balanced initial-cluster condition, and the empirical evidence (single runs, no code/data, one-sentence A/B) is too thin to verify the production claims. With revisions addressing the balance assumption and reporting more statistical detail, this could be a useful contribution.

major comments (3)
  1. [Appendix E and §3.2; also Abstract and Contributions] The claimed O(n^2d/K) time and O(n^2/K^2) memory are derived by replacing Σ_k n_k^2 with n^2/K, valid only when every initial cluster has n_k ≈ n/K. Mini-Batch K-Means provides no such guarantee: a single initial cluster can contain Θ(n) points, in which case Stage 2 is Θ(n^2d) time and Θ(n^2) memory even if K=Θ(n). Nothing in Algorithm 1 enforces balance or bounds max_k n_k, and the paper does not report Stage-1 cluster-size distributions for the 100K, 5M, or 38M runs. The abstract's statement that the algorithm is 'linear in n when K grows proportionally with n' is therefore not a theorem about Algorithm 1 as specified; it is an empirical assumption about Mini-Batch K-Means outputs. Moreover, the production run uses K=0.04%·n (≈15,000 for 38M), not K=Θ(n), so the linear regime is not demonstrated. Please either add a balancing/post-splitting mechanism with a provable bound on max_k n_k
  2. [Table 1 and §4.1] All runtime comparisons in Table 1 are single values with no repetitions, variance, or enough hardware/configuration detail to assess measurement noise. The claimed 10–1000× speedups against six baselines are central to the paper's practical contribution, but as reported they cannot be distinguished from noise or implementation artifacts. Table 3 likewise reports Stage 1 and Stage 2 times without repetitions. Please provide at least 3–5 repeated runs with mean±std for each method, and ideally release code/data or a detailed profiling setup. This is also relevant to the production '50-fold' reduction claims, though those are at least order-of-magnitude plausible.
  3. [§5 and §4.3] The production A/B validation is a single sentence claiming 'statistically significant positive impact on business metrics' without effect size, sample size, or metric definitions. The 0.7% relevance gap on 5,000 representative–member pairs has no confidence interval, and the §4.3 correlation study reports only Pearson/Spearman values and p<0.001, without showing the scatter distribution or inter-rater agreement for the LLM-based critic. Since the paper motivates the guardrail through safety and quality, these empirical claims need supporting details or should be softened.
minor comments (5)
  1. [Appendix G] Specify which embedding model is used for each dataset (only the internal recommendation use case mentions all-MiniLM-L6-v2), and report the embedding dimension and attribute-space size for the public datasets.
  2. [§4.3 / Figure 5] Add a description of how the 7,000 pairs were sampled across the seven similarity buckets, and report confidence intervals and the number of LLM-rating calls used for the correlation. A single scatterplot without sample density is hard to interpret.
  3. [Appendix D.4] The proof of Theorem 4 contains a redundant branch: after establishing R_k ∩ Bα(i) is always nonempty by Theorem 1, the 'leave unchanged' case is vacuous. This is not an error but makes the proof harder to follow.
  4. [Appendix F.1 / Figures 3–4] The figures show coverage curves, but the captions do not state which dataset or hyperparameters are used for each method, nor whether the plotted clusters are final clusters after Stage 2 or initial clusters. Please clarify.
  5. [Throughout] The term 'α-ball' for a similarity threshold is nonstandard (balls are usually defined by distance); since similarity is a proximity measure, a brief clarification in §3.1 would help readers. Also, a few references to appendix B in the main text point to the illustration appendix while the main text cites 'Appendix B' in Remark 3; check consistency.

Circularity Check

0 steps flagged

No significant circularity: the guardrail is enforced by construction and the complexity claims are conditional, not circular.

full rationale

The paper's central guarantee, constraint (1), is not a fitted prediction but a direct invariant of Algorithm 1: line 7 defines M_ij = 1 only when f_sim(E_i,E_j) >= alpha and A_i = A_j, and line 14 assigns a point only to a representative r* whose matched set contains it. Thus the proof in Appendix D that every point is assigned to a representative satisfying similarity and attribute equality is a straightforward correctness argument, not a quantity derived from the same data it purports to predict. The set-cover approximation bound invokes the external Johnson–Chvátal greedy set-cover result, and the per-cluster argument is self-contained. The complexity analysis in Appendix E explicitly conditions the advertised O(n^2/K^2) memory and O(n^2 d/K) time on 'roughly balanced initial clusters with n_k ≈ n/K'; the absence of a balance guarantee is a genuine limitation or correctness risk in the headline 'linear in n when K = Θ(n)' claim, but it is not a circular step—it is an unverified assumption about Mini-Batch K-Means behavior. The paper also does not rely on load-bearing self-citations: its references are to external methods (Sculley, Chvátal, Johnson, etc.), and no 'uniqueness theorem' or prior-work ansatz is imported to force the algorithm's design. The Section 4.3 proxy validation is an empirical correlation study, not a self-definitional construction. Overall, the derivation chain does not reduce to its own inputs; any concerns about balance or proxy validity are correctness/robustness issues, not circularity.

Axiom & Free-Parameter Ledger

2 free parameters · 5 axioms · 0 invented entities

The central theorem is essentially a proof about an algorithm, so it imports few external entities. The main non-standard commitments are empirical: balanced clusters and embedding-similarity-as-proxy. No new entities are postulated.

free parameters (2)
  • similarity threshold α = 0.75 (Shopping Personas), 0.3 (AG News), 0.4 (Cosmopedia), 0.77 (production)
    User-specified guardrail threshold; in experiments chosen to hit ~50× data reduction, in production chosen by offline validation. All guarantees are conditional on α.
  • initial cluster count K = 0.04% of n in benchmarks (K≈40 for n=100K); production K not stated
    Controls the n²/K vs data-reduction trade-off; the claimed linear-in-n scaling holds only when K grows with n and clusters are balanced.
axioms (5)
  • standard math Greedy set-cover (Johnson–Chvátal) achieves |R_k| ≤ OPT_k·(1+ln|C_k|)
    Invoked in Appendix D.5; standard approximation result.
  • standard math f_sim is symmetric and f_sim(x,x)≥α, and attribute equality is reflexive/symmetric
    Needed for reflexivity/symmetry of ~α in Appendix D.1 so the cover exists and termination holds; true for cosine similarity.
  • domain assumption Initial clusters are approximately balanced: n_k ≈ n/K
    Appendix E derives O(n²d/K) time and O(n²/K²) memory only under this assumption; Mini-Batch K-Means usually balances but gives no worst-case guarantee.
  • domain assumption Embedding cosine similarity is a valid proxy for downstream LLM output quality/relevance
    Introduced in §4.3 / Appendix F.2; the practical 'quality guardrail' meaning depends on this correlation. Supported by a single internal study (r≈0.78), not a general proof.
  • domain assumption Attribute equality checks (e.g., household composition) capture safety-relevant constraints exactly
    Used in §5; assumes the categorical fields stored are correct and sufficient for product-safety filtering.

pith-pipeline@v1.3.0-alltime-deepseek · 16116 in / 17847 out tokens · 184766 ms · 2026-08-01T11:57:06.030471+00:00 · methodology

0 comments
read the original abstract

Scaling LLM-based applications to millions of users is bottlenecked by the inference cost and latency of modern foundation models. A natural fix is to cluster the inputs and call the LLM only on cluster representatives, letting other members inherit the output -- but this is only safe if each member is measurably close to its representative. Existing clustering methods do not offer such per-sample quality control at scale: none jointly guarantee a minimal within-cluster similarity, exact matching of categorical attributes, and scalability to tens of millions of samples. We propose a two-stage algorithm that generates initial clusters with Mini-batch K-Means, then greedily selects representatives within each initial cluster -- a step equivalent to the Johnson-Chvatal heuristic for Set Cover over alpha-balls in embedding space. The algorithm enforces the similarity and attribute guardrails exactly by construction, and runs in $O(nd + n^2 d/K)$ time and $O(nd + n^2/K^2)$ memory for $n$ samples, feature dimension $d$, and $K$ initial clusters -- linear in $n$ when $K$ grows proportionally with $n$. We provide benchmarks against common clustering methods on internal and public datasets: our method not only delivers per-sample guardrails but also runs 10-1000x faster and scales to data sizes where most standard methods become intractable. Deployed on 38 million customers for a persona-based recommender, the clustering method cut downstream cost and latency by 50-fold while preserving personalization and unblocked the production launch.

Figures

Figures reproduced from arXiv: 2607.19704 by Ali Dashti, Francesc Moreno-Noguer, Longshaokan Wang, Punit Ghodasara, Roman Wang, Wai Tsang Keung.

Figure 1
Figure 1. Figure 1: High-level overview of the personalized recommendation pipeline. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: A. Visualization of the embeddings of six sentences projected to a 2D plot. B. The six embeddings are clustered into two initial clusters by Mini-batch K-means. C. Cosine similarity matrix of initial cluster 1 (left) and after filtering with a similarity threshold of 0.7 (right). “The customer likes team sports” has the highest number of matches and is selected as the representative. D. The final clusters.… view at source ↗
Figure 3
Figure 3. Figure 3: Sample coverage by sorted clusters. Appendix F. Additional Experiment Results F.1. Cluster Size Distributions As detailed in Section 3, the iterative greedy selection of representative samples in the proposed algorithm by design leads to highly skewed cluster size distribution, with practical advantages: We can remove a large percentage of tail clusters at the cost of removing a small percentage of samples… view at source ↗
Figure 4
Figure 4. Figure 4: Sample coverage by sorted clusters on Cosmopedia. [PITH_FULL_IMAGE:figures/full_fig_p017_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Recommendation relevance rating by member-representative similarity. [PITH_FULL_IMAGE:figures/full_fig_p018_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

39 extracted references · 2 canonical work pages

  1. [1]

    Amro Abbas, Kushal Tirumala, Dániel Simig, Surya Ganguli, and Ari S. Morcos. Semdedup: Data-efficient learning at web-scale through semantic deduplication, 2023. URL https: //arxiv.org/abs/2303.09540

    Pith/arXiv arXiv 2023

  2. [2]

    Mohammad Al Hasan, Saeed Salem, and Mohammed J. Zaki. SimClus: an effective algorithm for clustering with a lower bound on similarity.Knowledge and Information Systems, 28(3): 665–685, 2011. doi: 10.1007/s10115-010-0360-6

    work page doi:10.1007/s10115-010-0360-6 2011

  3. [3]

    Static and dynamic information organization with star clusters

    Javed Aslam, Katya Pelekhov, and Daniela Rus. Static and dynamic information organization with star clusters. InProceedings of the Seventh International Conference on Information and Knowledge Management (CIKM ’98), pages 208–217. ACM, 1998. doi: 10.1145/288627. 288659

    work page doi:10.1145/288627 1998

  4. [4]

    Cosmopedia, February 2024

    Loubna Ben Allal, Anton Lozhkov, Guilherme Penedo, Thomas Wolf, and Leandro von Werra. Cosmopedia, February 2024. URL https://huggingface.co/datasets/ HuggingFaceTB/cosmopedia

    2024

  5. [5]

    Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S

    Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri Chatterji, Annie Chen, Kathleen Creel, Jared Quincy Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stef...

    Pith/arXiv arXiv 2022

  6. [6]

    A.Z. Broder. On the resemblance and containment of documents. InProceedings. Compression and Complexity of SEQUENCES 1997 (Cat. No.97TB100171), pages 21–29, 1997. doi: 10.1109/SEQUEN.1997.666900

    arXiv 1997

  7. [7]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Sco...

    1901

  8. [8]

    V . Chvátal. A greedy heuristic for the set-covering problem.Mathematics of Operations Research, 4(3):233–235, 1979. doi: 10.1287/moor.4.3.233

    doi:10.1287/moor.4.3.233 1979

  9. [9]

    Del Corso, Antonio Gulli, and Francesco Romani

    Gianna M. Del Corso, Antonio Gulli, and Francesco Romani. Ranking a stream of news. In Proceedings of the 14th International World Wide Web Conference, pages 97–106, Chiba, Japan, 2005

    2005

  10. [10]

    A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the em algorithm.Journal of the Royal Statistical Society. Series B (Methodological), 39(1): 1–38, 1977. ISSN 00359246. URLhttp://www.jstor.org/stable/2984875

    arXiv 1977

  11. [11]

    Inference economics of language models, 2025

    Ege Erdil. Inference economics of language models, 2025. URL https://arxiv.org/ abs/2506.04645

    Pith/arXiv arXiv 2025

  12. [12]

    Llm-ensemble: Optimal large language model en- semble method for e-commerce product attribute value extraction

    Chenhao Fang, Xiaohan Li, Zezhong Fan, Jianpeng Xu, Kaushiki Nag, Evren Korpeoglu, Sushant Kumar, and Kannan Achan. Llm-ensemble: Optimal large language model en- semble method for e-commerce product attribute value extraction. InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Informa- tion Retrieval, SIGIR ’24, p...

    arXiv 2024

  13. [13]

    The anatomy of a news search engine

    Antonio Gulli. The anatomy of a news search engine. InProceedings of the 14th International World Wide Web Conference, pages 880–881, Chiba, Japan, 2005

    2005

  14. [14]

    Lilium: ebay’s large language models for e-commerce, 2024

    Christian Herold, Michael Kozielski, Leonid Ekimov, Pavel Petrushkov, Pierre-Yves Vanden- bussche, and Shahram Khadivi. Lilium: ebay’s large language models for e-commerce, 2024. URLhttps://arxiv.org/abs/2406.12023

    Pith/arXiv arXiv 2024

  15. [15]

    Springer Science & Business Media, 2008

    Rob Hyndman, Anne B Koehler, J Keith Ord, and Ralph D Snyder.Forecasting with exponential smoothing: the state space approach. Springer Science & Business Media, 2008

    2008

  16. [16]

    David S. Johnson. Approximation algorithms for combinatorial problems.Journal of Computer and System Sciences, 9(3):256–278, 1974. doi: 10.1016/S0022-0000(74)80044-9. 7 EFFICIENTCLUSTERING WITHPROVABLEGUARDRAILS FORLLM INFERENCE ATSCALE

    doi:10.1016/s0022-0000(74)80044-9 1974

  17. [17]

    Large language models are zero-shot reasoners

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. InProceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, Red Hook, NY , USA, 2022. Curran Associates Inc. ISBN 9781713871088

    2022

  18. [18]

    Efficient memory management for large language model serving with pagedattention

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. InProceedings of the 29th Symposium on Operating Systems Principles, SOSP ’23, pages 611–626, New York, NY , USA, 2023. Association for Computing Machi...

    arXiv 2023

  19. [19]

    Ecomgpt: instruction-tuning large language models with chain-of-task tasks for e-commerce

    Yangning Li, Shirong Ma, Xiaobin Wang, Shen Huang, Chengyue Jiang, Hai-Tao Zheng, Pengjun Xie, Fei Huang, and Yong Jiang. Ecomgpt: instruction-tuning large language models with chain-of-task tasks for e-commerce. InProceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artifici...

    2024

  20. [20]

    Throughput-optimal scheduling algorithms for llm inference and ai agents, 2025

    Yueying Li, Jim Dai, and Tianyi Peng. Throughput-optimal scheduling algorithms for llm inference and ai agents, 2025. URLhttps://arxiv.org/abs/2504.07347

    Pith/arXiv arXiv 2025

  21. [21]

    Language model alignment for conversational shopping at amazon

    Chen Luo, Dimitri Papadimitriou, Hariharan Muralidharan, Dhineshkumar Ramasubbu, Aakash Kolekar, Wenju Xu, Cong Xu, Anirudh Srinivasan, Mukesh Jain, and Qi He. Language model alignment for conversational shopping at amazon. InProceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’25, pages 4...

    arXiv 2025

  22. [22]

    Multivariate observations

    J MacQueen. Multivariate observations. InProceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, volume 1, pages 281–297, 1967

    1967

  23. [23]

    Pedregosa, G

    F. Pedregosa, G. Varoquaux, A. Gramfort, V . Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V . Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python.Journal of Machine Learning Research, 12:2825–2830, 2011

    2011

  24. [24]

    Large language model based long-tail query rewriting in taobao search

    Wenjun Peng, Guiyang Li, Yue Jiang, Zilong Wang, Dan Ou, Xiaoyi Zeng, Derong Xu, Tong Xu, and Enhong Chen. Large language model based long-tail query rewriting in taobao search. InCompanion Proceedings of the ACM Web Conference 2024, WWW ’24, pages 20–28, New York, NY , USA, 2024. Association for Computing Machinery. ISBN 9798400701726. doi: 10.1145/35893...

    arXiv 2024

  25. [25]

    Efficiently scaling trans- former inference.ArXiv, abs/2211.05102, 2022

    Reiner Pope, Sholto Douglas, Aakanksha Chowdhery, Jacob Devlin, James Bradbury, Anselm Levskaya, Jonathan Heek, Kefan Xiao, Shivani Agrawal, and Jeff Dean. Efficiently scaling trans- former inference.ArXiv, abs/2211.05102, 2022. URL https://api.semanticscholar. org/CorpusID:253420623. 8 EFFICIENTCLUSTERING WITHPROVABLEGUARDRAILS FORLLM INFERENCE ATSCALE

    Pith/arXiv arXiv 2022

  26. [26]

    Sentence-bert: Sentence embeddings using siamese bert- networks

    Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert- networks. InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11 2019. URL https://arxiv. org/abs/1908.10084

    Pith/arXiv arXiv 2019

  27. [27]

    From words to watts: Benchmarking the energy costs of large language model inference

    Siddharth Samsi, Dan Zhao, Joseph McDonald, Baolin Li, Adam Michaleas, Michael Jones, William Bergeron, Jeremy Kepner, Devesh Tiwari, and Vijay Gadepally. From words to watts: Benchmarking the energy costs of large language model inference. In2023 IEEE High Performance Extreme Computing Conference (HPEC), pages 1–9. IEEE, 2023

    2023

  28. [28]

    D. Sculley. Web-scale k-means clustering. InProceedings of the 19th International Conference on World Wide Web, WWW ’10, pages 1177–1178, New York, NY , USA, 2010. Association for Computing Machinery. ISBN 9781605587998. doi: 10.1145/1772690.1772862. URL https://doi.org/10.1145/1772690.1772862

    arXiv 2010

  29. [29]

    Bayesian intermittent demand fore- casting for large inventories

    Matthias Seeger, David Salinas, and Valentin Flunkert. Bayesian intermittent demand fore- casting for large inventories. InProceedings of the 30th International Conference on Neural Information Processing Systems, pages 4653–4661, 2016

    2016

  30. [30]

    Normalized cuts and image segmentation.IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8):888–905, 2000

    Jianbo Shi and Jitendra Malik. Normalized cuts and image segmentation.IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8):888–905, 2000

    2000

  31. [31]

    A tutorial on spectral clustering.Statistics and Computing, 17(4):395–416, 2007

    Ulrike von Luxburg. A tutorial on spectral clustering.Statistics and Computing, 17(4):395–416, 2007

    2007

  32. [32]

    Joe H. Ward. Hierarchical grouping to optimize an objective function.Journal of the Ameri- can Statistical Association, 58:236–244, 1963. URL https://api.semanticscholar. org/CorpusID:32863022

    1963

  33. [33]

    Webshop: Towards scalable real-world web interaction with grounded language agents

    Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. Webshop: Towards scalable real-world web interaction with grounded language agents. In S. Koyejo, S. Mo- hamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors,Advances in Neural Infor- mation Processing Systems, volume 35, pages 20744–20757. Curran Associates, Inc., 2022. URL https://proceed...

    2022

  34. [34]

    Birch: an efficient data clustering method for very large databases

    Tian Zhang, Raghu Ramakrishnan, and Miron Livny. Birch: an efficient data clustering method for very large databases. InProceedings of the 1996 ACM SIGMOD international conference on Management of data, pages 103–114. ACM, 1996. doi: 10.1145/233269.233324

    arXiv 1996

  35. [35]

    all-MiniLM-L6-v2

    Kan Zhu, Yufei Gao, Yilong Zhao, Liangyu Zhao, Gefei Zuo, Yile Gu, Dedong Xie, Tian Tang, Qinyu Xu, Zihao Ye, Keisuke Kamahori, Chien-Yu Lin, Ziren Wang, Stephanie Wang, Arvind Krishnamurthy, and Baris Kasikci. Nanoflow: Towards optimal large language model serving throughput, 2025. URLhttps://arxiv.org/abs/2408.12757. Appendix A. Design Considerations an...

    Pith/arXiv arXiv 2025

  36. [37]

    2.K-Means: ‘init’: ‘k-means++’, ‘n_init’: ‘auto’, ‘max_iter’: 300, ‘algorithm’: ‘lloyd’

    Mini-batch K-Means: ‘max_iter’: 100, ‘batch_size’: 10240, ‘init_size’: 30720, ‘random_state’: 123. 2.K-Means: ‘init’: ‘k-means++’, ‘n_init’: ‘auto’, ‘max_iter’: 300, ‘algorithm’: ‘lloyd’

  37. [38]

    4.BIRCH: ‘threshold’: 0.5, ‘branching_factor’: 50

    Agglomerative Clustering: ‘metric’: ‘euclidean’, ‘linkage’: ‘ward’, ‘compute_full_tree’: ‘auto’. 4.BIRCH: ‘threshold’: 0.5, ‘branching_factor’: 50

  38. [39]

    6.Gaussian Mixture: ‘covariance_type’: ‘full’, ‘init_params’: ‘kmeans’, ‘max_iter’: 100

    Spectral Clustering: ‘affinity’: ‘nearest_neighbors’, ‘n_neighbors’: 10, ‘eigen_solver’: ‘lobpcg’. 6.Gaussian Mixture: ‘covariance_type’: ‘full’, ‘init_params’: ‘kmeans’, ‘max_iter’: 100. 20

  39. [2020]

    URL https://proceedings.neurips.cc/paper_files/paper/2020/ file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf

    2020