pith. sign in

arxiv: 2607.00448 · v1 · pith:SN2NVYBInew · submitted 2026-07-01 · 💻 cs.IR · cs.AI

Real-Time Hard Negative Sampling via LLM-based Clustering for Large-Scale Two-Tower Retrieval

Ivan Ji , Liuyi Hu , Harrison (Zihao) Zhao , Lei Huang , Qunshu Zhang , Max (Xiangjun) Fan , Aameek Singh This is my paper

Pith reviewed 2026-07-02 06:48 UTC · model grok-4.3

classification 💻 cs.IR cs.AI
keywords hard negative samplingtwo-tower modelsLLM clusteringrecommendation retrievalself-supervised learninginformation retrieval
0
0 comments X

The pith

LLM-based clustering samples hard negatives from media clusters to train stronger two-tower retrieval models.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to demonstrate that conventional in-batch and out-of-batch negative sampling produces negatives too easy for two-tower models to learn from, limiting their effectiveness in large-scale retrieval. It introduces a self-supervised method that first uses a large language model to cluster media representations and then draws negatives from within the same cluster during training. The framework supports real-time operation on billions of training examples while adding little computational cost. Experiments on public datasets and a production deployment show gains over standard industry sampling. The same method also reduces popularity bias and interrupts feedback loops in live recommendation systems.

Core claim

A real-time hard negative sampling framework that uses an LLM to form clusters of media representations and then selects negatives from the same cluster supplies more informative training signals than in-batch or out-of-batch methods for large-scale two-tower models.

What carries the argument

LLM clustering of media representations to select same-cluster negatives in real time.

If this is right

  • The sampling method integrates directly into existing production pipelines while handling billions of training points with low added cost.
  • Models trained with the new negatives outperform those trained with conventional industry sampling on public datasets.
  • Online deployment confirms the offline gains and shows measurable reduction in popularity bias.
  • Drawing negatives from within LLM clusters can interrupt feedback loops that reinforce popular items.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same clustering step could be reused to improve negative selection in embedding-based retrieval outside recommendation systems.
  • Periodic refresh of the LLM clusters might allow the negative distribution to evolve with changing item content over time.
  • If clusters capture fine-grained similarity, the approach could help models distinguish between items that are semantically close but not identical.

Load-bearing premise

A large language model can form clusters of media items that reliably contain hard, informative negatives without adding heavy computation at scale.

What would settle it

A side-by-side run on a public dataset in which the proposed cluster-based sampling produces retrieval metrics no better than, or worse than, standard in-batch negative sampling.

read the original abstract

The two-tower model has been widely used for large-scale recommendation systems, particularly in the retrieval stage. Industry standards for training two-tower models typically involve in-batch and/or out-of-batch negative sampling. However, these methods often produce easy negatives that models can quickly learn, failing to sufficiently challenge the model. To address this issue, a novel self-supervised hard negative sampling technique is proposed that leverages a large language model (LLM) to generate hard negatives from the same cluster during model training. By utilizing the LLM to learn media representations, the proposed approach ensures that the generated negatives are more challenging and informative. This real-time sampling framework is designed for seamless integration into production models, capable of handling billions of training data points with minimal computational complexity. Experiments on public datasets, along with deployment to a large-scale online system, demonstrate that the proposed negative sampling technique outperforms widely used industry methods. Furthermore, analysis in industrial applications reveals that this sampling method can help break inherent feedback loops in recommendations and significantly reduce popularity bias.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper claims to introduce a self-supervised hard negative sampling technique for large-scale two-tower retrieval models. It uses an LLM to learn media representations, form clusters, and sample hard negatives from the same cluster in real time during training. The method is said to handle billions of training points with minimal computational complexity, outperform industry standards on public datasets, succeed in online deployment, and help break feedback loops while reducing popularity bias.

Significance. If the empirical claims hold, this work could significantly impact large-scale recommendation systems by providing a more effective way to generate informative negatives using LLMs, leading to better trained retrieval models and mitigation of common biases like popularity bias.

major comments (2)
  1. [Abstract and Method] The assertion that the framework handles billions of training data points with minimal computational complexity and enables real-time hard-negative sampling during model training lacks any description of the LLM clustering mechanism, whether inference is precomputed or online, the specific clustering algorithm, or how selection avoids high complexity costs. This directly undermines the scalability and 'real-time' claims central to the contribution.
  2. [Experiments] The abstract states that experiments on public datasets and online deployment demonstrate outperformance, but no specific metrics, baselines, ablation studies, or error analysis are provided, leaving the central outperformance claim without supporting evidence.
minor comments (1)
  1. [Abstract] The term 'self-supervised' is used but the supervision signal from the LLM clustering is not clearly distinguished from standard supervised approaches.

Simulated Author's Rebuttal

2 responses · 0 unresolved

Thank you for the constructive feedback. We address each major comment below and will revise the manuscript to strengthen the presentation of the method and experimental evidence.

read point-by-point responses

  1. Referee: [Abstract and Method] The assertion that the framework handles billions of training data points with minimal computational complexity and enables real-time hard-negative sampling during model training lacks any description of the LLM clustering mechanism, whether inference is precomputed or online, the specific clustering algorithm, or how selection avoids high complexity costs. This directly undermines the scalability and 'real-time' claims central to the contribution.

    Authors: We agree that the current description is insufficient to support the scalability claims. In the revised manuscript we will expand the method section with a concrete description of the LLM clustering pipeline, including the specific LLM used for media embeddings, the clustering algorithm (k-means on the embeddings), confirmation that clustering is performed offline once on the item corpus, and the use of an approximate nearest-neighbor index (e.g., FAISS) to retrieve same-cluster negatives in constant time during training. This pre-computation step is what enables real-time sampling over billions of items with negligible per-step cost. revision: yes

  2. Referee: [Experiments] The abstract states that experiments on public datasets and online deployment demonstrate outperformance, but no specific metrics, baselines, ablation studies, or error analysis are provided, leaving the central outperformance claim without supporting evidence.

    Authors: We acknowledge that the manuscript as currently written does not supply the quantitative details needed to substantiate the outperformance claim. In the revision we will add a dedicated experiments section that reports concrete metrics (e.g., Recall@K, NDCG), explicit baselines (in-batch, out-of-batch, and random negatives), ablation results on cluster count and LLM choice, and error analysis, together with the corresponding numbers from the online A/B test. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical technique validated externally

full rationale

The paper describes an empirical self-supervised sampling method using LLM clustering for hard negatives in two-tower models. Claims of outperformance rest on experiments on public datasets and online deployment rather than any derivation, equation, or prediction that reduces to fitted inputs by construction. No self-definitional steps, fitted parameters renamed as predictions, or load-bearing self-citations appear in the provided text. The method is presented as a practical engineering contribution with external falsifiability, making the derivation chain self-contained against benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests primarily on the domain assumption that LLM-derived clusters yield sufficiently hard negatives; no free parameters or invented entities are explicitly detailed in the abstract.

axioms (1)
  • domain assumption LLM can learn useful media representations to form clusters yielding hard negatives
    The method depends on this capability for generating informative negatives from same-cluster items.

pith-pipeline@v0.9.1-grok · 5732 in / 1242 out tokens · 35800 ms · 2026-07-02T06:48:47.201339+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

35 extracted references · 7 canonical work pages · 2 internal anchors

  1. [1]

    Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval , pages=

    Cross-batch negative sampling for training two-tower recommenders , author=. Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval , pages=

  2. [2]

    Proceedings of the 10th ACM conference on recommender systems , pages=

    Deep neural networks for youtube recommendations , author=. Proceedings of the 10th ACM conference on recommender systems , pages=

  3. [3]

    Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pages=

    Cascade ranking for operational e-commerce search , author=. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pages=

  4. [4]

    Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=

    Efficient cost-aware cascade ranking in multi-stage retrieval , author=. Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=

  5. [5]

    Proceedings of the 22nd ACM international conference on Information & Knowledge Management , pages=

    Learning deep structured semantic models for web search using clickthrough data , author=. Proceedings of the 22nd ACM international conference on Information & Knowledge Management , pages=

  6. [6]

    Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , pages=

    Embedding-based retrieval in facebook search , author=. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , pages=

  7. [7]

    IEEE Transactions on Neural Networks , volume=

    Adaptive importance sampling to accelerate training of a neural probabilistic language model , author=. IEEE Transactions on Neural Networks , volume=. 2008 , publisher=

    2008

  8. [8]

    International Workshop on Artificial Intelligence and Statistics , pages=

    Quick training of probabilistic neural nets by importance sampling , author=. International Workshop on Artificial Intelligence and Statistics , pages=. 2003 , organization=

    2003

  9. [9]

    Companion proceedings of the web conference 2020 , pages=

    Mixed negative sampling for learning two-tower neural networks in recommendations , author=. Companion proceedings of the web conference 2020 , pages=

    2020

  10. [10]

    Proceedings of the 27th ACM international conference on information and knowledge management , pages=

    Recurrent neural networks with top-k gains for session-based recommendations , author=. Proceedings of the 27th ACM international conference on information and knowledge management , pages=

  11. [11]

    Session-based Recommendations with Recurrent Neural Networks

    Session-based Recommendations with Recurrent Neural Networks , author=. arXiv preprint arXiv:1511.06939 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  12. [12]

    arXiv preprint arXiv:1909.10506 , year=

    Learning dense representations for entity retrieval , author=. arXiv preprint arXiv:1909.10506 , year=

    work page arXiv 1909

  13. [13]

    arXiv:2010.03240 [cs.IR] https://arxiv.org/abs/2010.03240

    Bias and debias in recommender system: a survey and future directions (2020) , author=. arXiv preprint arXiv:2010.03240 , year=

    work page arXiv 2020

  14. [14]

    The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval , pages=

    Should I follow the crowd? A probabilistic analysis of the effectiveness of popularity in recommender systems , author=. The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval , pages=

  15. [15]

    Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval , pages=

    Controlling fairness and bias in dynamic learning-to-rank , author=. Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval , pages=

  16. [16]

    Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=

    Computationally efficient optimization of plackett-luce ranking models for relevance and fairness , author=. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=

  17. [17]

    Impact of recommender systems on sales volume and diversity.(2014) , author=

    2014

  18. [18]

    Information Systems Research , volume=

    How do recommender systems affect sales diversity? A cross-category investigation via randomized field experiment , author=. Information Systems Research , volume=. 2019 , publisher=

    2019

  19. [19]

    Proceedings of the eleventh ACM conference on recommender systems , pages=

    Controlling popularity bias in learning-to-rank recommendation , author=. Proceedings of the eleventh ACM conference on recommender systems , pages=

  20. [20]

    Proceedings of the fifth ACM conference on Recommender systems , pages=

    Item popularity and recommendation accuracy , author=. Proceedings of the fifth ACM conference on Recommender systems , pages=

  21. [21]

    Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining , pages=

    Model-agnostic counterfactual reasoning for eliminating popularity bias in recommender system , author=. Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining , pages=

  22. [22]

    arXiv preprint arXiv:1901.07555 , year=

    Managing popularity bias in recommender systems with personalized re-ranking , author=. arXiv preprint arXiv:1901.07555 , year=

    work page arXiv 1901

  23. [23]

    Proceedings of the 14th ACM International Conference on Web Search and Data Mining , pages=

    Popularity-opportunity bias in collaborative filtering , author=. Proceedings of the 14th ACM International Conference on Web Search and Data Mining , pages=

  24. [24]

    arXiv preprint arXiv:2306.04039 , year=

    Revisiting Neural Retrieval on Accelerators , author=. arXiv preprint arXiv:2306.04039 , year=

    work page arXiv

  25. [25]

    Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations

    Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations , author=. arXiv preprint arXiv:2402.17152v3 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  26. [26]

    Proceedings of the 7th ACM international conference on Web search and data mining , pages=

    Improving pairwise learning for item recommendation from implicit feedback , author=. Proceedings of the 7th ACM international conference on Web search and data mining , pages=

  27. [27]

    Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval , pages=

    Optimizing top-n collaborative filtering via dynamic negative item sampling , author=. Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval , pages=

  28. [28]

    Proceedings of the ACM Web Conference 2022 , pages=

    Learning recommenders for implicit feedback with importance resampling , author=. Proceedings of the ACM Web Conference 2022 , pages=

    2022

  29. [29]

    Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval , pages=

    Irgan: A minimax game for unifying generative and discriminative information retrieval models , author=. Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval , pages=

  30. [30]

    Proceedings of the 13th ACM conference on recommender systems , pages=

    Sampling-bias-corrected neural modeling for large corpus item recommendations , author=. Proceedings of the 13th ACM conference on recommender systems , pages=

  31. [31]

    Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages=

    Trinity: Syncretizing Multi-/Long-Tail/Long-Term Interests All in One , author=. Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages=

  32. [32]

    arXiv preprint arXiv:2411.10057 , year=

    KuaiFormer: Transformer-Based Retrieval at Kuaishou , author=. arXiv preprint arXiv:2411.10057 , year=

    work page arXiv

  33. [33]

    International Conference on Learning Representations , year=

    Approximate Nearest Neighbor Negative Contrastive Estimation for Dense Text Retrieval , author=. International Conference on Learning Representations , year=

  34. [34]

    Advances in Neural Information Processing Systems , volume=

    Simplify and Robustify Negative Sampling for Implicit Collaborative Filtering , author=. Advances in Neural Information Processing Systems , volume=

  35. [35]

    Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , pages=

    On Sampled Metrics for Item Recommendation , author=. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , pages=