REVIEW 4 major objections 5 minor 37 references
Epinet for Content Cold Start
T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read Epinet exploration boosts cold-start impressions by 17 percent
desk verdict First production deployment of epinet-based Thompson sampling for cold-start retrieval, with credible A/B gains, but the treatment is confounded with an architectural change. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The epinet is a small trainable network $\sigma_{\eta}(x,z)$ with a fixed prior network, added to a base MLP $g_{\theta}(x)$ to form the overarch $f_{\theta,\eta}(x,z) = g_{\theta}(x) + \sigma_{\eta}(x,z)$; an epistemic index $z \sim \mathcal{N}(0, I_{d_z})$ is sampled per inference. The overarch scores each candidate item by this sum, so ranking by it approximates Thompson sampling without conjugacy. The epinet is trained only on the watch-score label with binary cross-entropy, while the user and item embeddings (dimension 128) are trained on four labels; a stop-gradient keeps the epinet from reshaping the embeddings. The prior network, which is fixed, ensures sufficient initial diversity across epistemic indices.
What would settle it
Run a treatment arm with the same overarch but a fixed (non-sampled) epistemic index, or a base MLP without the epinet, under the watch-score objective. If engagement gains vanish, the epistemic-index sampling is the operative mechanism; if they persist, the paper's attribution to exploration is not supported.
Extended reading notes
Core claim
On its own terms, the paper establishes that an epinet-augmented overarch—a base MLP plus a small epinet that takes a sampled epistemic index—outperforms a point-estimate baseline in the cold-start retrieval stage of Facebook Reels. The treatment improved like per impression and video completion per impression across impression-count buckets, with the largest gains for the least-seen videos, and reallocated traffic toward low-impression content while modestly pulling from mature videos. Aggregated across cold-start content, impressions rose 17%. The authors present this as the first demonstration that epinets can approximate Thompson sampling effectively at production scale, resolving the open question of whether the method works online rather than only offline.
Load-bearing premise
The treatment arm adds a base MLP and an epinet and trains the overarch only on watch score, while the control is a point estimate trained on all four labels; the paper assumes the observed gains come from the sampled epistemic index rather than from these simultaneous changes.
Editorial extensions
If this is right
- If epinets work at cold-start retrieval, the same machinery can be moved to later ranking stages, where exploration is currently suppressed by the funnel's point-estimate scores.
- Because only the watch-score label was used to train the epinet, the method leaves room to incorporate like, share, and video-view-second labels into the uncertainty model.
- The 17% impression boost shows that exploration can increase traffic to under-explored content without sacrificing engagement efficiency, contradicting the assumption that exploration necessarily hurts user experience.
- The approach achieves these gains at a fraction of the compute of deep ensembles, since the epinet is small and epistemic indices can be batched.
Reading between the lines
- The paper does not report an ablation that isolates the epistemic-index sampling from the architectural change, so the gains could partly come from the added base MLP or the switch to a watch-score-only objective; a fixed-index epinet arm would separate these.
- Because the overarch is trained only on watch score, the improvement might reflect better objective alignment rather than exploration per se; training the epinet on other labels in a follow-up would test this.
- A five-day experiment is short relative to video content lifecycles, and the authors note that data from other generators dilutes their method's collected data; longer runs or a more concentrated data pool could show whether the effect grows or decays.
- The method perturbs scores by a sampled additive term; at ranking stages where the top items are tightly clustered, even small index-dependent perturbations could change user experience in ways the cold-start bucket averages may hide.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper reports the first online deployment of epinet-based approximate Thompson sampling for content cold start in the retrieval stage of Facebook Reels. The method augments user/item embedding towers with an overarch model consisting of a base MLP plus an epinet, trains the overarch on a single watch-score label, and scores candidate items with one sampled epistemic index. A five-day A/B test with roughly 12 million users per arm is described, reporting increases in like per impression, video completion per impression, watch score per impression, and a 17% aggregate increase in impressions for cold-start content.
Significance. If the attribution holds, the paper demonstrates a valuable practical result: cheap epistemic-uncertainty sampling can be integrated into a massive production retrieval funnel and improve cold-start engagement. The deployment scale, the clear description of the production pipeline, and the explicit discussion of data-leakage risks are strengths. However, the central claim that the observed gains come from epinets as opposed to simultaneous architectural and objective changes is not established by the reported experiment, so the significance for the exploration literature is currently conditional on additional isolation of the exploration mechanism.
major comments (4)
- [Section 6.1 and Section 5.3] The treatment and control differ on more than the exploration mechanism: the treatment adds an entire overarch component (base MLP plus epinet), changes the training label set for that component to a single watch-score signal, and samples an epistemic index, while the control maintains a point estimate without an overarch. The conclusion that epinets improve cold-start recommendations therefore requires an ablation that controls for the architectural and objective changes, such as an overarch with a fixed (non-sampled) epistemic index, or a base-MLP-only overarch trained on the same label. Without such an ablation, the reported gains cannot be attributed to the epistemic index.
- [Section 6.1, paragraph on data leakage] The paper acknowledges cross-arm data leakage and dilution concerns but dismisses them with the statement that 'upon inspection the embeddings for each user were sufficiently different.' This is not a quantitative analysis; no distance measure, threshold, or comparison is provided. Since the pooled training data is shared across generators and treatment arms, the authors need to supply actual evidence, such as embedding distance distributions or a direct leakage test, to support the claim that generalization across treatment groups is negligible.
- [Section 5.3 and Section 5.5] The epinet architecture and Thompson-sampling approximation are designed around linear/Gaussian outputs, but here the epinet is trained with binary cross-entropy on a single watch-score label and only one epistemic index is sampled per decision. The paper itself acknowledges that the extension to sigmoidal outputs is ad-hoc and that more appropriate forms are future work. Given that the whole empirical claim rests on this approximation being a faithful enough posterior sample, the authors should provide at least an offline calibration check or a fixed-index comparison showing that the sampled index actually induces exploration-relevant variability rather than simply adding noise.
- [Section 6.2 and Figure 6] The claim in the abstract of improvements in 'user traffic and engagement efficiency' is supported partly by a 17% boost in impressions aggregated across all impression counts, but no confidence interval or significance test is given for this aggregate number. Moreover, impressions are a quantity the algorithm directly controls by reallocating recommendations, so the aggregate impression increase does not by itself establish a user-level traffic or engagement benefit. The paper should report a user-level or system-level engagement metric with uncertainty, or temper the abstract's claim accordingly.
minor comments (5)
- [Section 3] There is a typo in the observation-space paragraph: 'secommendation systems' should be 'recommendation systems.'
- [Section 6.1] The sentence 'We hypothesize that theses are the reasons...' contains a typo: 'theses' should be 'these.'
- [References] References [22] and [23] appear to be the same NeurIPS 2023 paper by Osband et al. on epistemic neural networks; one should be removed or the citations should be merged.
- [Section 5.4] The loss expression uses BCE without explicitly defining the binary cross-entropy function for vector inputs; please clarify that BCE is applied elementwise and averaged.
- [Figures 3-6] The figures lack axis labels and numeric tick values, which makes it difficult to assess the magnitude of the reported effects beyond the text; please add labeled axes or a table of exact percentage changes and confidence intervals.
Circularity Check
No significant circularity: the paper's central claim is an online A/B deployment result, and no derivation or fitted prediction reduces to its inputs by construction.
full rationale
This is an empirical deployment paper rather than a derivation chain. The load-bearing evidence is a live A/B test on Facebook Reels cold-start retrieval, reported in Section 6 as percentage changes in impressions, like rate, video completion rate, and watch score relative to a point-estimate control. There is no fitted parameter that is later renamed as a prediction, and no equation in the paper is shown to be equivalent to an input by definition. The method builds on epinets from Osband et al. [22] and epistemic neural networks from [23], but those are external, machine-checked or independently published results, not results derived inside this paper; the paper explicitly describes the architecture as a modification of [22] and [37], not as a first-principles derivation. Citations [36] and [37] include co-author Zheqing Zhu and are self-citations in a broad sense, but they are used only as background motivation (e.g., that deep ensembles perform well offline and that epinet has been tried offline); they do not supply the online experimental conclusion. The claim that this is the first online deployment is a novelty statement, not a circular inference. The absence of an ablation separating the epinet mechanism from the added base MLP or the single-label overarch training is a legitimate experimental-design concern, but it is a threat to internal validity and attribution, not circularity. Under the review rules, such concerns belong to correctness risk, not to the circularity score. Therefore the appropriate finding is no significant circularity, score 0.
Assumptions & free parameters
free parameters (4)
- Epistemic index dimension d_z =
5
- Overarch MLP hidden dimensions =
[384, 256]
- Cold-start impression threshold =
10,000
- Epistemic indices sampled per decision =
1
assumptions (5)
- domain assumption The recommendation objective can be represented by a known scalar weighting of observed labels.
- domain assumption Generalization across users and items with similar features is captured by the trained user and item embeddings.
- ad hoc to paper Cross-arm data leakage and dilution are negligible because treatment and control user embeddings are sufficiently different.
- ad hoc to paper The epinet architecture, designed for linear/Gaussian outputs, remains well-calibrated when trained with binary cross-entropy on a single label.
- ad hoc to paper A single sampled epistemic index per decision is a sufficient approximation of Thompson sampling.
Cite this review
Pith. "Pith review of Epinet for Content Cold Start." pith.science (2026). https://pith.science/paper/EB7CPWB7
@misc{pith2026241204484,
author = {Pith},
title = {Pith review of: Epinet for Content Cold Start},
year = {2026},
howpublished = {\url{https://pith.science/paper/EB7CPWB7}},
note = {Machine review of arXiv:2412.04484}
}
read the original abstract
The exploding popularity of online content and its user base poses an evermore challenging matching problem for modern recommendation systems. Unlike other frontiers of machine learning such as natural language, recommendation systems are responsible for collecting their own data. Simply exploiting current knowledge can lead to pernicious feedback loops but naive exploration can detract from user experience and lead to reduced engagement. This exploration-exploitation trade-off is exemplified in the classic multi-armed bandit problem for which algorithms such as upper confidence bounds (UCB) and Thompson sampling (TS) demonstrate effective performance. However, there have been many challenges to scaling these approaches to settings which do not exhibit a conjugate prior structure. Recent scalable approaches to uncertainty quantification via epinets have enabled efficient approximations of Thompson sampling even when the learning model is a complex neural network. In this paper, we demonstrate the first application of epinets to an online recommendation system. Our experiments demonstrate improvements in both user traffic and engagement efficiency on the Facebook Reels online video platform.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Marc Abeille and Alessandro Lazaric. 2017. Linear thompson sampling revisited. In Artificial Intelligence and Statistics. PMLR, 176–184
2017
-
[2]
Rajeev Agrawal. 1995. Sample mean based index policies by o (log n) regret for the multi-armed bandit problem. Advances in applied probability 27, 4 (1995), 1054–1078
work page 1995
-
[3]
P Auer. 2002. Finite-time Analysis of the Multiarmed Bandit Problem
2002
-
[4]
Stéphane Caron and Smriti Bhagat. 2013. Mixing bandits: A recipe for improved cold-start recommendations in a social network. In Proceedings of the 7th Work- shop on Social Network Mining and Analysis . 1–9
work page 2013
-
[5]
Armen Der Kiureghian and Ove Ditlevsen. 2009. Aleatory or epistemic? Does it matter? Structural safety 31, 2 (2009), 105–112
work page 2009
-
[6]
Vikranth Dwaracherla, Xiuyuan Lu, Morteza Ibrahimi, Ian Osband, Zheng Wen, and Benjamin Van Roy. 2020. Hypermodels for exploration. arXiv preprint arXiv:2006.07464 (2020)
arXiv 2020
-
[7]
Vikranth Dwaracherla, Zheng Wen, Ian Osband, Xiuyuan Lu, Seyed Mohammad Asghari, and Benjamin Van Roy. 2022. Ensembles for Uncertainty Estimation: Benefits of Prior Functions and Bootstrapping. arXiv:2206.03633 [cs.LG] https: //arxiv.org/abs/2206.03633
arXiv 2022
-
[8]
Crícia Z Felício, Klérisson VR Paixão, Celia AZ Barcelos, and Philippe Preux. 2017. A multi-armed bandit model selection for cold-start user recommendation. In Proceedings of the 25th conference on user modeling, adaptation and personalization. 32–40
work page 2017
Show all 37 references
-
[9]
John C Gittins. 1979. Bandit processes and dynamic allocation indices. Journal of the Royal Statistical Society Series B: Statistical Methodology 41, 2 (1979), 148–164
1979
-
[10]
Nicolas Gutowski, Tassadit Amghar, Olivier Camp, and Fabien Chhel. 2019. Global versus individual accuracy in contextual multi-armed bandit. In Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing (Limassol, Cyprus) (SAC ’19). Association for Computing Machinery, ...
2019
-
[11]
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. 2022. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556 (2022)
2022 arXiv
-
[12]
Jiri Hron, Alexander G de G Matthews, and Zoubin Ghahramani. 2017. Variational Gaussian dropout is not Bayesian. arXiv preprint arXiv:1711.02989 (2017)
2017 arXiv
-
[13]
Jun Hu, Wenwen Xia, Xiaolu Zhang, Chilin Fu, Weichang Wu, Zhaoxin Huan, Ang Li, Zuoli Tang, and Jun Zhou. 2024. Enhancing sequential recommendation via llm-based semantic embedding learning. In Companion Proceedings of the ACM on Web Conference 2024. 103–111
2024
-
[14]
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361 (2020)
2020 arXiv
-
[15]
Alex Kendall and Yarin Gal. 2017. What uncertainties do we need in bayesian deep learning for computer vision? Advances in neural information processing systems 30 (2017)
2017
-
[16]
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. 2017. Sim- ple and Scalable Predictive Uncertainty Estimation using Deep Ensembles. In Advances in Neural Information Processing Systems , I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanat...
2017
-
[17]
Lihong Li, Wei Chu, John Langford, and Robert E Schapire. 2010. A contextual- bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on World wide web . 661–670
2010
-
[18]
Xiuyuan Lu and Benjamin Van Roy. 2017. Ensemble sampling.Advances in neural information processing systems 30 (2017)
2017
-
[19]
Hai Thanh Nguyen and Anders Kofod-Petersen. 2014. Using multi-armed bandit to solve cold-start problems in recommender systems at telco. In Mining Intel- ligence and Knowledge Exploration: Second International Conference, MIKE 2014, Cork, Ireland, December 10-12, 2014. Proceed...
2014
-
[20]
Ian Osband. 2016. Risk versus uncertainty in deep learning: Bayes, bootstrap and the dangers of dropout. In NIPS workshop on bayesian deep learning , Vol. 192. MIT Press
2016
-
[21]
Ian Osband and Benjamin Van Roy. 2015. Bootstrapped Thompson Sampling and Deep Exploration. arXiv:1507.00300 [stat.ML] https://arxiv.org/abs/1507.00300
2015 arXiv
-
[22]
Ian Osband, Zheng Wen, Seyed Mohammad Asghari, Vikranth Dwaracherla, Morteza Ibrahimi, Xiuyuan Lu, and Benjamin Van Roy. 2023. Epistemic neural networks. Advances in Neural Information Processing Systems 36 (2023), 2795– 2823
2023
-
[23]
Ian Osband, Zheng Wen, Seyed Mohammad Asghari, Vikranth Dwaracherla, MORTEZA IBRAHIMI, Xiuyuan Lu, and Benjamin Van Roy. 2023. Epistemic Neural Networks. In Advances in Neural Information Processing Systems , A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine ...
2023
-
[24]
Chao Qin, Zheng Wen, Xiuyuan Lu, and Benjamin Van Roy. 2022. An analysis of ensemble sampling. Advances in Neural Information Processing Systems 35 (2022), 21602–21614
2022
-
[25]
Zhaopeng Qiu, Xian Wu, Jingyue Gao, and Wei Fan. 2021. U-BERT: Pre-training user representations for improved recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 4320–4327
2021
-
[26]
William R Thompson. 1933. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples.Biometrika 25, 3-4 (1933), 285–294
1933
-
[27]
Aleksandr Vorobev, Damien Lefortier, Gleb Gusev, and Pavel Serdyukov. 2015. Gathering Additional Feedback on Search Results by Multi-Armed Bandits with Respect to Production Ranking. InProceedings of the 24th International Conference on World Wide Web (Florence, Italy) (WWW ’1...
2015
-
[28]
Lu Wang, Chengyu Wang, Keqiang Wang, and Xiaofeng He. 2017. BiUCB: A Contextual Bandit Algorithm for Cold-Start and Diversified Recommendation. In 2017 IEEE International Conference on Big Knowledge (ICBK) . 248–253. https: //doi.org/10.1109/ICBK.2017.49
2017 doi
-
[29]
Max Welling and Yee W Teh. 2011. Bayesian learning via stochastic gradient Langevin dynamics. In Proceedings of the 28th international conference on machine learning (ICML-11). Citeseer, 681–688
2011
-
[30]
Chuhan Wu, Fangzhao Wu, Tao Qi, and Yongfeng Huang. 2021. Empowering news recommendation with pre-trained language models. In Proceedings of the 44th international ACM SIGIR conference on research and development in informa- tion retrieval. 1652–1656
2021
-
[31]
Pan Xu, Zheng Wen, Handong Zhao, and Quanquan Gu. 2020. Neural contex- tual bandits with deep representation and shallow exploration. arXiv preprint arXiv:2012.01780 (2020)
2020 arXiv
-
[32]
Yisong Yue and Carlos Guestrin. 2011. Linear submodular bandits and their application to diversified retrieval. Advances in Neural Information Processing Systems 24 (2011)
2011
-
[33]
Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhao- jie Gong, Fangda Gu, Michael He, et al. 2024. Actions speak louder than words: Trillion-parameter sequential transducers for generative recommendations.arXiv preprint arXiv:2402.17152 (2024)
2024 arXiv
-
[34]
Weitong Zhang, Dongruo Zhou, Lihong Li, and Quanquan Gu. 2020. Neural thompson sampling. arXiv preprint arXiv:2010.00827 (2020)
2020 arXiv
-
[35]
Dongruo Zhou, Lihong Li, and Quanquan Gu. 2020. Neural contextual bandits with ucb-based exploration. In International Conference on Machine Learning . PMLR, 11492–11502
2020
-
[36]
Zheqing Zhu and Benjamin Van Roy. 2023. Deep Exploration for Recommenda- tion Systems. In Proceedings of the 17th ACM Conference on Recommender Systems (Singapore, Singapore) (RecSys ’23). Association for Computing Machinery, New York, NY, USA, 963–970. https://doi.org/10.1145...
2023
-
[37]
Zheqing Zhu and Benjamin Van Roy. 2023. Scalable neural contextual bandit for recommender systems. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management . 3636–3646
2023
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.