Pith. sign in

REVIEW 4 major objections 5 minor 37 references

Epinet for Content Cold Start

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Pith's one-line read Epinet exploration boosts cold-start impressions by 17 percent

desk verdict First production deployment of epinet-based Thompson sampling for cold-start retrieval, with credible A/B gains, but the treatment is confounded with an architectural change. read the letter →

arxiv 2412.04484 v1 pith:EB7CPWB7 submitted 2024-11-20 cs.IR cs.AI

classification cs.IRcs.AI
keywords recommendationsystemsThompsonsamplingcontextualbanditepinetepistemicneuralnetworkscontentcoldstartexploration-exploitationtrade-offonlineA/Btest
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Recommender systems face a core dilemma: exploiting known preferences creates feedback loops, while naive exploration wastes user attention. This paper reports the first online production deployment of epinets, a scalable method for approximating Bayesian posterior sampling, to manage that trade-off in the cold-start retrieval stage of Facebook Reels. New videos are shown to fewer than 10,000 users and only about 1% of recommendations get a like, so exploration must be targeted. In a five-day A/B test with roughly 12 million users per arm, an epinet-based Thompson sampling policy improved like rate, video completion rate, and watch score per impression, and produced a 17% overall boost in impressions. The central claim is that cheap epistemic-uncertainty sampling can be integrated into a production retrieval funnel and measurably improve both traffic and engagement efficiency.

What carries the argument

The epinet is a small trainable network $\sigma_{\eta}(x,z)$ with a fixed prior network, added to a base MLP $g_{\theta}(x)$ to form the overarch $f_{\theta,\eta}(x,z) = g_{\theta}(x) + \sigma_{\eta}(x,z)$; an epistemic index $z \sim \mathcal{N}(0, I_{d_z})$ is sampled per inference. The overarch scores each candidate item by this sum, so ranking by it approximates Thompson sampling without conjugacy. The epinet is trained only on the watch-score label with binary cross-entropy, while the user and item embeddings (dimension 128) are trained on four labels; a stop-gradient keeps the epinet from reshaping the embeddings. The prior network, which is fixed, ensures sufficient initial diversity across epistemic indices.

What would settle it

Run a treatment arm with the same overarch but a fixed (non-sampled) epistemic index, or a base MLP without the epinet, under the watch-score objective. If engagement gains vanish, the epistemic-index sampling is the operative mechanism; if they persist, the paper's attribution to exploration is not supported.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that an epinet-augmented overarch—a base MLP plus a small epinet that takes a sampled epistemic index—outperforms a point-estimate baseline in the cold-start retrieval stage of Facebook Reels. The treatment improved like per impression and video completion per impression across impression-count buckets, with the largest gains for the least-seen videos, and reallocated traffic toward low-impression content while modestly pulling from mature videos. Aggregated across cold-start content, impressions rose 17%. The authors present this as the first demonstration that epinets can approximate Thompson sampling effectively at production scale, resolving the open question of whether the method works online rather than only offline.

Load-bearing premise

The treatment arm adds a base MLP and an epinet and trains the overarch only on watch score, while the control is a point estimate trained on all four labels; the paper assumes the observed gains come from the sampled epistemic index rather than from these simultaneous changes.

Editorial extensions

If this is right

  • If epinets work at cold-start retrieval, the same machinery can be moved to later ranking stages, where exploration is currently suppressed by the funnel's point-estimate scores.
  • Because only the watch-score label was used to train the epinet, the method leaves room to incorporate like, share, and video-view-second labels into the uncertainty model.
  • The 17% impression boost shows that exploration can increase traffic to under-explored content without sacrificing engagement efficiency, contradicting the assumption that exploration necessarily hurts user experience.
  • The approach achieves these gains at a fraction of the compute of deep ensembles, since the epinet is small and epistemic indices can be batched.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not report an ablation that isolates the epistemic-index sampling from the architectural change, so the gains could partly come from the added base MLP or the switch to a watch-score-only objective; a fixed-index epinet arm would separate these.
  • Because the overarch is trained only on watch score, the improvement might reflect better objective alignment rather than exploration per se; training the epinet on other labels in a follow-up would test this.
  • A five-day experiment is short relative to video content lifecycles, and the authors note that data from other generators dilutes their method's collected data; longer runs or a more concentrated data pool could show whether the effect grows or decays.
  • The method perturbs scores by a sampled additive term; at ranking stages where the top items are tightly clustered, even small index-dependent perturbations could change user experience in ways the cold-start bucket averages may hide.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper reports the first online deployment of epinet-based approximate Thompson sampling for content cold start in the retrieval stage of Facebook Reels. The method augments user/item embedding towers with an overarch model consisting of a base MLP plus an epinet, trains the overarch on a single watch-score label, and scores candidate items with one sampled epistemic index. A five-day A/B test with roughly 12 million users per arm is described, reporting increases in like per impression, video completion per impression, watch score per impression, and a 17% aggregate increase in impressions for cold-start content.

Significance. If the attribution holds, the paper demonstrates a valuable practical result: cheap epistemic-uncertainty sampling can be integrated into a massive production retrieval funnel and improve cold-start engagement. The deployment scale, the clear description of the production pipeline, and the explicit discussion of data-leakage risks are strengths. However, the central claim that the observed gains come from epinets as opposed to simultaneous architectural and objective changes is not established by the reported experiment, so the significance for the exploration literature is currently conditional on additional isolation of the exploration mechanism.

major comments (4)
  1. [Section 6.1 and Section 5.3] The treatment and control differ on more than the exploration mechanism: the treatment adds an entire overarch component (base MLP plus epinet), changes the training label set for that component to a single watch-score signal, and samples an epistemic index, while the control maintains a point estimate without an overarch. The conclusion that epinets improve cold-start recommendations therefore requires an ablation that controls for the architectural and objective changes, such as an overarch with a fixed (non-sampled) epistemic index, or a base-MLP-only overarch trained on the same label. Without such an ablation, the reported gains cannot be attributed to the epistemic index.
  2. [Section 6.1, paragraph on data leakage] The paper acknowledges cross-arm data leakage and dilution concerns but dismisses them with the statement that 'upon inspection the embeddings for each user were sufficiently different.' This is not a quantitative analysis; no distance measure, threshold, or comparison is provided. Since the pooled training data is shared across generators and treatment arms, the authors need to supply actual evidence, such as embedding distance distributions or a direct leakage test, to support the claim that generalization across treatment groups is negligible.
  3. [Section 5.3 and Section 5.5] The epinet architecture and Thompson-sampling approximation are designed around linear/Gaussian outputs, but here the epinet is trained with binary cross-entropy on a single watch-score label and only one epistemic index is sampled per decision. The paper itself acknowledges that the extension to sigmoidal outputs is ad-hoc and that more appropriate forms are future work. Given that the whole empirical claim rests on this approximation being a faithful enough posterior sample, the authors should provide at least an offline calibration check or a fixed-index comparison showing that the sampled index actually induces exploration-relevant variability rather than simply adding noise.
  4. [Section 6.2 and Figure 6] The claim in the abstract of improvements in 'user traffic and engagement efficiency' is supported partly by a 17% boost in impressions aggregated across all impression counts, but no confidence interval or significance test is given for this aggregate number. Moreover, impressions are a quantity the algorithm directly controls by reallocating recommendations, so the aggregate impression increase does not by itself establish a user-level traffic or engagement benefit. The paper should report a user-level or system-level engagement metric with uncertainty, or temper the abstract's claim accordingly.
minor comments (5)
  1. [Section 3] There is a typo in the observation-space paragraph: 'secommendation systems' should be 'recommendation systems.'
  2. [Section 6.1] The sentence 'We hypothesize that theses are the reasons...' contains a typo: 'theses' should be 'these.'
  3. [References] References [22] and [23] appear to be the same NeurIPS 2023 paper by Osband et al. on epistemic neural networks; one should be removed or the citations should be merged.
  4. [Section 5.4] The loss expression uses BCE without explicitly defining the binary cross-entropy function for vector inputs; please clarify that BCE is applied elementwise and averaged.
  5. [Figures 3-6] The figures lack axis labels and numeric tick values, which makes it difficult to assess the magnitude of the reported effects beyond the text; please add labeled axes or a table of exact percentage changes and confidence intervals.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's central claim is an online A/B deployment result, and no derivation or fitted prediction reduces to its inputs by construction.

full rationale

This is an empirical deployment paper rather than a derivation chain. The load-bearing evidence is a live A/B test on Facebook Reels cold-start retrieval, reported in Section 6 as percentage changes in impressions, like rate, video completion rate, and watch score relative to a point-estimate control. There is no fitted parameter that is later renamed as a prediction, and no equation in the paper is shown to be equivalent to an input by definition. The method builds on epinets from Osband et al. [22] and epistemic neural networks from [23], but those are external, machine-checked or independently published results, not results derived inside this paper; the paper explicitly describes the architecture as a modification of [22] and [37], not as a first-principles derivation. Citations [36] and [37] include co-author Zheqing Zhu and are self-citations in a broad sense, but they are used only as background motivation (e.g., that deep ensembles perform well offline and that epinet has been tried offline); they do not supply the online experimental conclusion. The claim that this is the first online deployment is a novelty statement, not a circular inference. The absence of an ablation separating the epinet mechanism from the added base MLP or the single-label overarch training is a legitimate experimental-design concern, but it is a threat to internal validity and attribution, not circularity. Under the review rules, such concerns belong to correctness risk, not to the circularity score. Therefore the appropriate finding is no significant circularity, score 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central empirical claim rests on production-system assumptions rather than mathematical derivation. The two heaviest assumptions are that aggregated cross-arm data does not invalidate the A/B comparison and that a single-label, sigmoid-output epinet still produces usable uncertainty for exploration. Additional unablated choices include d_z=5, hidden dimensions [384,256], one epistemic index per decision, and the 10,000-impression cold-start threshold.

free parameters (4)
  • Epistemic index dimension d_z = 5
    Chosen in Section 6.1 with no sensitivity analysis; it directly controls the dimension of the sampled uncertainty perturbation.
  • Overarch MLP hidden dimensions = [384, 256]
    Chosen in Section 6.1 for both the base MLP and the epinet; no ablation shows how this choice affects the result.
  • Cold-start impression threshold = 10,000
    Defines the cold-start corpus in Section 4 and the stratification in Section 6.2; changing this threshold changes all reported aggregates.
  • Epistemic indices sampled per decision = 1
    Algorithm 2 samples a single epistemic index per timestep for both training and inference, which is an unablated approximation of the Thompson sampling step.
assumptions (5)
  • domain assumption The recommendation objective can be represented by a known scalar weighting of observed labels.
    Section 3 defines the reward function as a suitable weighting of labels, but the deployed weighting is never specified, so the claimed objective alignment is assumed.
  • domain assumption Generalization across users and items with similar features is captured by the trained user and item embeddings.
    Sections 3 and 5.2 assume the two-tower embeddings generalize to cold items, but no cold-start generalization error analysis is provided.
  • ad hoc to paper Cross-arm data leakage and dilution are negligible because treatment and control user embeddings are sufficiently different.
    Section 6.1 asserts this after inspection, with no quantitative evidence. The A/B validity depends on it because training data is aggregated across generators and treatment arms.
  • ad hoc to paper The epinet architecture, designed for linear/Gaussian outputs, remains well-calibrated when trained with binary cross-entropy on a single label.
    Section 5.3 states that deriving appropriate epinet forms for softmax/sigmoidal outputs is future work, yet the deployed epinet is trained with BCE on the watch-score label.
  • ad hoc to paper A single sampled epistemic index per decision is a sufficient approximation of Thompson sampling.
    Algorithm 2 samples one z per timestep, while epinet theory typically considers distributions over many indices; no analysis of this one-sample approximation is supplied.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Epinet for Content Cold Start." pith.science (2026). https://pith.science/paper/EB7CPWB7

@misc{pith2026241204484,
  author       = {Pith},
  title        = {Pith review of: Epinet for Content Cold Start},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EB7CPWB7}},
  note         = {Machine review of arXiv:2412.04484}
}
read the original abstract

The exploding popularity of online content and its user base poses an evermore challenging matching problem for modern recommendation systems. Unlike other frontiers of machine learning such as natural language, recommendation systems are responsible for collecting their own data. Simply exploiting current knowledge can lead to pernicious feedback loops but naive exploration can detract from user experience and lead to reduced engagement. This exploration-exploitation trade-off is exemplified in the classic multi-armed bandit problem for which algorithms such as upper confidence bounds (UCB) and Thompson sampling (TS) demonstrate effective performance. However, there have been many challenges to scaling these approaches to settings which do not exhibit a conjugate prior structure. Recent scalable approaches to uncertainty quantification via epinets have enabled efficient approximations of Thompson sampling even when the learning model is a complex neural network. In this paper, we demonstrate the first application of epinets to an online recommendation system. Our experiments demonstrate improvements in both user traffic and engagement efficiency on the Facebook Reels online video platform.

Figures

Figures reproduced from arXiv: 2412.04484 by the authors.

Figure 1
Figure 1. The above diagram depicts the training setup of our recommendation system. Note that the epinet is part of the [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. The above diagram depicts the content cold start [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. We depict percentage change in like per impres [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: We depict percentage change in video view comple [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: We depict percentage change in watch score per [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: We depict the percentage change in impressions for [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 20 canonical work pages

  1. [1]

    Marc Abeille and Alessandro Lazaric. 2017. Linear thompson sampling revisited. In Artificial Intelligence and Statistics. PMLR, 176–184

  2. [2]

    Rajeev Agrawal. 1995. Sample mean based index policies by o (log n) regret for the multi-armed bandit problem. Advances in applied probability 27, 4 (1995), 1054–1078

  3. [3]

    P Auer. 2002. Finite-time Analysis of the Multiarmed Bandit Problem

  4. [4]

    Stéphane Caron and Smriti Bhagat. 2013. Mixing bandits: A recipe for improved cold-start recommendations in a social network. In Proceedings of the 7th Work- shop on Social Network Mining and Analysis . 1–9

  5. [5]

    Armen Der Kiureghian and Ove Ditlevsen. 2009. Aleatory or epistemic? Does it matter? Structural safety 31, 2 (2009), 105–112

  6. [6]

    Vikranth Dwaracherla, Xiuyuan Lu, Morteza Ibrahimi, Ian Osband, Zheng Wen, and Benjamin Van Roy. 2020. Hypermodels for exploration. arXiv preprint arXiv:2006.07464 (2020)

  7. [7]

    Vikranth Dwaracherla, Zheng Wen, Ian Osband, Xiuyuan Lu, Seyed Mohammad Asghari, and Benjamin Van Roy. 2022. Ensembles for Uncertainty Estimation: Benefits of Prior Functions and Bootstrapping. arXiv:2206.03633 [cs.LG] https: //arxiv.org/abs/2206.03633

  8. [8]

    Crícia Z Felício, Klérisson VR Paixão, Celia AZ Barcelos, and Philippe Preux. 2017. A multi-armed bandit model selection for cold-start user recommendation. In Proceedings of the 25th conference on user modeling, adaptation and personalization. 32–40

Show all 37 references
  1. [9]

    John C Gittins. 1979. Bandit processes and dynamic allocation indices. Journal of the Royal Statistical Society Series B: Statistical Methodology 41, 2 (1979), 148–164

  2. [10]

    Nicolas Gutowski, Tassadit Amghar, Olivier Camp, and Fabien Chhel. 2019. Global versus individual accuracy in contextual multi-armed bandit. In Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing (Limassol, Cyprus) (SAC ’19). Association for Computing Machinery, ...

  3. [11]

    Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. 2022. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556 (2022)

  4. [12]

    Jiri Hron, Alexander G de G Matthews, and Zoubin Ghahramani. 2017. Variational Gaussian dropout is not Bayesian. arXiv preprint arXiv:1711.02989 (2017)

  5. [13]

    Jun Hu, Wenwen Xia, Xiaolu Zhang, Chilin Fu, Weichang Wu, Zhaoxin Huan, Ang Li, Zuoli Tang, and Jun Zhou. 2024. Enhancing sequential recommendation via llm-based semantic embedding learning. In Companion Proceedings of the ACM on Web Conference 2024. 103–111

  6. [14]

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361 (2020)

  7. [15]

    Alex Kendall and Yarin Gal. 2017. What uncertainties do we need in bayesian deep learning for computer vision? Advances in neural information processing systems 30 (2017)

  8. [16]

    Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. 2017. Sim- ple and Scalable Predictive Uncertainty Estimation using Deep Ensembles. In Advances in Neural Information Processing Systems , I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanat...

  9. [17]

    Lihong Li, Wei Chu, John Langford, and Robert E Schapire. 2010. A contextual- bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on World wide web . 661–670

  10. [18]

    Xiuyuan Lu and Benjamin Van Roy. 2017. Ensemble sampling.Advances in neural information processing systems 30 (2017)

  11. [19]

    Hai Thanh Nguyen and Anders Kofod-Petersen. 2014. Using multi-armed bandit to solve cold-start problems in recommender systems at telco. In Mining Intel- ligence and Knowledge Exploration: Second International Conference, MIKE 2014, Cork, Ireland, December 10-12, 2014. Proceed...

  12. [20]

    Ian Osband. 2016. Risk versus uncertainty in deep learning: Bayes, bootstrap and the dangers of dropout. In NIPS workshop on bayesian deep learning , Vol. 192. MIT Press

  13. [21]

    Ian Osband and Benjamin Van Roy. 2015. Bootstrapped Thompson Sampling and Deep Exploration. arXiv:1507.00300 [stat.ML] https://arxiv.org/abs/1507.00300

  14. [22]

    Ian Osband, Zheng Wen, Seyed Mohammad Asghari, Vikranth Dwaracherla, Morteza Ibrahimi, Xiuyuan Lu, and Benjamin Van Roy. 2023. Epistemic neural networks. Advances in Neural Information Processing Systems 36 (2023), 2795– 2823

  15. [23]

    Ian Osband, Zheng Wen, Seyed Mohammad Asghari, Vikranth Dwaracherla, MORTEZA IBRAHIMI, Xiuyuan Lu, and Benjamin Van Roy. 2023. Epistemic Neural Networks. In Advances in Neural Information Processing Systems , A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine ...

  16. [24]

    Chao Qin, Zheng Wen, Xiuyuan Lu, and Benjamin Van Roy. 2022. An analysis of ensemble sampling. Advances in Neural Information Processing Systems 35 (2022), 21602–21614

  17. [25]

    Zhaopeng Qiu, Xian Wu, Jingyue Gao, and Wei Fan. 2021. U-BERT: Pre-training user representations for improved recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 4320–4327

  18. [26]

    William R Thompson. 1933. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples.Biometrika 25, 3-4 (1933), 285–294

  19. [27]

    Aleksandr Vorobev, Damien Lefortier, Gleb Gusev, and Pavel Serdyukov. 2015. Gathering Additional Feedback on Search Results by Multi-Armed Bandits with Respect to Production Ranking. InProceedings of the 24th International Conference on World Wide Web (Florence, Italy) (WWW ’1...

  20. [28]

    Lu Wang, Chengyu Wang, Keqiang Wang, and Xiaofeng He. 2017. BiUCB: A Contextual Bandit Algorithm for Cold-Start and Diversified Recommendation. In 2017 IEEE International Conference on Big Knowledge (ICBK) . 248–253. https: //doi.org/10.1109/ICBK.2017.49

  21. [29]

    Max Welling and Yee W Teh. 2011. Bayesian learning via stochastic gradient Langevin dynamics. In Proceedings of the 28th international conference on machine learning (ICML-11). Citeseer, 681–688

  22. [30]

    Chuhan Wu, Fangzhao Wu, Tao Qi, and Yongfeng Huang. 2021. Empowering news recommendation with pre-trained language models. In Proceedings of the 44th international ACM SIGIR conference on research and development in informa- tion retrieval. 1652–1656

  23. [31]

    Pan Xu, Zheng Wen, Handong Zhao, and Quanquan Gu. 2020. Neural contex- tual bandits with deep representation and shallow exploration. arXiv preprint arXiv:2012.01780 (2020)

  24. [32]

    Yisong Yue and Carlos Guestrin. 2011. Linear submodular bandits and their application to diversified retrieval. Advances in Neural Information Processing Systems 24 (2011)

  25. [33]

    Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhao- jie Gong, Fangda Gu, Michael He, et al. 2024. Actions speak louder than words: Trillion-parameter sequential transducers for generative recommendations.arXiv preprint arXiv:2402.17152 (2024)

  26. [34]

    Weitong Zhang, Dongruo Zhou, Lihong Li, and Quanquan Gu. 2020. Neural thompson sampling. arXiv preprint arXiv:2010.00827 (2020)

  27. [35]

    Dongruo Zhou, Lihong Li, and Quanquan Gu. 2020. Neural contextual bandits with ucb-based exploration. In International Conference on Machine Learning . PMLR, 11492–11502

  28. [36]

    Zheqing Zhu and Benjamin Van Roy. 2023. Deep Exploration for Recommenda- tion Systems. In Proceedings of the 17th ACM Conference on Recommender Systems (Singapore, Singapore) (RecSys ’23). Association for Computing Machinery, New York, NY, USA, 963–970. https://doi.org/10.1145...

  29. [37]

    Zheqing Zhu and Benjamin Van Roy. 2023. Scalable neural contextual bandit for recommender systems. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management . 3636–3646

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.