{"id":"b4e01dbb-fee1-4c3e-aff9-ae6e7ffa5d6b","arxiv_id":"2412.04484","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"An epinet-based Thompson sampling module improved cold-start video recommendations in a live Facebook Reels A/B test, but the experiment does not isolate the uncertainty mechanism.","lead":"This paper reports on the first large-scale online test of an uncertainty-aware recommendation method called an epinet, deployed in Facebook Reels' cold-start retrieval. It claims the method served more new, engaging videos and raised engagement per impression compared with the existing point-estimate system.","discovery_kind":"new_application","skeptic_critique":null,"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports the first online deployment of epinet-based approximate Thompson sampling for content cold start in the retrieval stage of Facebook Reels. The method augments user/item embedding towers with an overarch model consisting of a base MLP plus an epinet, trains the overarch on a single watch-score label, and scores candidate items with one sampled epistemic index. A five-day A/B test with roughly 12 million users per arm is described, reporting increases in like per impression, video completion per impression, watch score per impression, and a 17% aggregate increase in impressions for cold-start content.","tokens_in":12308,"tokens_out":2273,"duration_ms":25959,"significance":"If the attribution holds, the paper demonstrates a valuable practical result: cheap epistemic-uncertainty sampling can be integrated into a massive production retrieval funnel and improve cold-start engagement. The deployment scale, the clear description of the production pipeline, and the explicit discussion of data-leakage risks are strengths. However, the central claim that the observed gains come from epinets as opposed to simultaneous architectural and objective changes is not established by the reported experiment, so the significance for the exploration literature is currently conditional on additional isolation of the exploration mechanism.","major_comments":[{"comment":"The treatment and control differ on more than the exploration mechanism: the treatment adds an entire overarch component (base MLP plus epinet), changes the training label set for that component to a single watch-score signal, and samples an epistemic index, while the control maintains a point estimate without an overarch. The conclusion that epinets improve cold-start recommendations therefore requires an ablation that controls for the architectural and objective changes, such as an overarch with a fixed (non-sampled) epistemic index, or a base-MLP-only overarch trained on the same label. Without such an ablation, the reported gains cannot be attributed to the epistemic index.","section":"Section 6.1 and Section 5.3"},{"comment":"The paper acknowledges cross-arm data leakage and dilution concerns but dismisses them with the statement that 'upon inspection the embeddings for each user were sufficiently different.' This is not a quantitative analysis; no distance measure, threshold, or comparison is provided. Since the pooled training data is shared across generators and treatment arms, the authors need to supply actual evidence, such as embedding distance distributions or a direct leakage test, to support the claim that generalization across treatment groups is negligible.","section":"Section 6.1, paragraph on data leakage"},{"comment":"The epinet architecture and Thompson-sampling approximation are designed around linear/Gaussian outputs, but here the epinet is trained with binary cross-entropy on a single watch-score label and only one epistemic index is sampled per decision. The paper itself acknowledges that the extension to sigmoidal outputs is ad-hoc and that more appropriate forms are future work. Given that the whole empirical claim rests on this approximation being a faithful enough posterior sample, the authors should provide at least an offline calibration check or a fixed-index comparison showing that the sampled index actually induces exploration-relevant variability rather than simply adding noise.","section":"Section 5.3 and Section 5.5"},{"comment":"The claim in the abstract of improvements in 'user traffic and engagement efficiency' is supported partly by a 17% boost in impressions aggregated across all impression counts, but no confidence interval or significance test is given for this aggregate number. Moreover, impressions are a quantity the algorithm directly controls by reallocating recommendations, so the aggregate impression increase does not by itself establish a user-level traffic or engagement benefit. The paper should report a user-level or system-level engagement metric with uncertainty, or temper the abstract's claim accordingly.","section":"Section 6.2 and Figure 6"}],"minor_comments":[{"comment":"There is a typo in the observation-space paragraph: 'secommendation systems' should be 'recommendation systems.'","section":"Section 3"},{"comment":"The sentence 'We hypothesize that theses are the reasons...' contains a typo: 'theses' should be 'these.'","section":"Section 6.1"},{"comment":"References [22] and [23] appear to be the same NeurIPS 2023 paper by Osband et al. on epistemic neural networks; one should be removed or the citations should be merged.","section":"References"},{"comment":"The loss expression uses BCE without explicitly defining the binary cross-entropy function for vector inputs; please clarify that BCE is applied elementwise and averaged.","section":"Section 5.4"},{"comment":"The figures lack axis labels and numeric tick values, which makes it difficult to assess the magnitude of the reported effects beyond the text; please add labeled axes or a table of exact percentage changes and confidence intervals.","section":"Figures 3-6"}],"recommendation":"major_revision","confidential_remarks":"The paper is a strong industrial deployment study, but the experimental design does not isolate the exploration mechanism. The authors should be encouraged to add even a small ablation (e.g., fixed epistemic index) or to substantially soften the attribution claim. Given the venue's tolerance for production-system papers, this seems fixable within revision, so I recommend major revision rather than rejection. I also note that the manuscript appears to be a camera-ready submission with placeholder ACM DOI and copyright lines; this should be corrected before final publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know: this is a genuine production A/B report, not a simulation. The authors deployed epinet-based approximate Thompson sampling in the retrieval stage of Facebook Reels, with 12 million users per arm over five days, and they report real engagement gains—like rate, completion rate, and a 17% boost in impressions for cold-start content. That alone makes it worth reading.\n\nWhat's new: this is the first online deployment of epinets in a recommender system, going beyond Osband et al.'s algorithmic work and Zhu & Van Roy's offline experiments. The integration details matter: dual-tower embeddings with a stop-gradient overarch, a single sampled epistemic index for training and inference, and a two-layer MLP epinet with a fixed prior. The authors also flag two practical issues most papers ignore—label mismatch between the embedding task and the overarch, and data dilution from training on pooled data—and give a reasonable explanation for why the experiment still worked.\n\nThe soft spot is mechanism. The treatment arm adds an entire overarch module (base MLP + epinet), trains it on a single label (watch score), and samples epistemic indices. The control is just the point-estimate embeddings, trained on four labels. So the observed gains could come from extra model capacity, the change in training signal, or the exploration. There is no ablation—no epinet with a fixed index, no base MLP without epinet—and the paper doesn't report numbers, only figures, for the per-bucket effect sizes and confidence intervals. That is a real hole if you want to conclude that epinets specifically drive the improvement. The authors are honest that the sigmoid/BCE epinet extension is ad-hoc, and they acknowledge the data-leakage concern but test it only indirectly.\n\nDoes the central claim hold up? Partially. The paper shows a credible system-level improvement from a system that includes epinets. It does not isolate the epinet mechanism. Given production constraints, I wouldn't call that fatal, but the title and abstract overstate the attribution.\n\nWho it's for: anyone working on exploration in recommender systems, cold-start, or practical Thompson sampling. A serious referee can get useful signal from this paper even while pushing for ablations and tabular results. I'd send it to peer review, and I'd bring it to a reading group as a real-world data point on the gap between bandit theory and production systems.","headline":"First production deployment of epinet-based Thompson sampling for cold-start retrieval, with credible A/B gains, but the treatment is confounded with an architectural change.","tokens_in":12823,"tokens_out":2598,"would_cite":true,"duration_ms":36187,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Epinet exploration boosts cold-start impressions by 17 percent","keywords":["recommendation systems","Thompson sampling","contextual bandit","epinet","epistemic neural networks","content cold start","exploration-exploitation trade-off","online A/B test"],"falsifier":"Run a treatment arm with the same overarch but a fixed (non-sampled) epistemic index, or a base MLP without the epinet, under the watch-score objective. If engagement gains vanish, the epistemic-index sampling is the operative mechanism; if they persist, the paper's attribution to exploration is not supported.","tokens_in":12113,"feed_emoji":"🎬","tokens_out":6577,"duration_ms":57964,"temperature":0.7,"pith_summary":"Recommender systems face a core dilemma: exploiting known preferences creates feedback loops, while naive exploration wastes user attention. This paper reports the first online production deployment of epinets, a scalable method for approximating Bayesian posterior sampling, to manage that trade-off in the cold-start retrieval stage of Facebook Reels. New videos are shown to fewer than 10,000 users and only about 1% of recommendations get a like, so exploration must be targeted. In a five-day A/B test with roughly 12 million users per arm, an epinet-based Thompson sampling policy improved like rate, video completion rate, and watch score per impression, and produced a 17% overall boost in impressions. The central claim is that cheap epistemic-uncertainty sampling can be integrated into a production retrieval funnel and measurably improve both traffic and engagement efficiency.","feed_headline":"Epinet exploration boosts cold-start impressions by 17%","feed_subtitle":"First online test of epistemic-net Thompson sampling lifts like, completion, and watch rates on Facebook Reels.","key_machinery":"The epinet is a small trainable network $\\sigma_{\\eta}(x,z)$ with a fixed prior network, added to a base MLP $g_{\\theta}(x)$ to form the overarch $f_{\\theta,\\eta}(x,z) = g_{\\theta}(x) + \\sigma_{\\eta}(x,z)$; an epistemic index $z \\sim \\mathcal{N}(0, I_{d_z})$ is sampled per inference. The overarch scores each candidate item by this sum, so ranking by it approximates Thompson sampling without conjugacy. The epinet is trained only on the watch-score label with binary cross-entropy, while the user and item embeddings (dimension 128) are trained on four labels; a stop-gradient keeps the epinet from reshaping the embeddings. The prior network, which is fixed, ensures sufficient initial diversity across epistemic indices.","core_discovery":"On its own terms, the paper establishes that an epinet-augmented overarch—a base MLP plus a small epinet that takes a sampled epistemic index—outperforms a point-estimate baseline in the cold-start retrieval stage of Facebook Reels. The treatment improved like per impression and video completion per impression across impression-count buckets, with the largest gains for the least-seen videos, and reallocated traffic toward low-impression content while modestly pulling from mature videos. Aggregated across cold-start content, impressions rose 17%. The authors present this as the first demonstration that epinets can approximate Thompson sampling effectively at production scale, resolving the open question of whether the method works online rather than only offline.","pith_inferences":["The paper does not report an ablation that isolates the epistemic-index sampling from the architectural change, so the gains could partly come from the added base MLP or the switch to a watch-score-only objective; a fixed-index epinet arm would separate these.","Because the overarch is trained only on watch score, the improvement might reflect better objective alignment rather than exploration per se; training the epinet on other labels in a follow-up would test this.","A five-day experiment is short relative to video content lifecycles, and the authors note that data from other generators dilutes their method's collected data; longer runs or a more concentrated data pool could show whether the effect grows or decays.","The method perturbs scores by a sampled additive term; at ranking stages where the top items are tightly clustered, even small index-dependent perturbations could change user experience in ways the cold-start bucket averages may hide."],"forward_implications":["If epinets work at cold-start retrieval, the same machinery can be moved to later ranking stages, where exploration is currently suppressed by the funnel's point-estimate scores.","Because only the watch-score label was used to train the epinet, the method leaves room to incorporate like, share, and video-view-second labels into the uncertainty model.","The 17% impression boost shows that exploration can increase traffic to under-explored content without sacrificing engagement efficiency, contradicting the assumption that exploration necessarily hurts user experience.","The approach achieves these gains at a fraction of the compute of deep ensembles, since the epinet is small and epistemic indices can be batched."],"supporting_citations":[{"why":"Supplies the epinet architecture and training procedure that the paper adapts to cold-start retrieval.","marker":"[22]"},{"why":"Defines the epistemic neural network framework that justifies treating the epinet output as an approximate posterior sample.","marker":"[23]"},{"why":"Shows that prior functions improve ensemble uncertainty in low-data regimes, motivating the fixed prior network inside the epinet.","marker":"[7]"},{"why":"Establishes Thompson sampling, the exploration algorithm the epinet approximates.","marker":"[26]"},{"why":"Provides the offline recommendation-system application of epinets that this paper extends to an online production setting.","marker":"[37]"},{"why":"Demonstrates offline that deep exploration via ensembles helps recommendation systems, setting up the need for a cheaper online alternative.","marker":"[36]"}],"fun_headline_variants":["Epinets go live: cold-start impressions +17% on Reels","First online epinets: 17% more cold-start impressions","Epinets at scale: Reels cold-start impressions up 17%","Production epinets boost cold-start impressions 17%","Epinets debut: 17% lift in Reels cold-start impressions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The treatment arm adds a base MLP and an epinet and trains the overarch only on watch score, while the control is a point estimate trained on all four labels; the paper assumes the observed gains come from the sampled epistemic index rather than from these simultaneous changes.","fun_headline_variants_meta":{"raw":{"variants":["Epinets go live: cold-start impressions +17% on Reels","First online epinets: 17% more cold-start impressions","Epinets at scale: Reels cold-start impressions up 17%","Production epinets boost cold-start impressions 17%","Epinets debut: 17% lift in Reels cold-start impressions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000295,"raw_usage":{"total_tokens":1664,"prompt_tokens":846,"completion_tokens":818,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":462,"completion_tokens_details":{"reasoning_tokens":723}},"tokens_in":462,"tokens_out":818,"duration_ms":7893,"temperature":1.0,"reasoning_tokens":723,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:00:39.537354+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a treatment arm with the same overarch but a fixed (non-sampled) epistemic index, or a base MLP without the epinet, under the watch-score objective. If engagement gains vanish, the epistemic-index sampling is the operative mechanism; if they persist, the paper's attribution to exploration is not supported.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes Thompson sampling, the exploration algorithm the epinet approximates."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the epinet architecture and training procedure that the paper adapts to cold-start retrieval."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the epistemic neural network framework that justifies treating the epinet output as an approximate posterior sample."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the offline recommendation-system application of epinets that this paper extends to an online production setting."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Demonstrates offline that deep exploration via ensembles helps recommendation systems, setting up the need for a cheaper online alternative."}],"review_version":1}