REVIEW 3 major objections 6 minor 1 cited by
Fatigue-Aware Ad Creative Selection
T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Adding a similarity-weighted fatigue score to an ad-selection bandit raised click-through rate and conversions in a one-week production experiment.
desk verdict A genuine production A/B test of a fatigue-aware ad creative selector, with a plausible but modest effect and an under-validated similarity metric. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the fatigue score $\kappa_{i,t}(a_t)=h_{i,t}\cdot s(a_t)$, where $h_{i,t}$ is the user's exposure-history vector and $s(a_t)$ is a vector of similarity scores between the candidate creative and each creative the user has recently seen. Each similarity score is a weighted average of bag-of-words text cosine similarity and image cosine similarity from a pre-trained image network, with text weighted three times higher. The score enters the logistic reward predictor through the quadratic terms $b_1\kappa + b_2\kappa^2$, so the fitted curve is allowed to be increasing, decreasing, or inverted-U rather than assuming the textbook shape. The model is retrained daily on the previous day's logs and used with Thompson sampling, and the similarity vectors are precomputed offline, leaving only a history lookup and an inner product per candidate at decision time.
What would settle it
Re-run the same one-week A/B test with the fatigue features included but with the history vector $h_{i,t}$ replaced by a scrambled or permuted version that destroys the true exposure signal while keeping the same distribution; if CTR and CVR gains persist, the improvement is not caused by the measured fatigue mechanism. A weaker and cheaper check is to print the fitted $b_1,b_2$ and the implied fatigue-response curve from the random-arm logs: the central claim requires the curve to be measurably negative in the high-fatigue region, not flat or positive.
Extended reading notes
Core claim
The central claim is that adding two fatigue features to the reward predictor—$b_1\kappa_{i,t}(a_t)+b_2\kappa_{i,t}(a_t)^2$ in the log-odds of a logistic model—produces better ad choices than either the same model without fatigue or random selection. In the one-week production A/B test, normalized click-through rate was 1.08 for the fatigue-aware algorithm versus 1.04 for the baseline (1.00 for random), and normalized post-impression conversion rate was 1.09 versus 1.04, with the CTR and post-impression CVR differences reaching statistical significance; on post-click CVR the baseline was numerically higher but the gap was not significant. The fatigue feature $\kappa_{i,t}(a_t)$ is an inner product of the user's recent exposure-history vector with a vector of precomputed text-and-image similarity scores, which is why it can personalize without adding per-impression computation. The paper reads the result as operational evidence for the two-factor model of advertising repetition: initial repetition helps, fatigue eventually dominates, and a bandit that rotates creatives before that point can raise both clicks and conversions.
Load-bearing premise
The load-bearing premise is that a single number per user and candidate ad—a weighted sum of past exposures, with text similarity counted three times as heavily as image similarity—really captures how tired a user is of an ad, and that one curved relationship between that number and the chance of clicking or converting applies across all users and campaigns.
Editorial extensions
If this is right
- Advertisers can add fatigue-awareness to an existing bandit by appending two features, with no change to the latency budget.
- Creatives similar to what a user has already seen are served less often as exposure accumulates, reducing wear-out.
- The fatigue feature can be tuned for different goals, since the observed relationship between fatigue, clicks, and conversions differs.
- The daily retraining schedule means the method handles creatives that are added and removed without retraining the similarity network online.
- The statistically significant production gains imply the fatigue score carries signal beyond the usual contextual features.
Reading between the lines
- An untested extension: the 3:1 text-to-image weighting is a free parameter, and learning it from data could change the size of the gains.
- The paper leaves these engineering choices untested: the 24-hour history window and one-impression-per-minute cap likely interact with the wear-out curve.
- The same two-feature correction could apply to any recommender with pairwise-similar items and an exposure log, which would make the production result a proof of concept for content fatigue beyond ads.
- The differing CTR and CVR fatigue curves visible in the paper's random-arm data suggest that optimizing clicks alone might over-rotate away from creatives that still convert.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a fatigue-aware ad creative selection algorithm for display advertising. The reward predictor in Eq. (2) augments a contextual logistic model with a quadratic term in a fatigue metric κ_i,t(a_t) = h_i,t · s(a_t), where h_i,t is the user's exposure history and s(a_t) is a similarity vector between the candidate creative and past creatives, computed as a weighted combination of text BoW and image MobileNetV2 cosine similarities (Section 4). The algorithm (Algorithm 2) embeds this predictor in a Thompson-sampling bandit. The paper reports a one-week production A/B test across three campaigns (Section 5) in which the fatigue-aware algorithm (FA) numerically outperforms a contextual-bandit baseline and random selection on CTR, overall CVR, and post-impression CVR, although only some differences are statistically significant. Post-hoc analyses in Section 6 examine the relationship between the fatigue measure and CTR/CVR.
Significance. If the result holds, the paper provides a simple, low-latency method for incorporating advertising fatigue into ad creative selection, with a real production deployment and realistic constraints. The derivation of the fatigue feature is non-circular because κ is constructed from exposure histories and precomputed similarities, not from the outcome. The production A/B test is a genuine strength, as are the practical details of implementation. However, the statistical support is partial, and the fatigue metric relies on an unvalidated similarity weighting, so the claim of demonstrated superiority needs further support before it can be accepted as stated.
major comments (3)
- [§5.2, Table 1] The statistical evidence for the central claim is weaker than the text suggests. Only CTR (1.08, P<0.1) and post-impression CVR (1.09, P<0.01) are marked significant for FA versus Baseline; overall CVR (1.12) is not significant, and post-click CVR numerically favors Baseline (1.05 vs 1.04). The paper should report confidence intervals, describe the testing procedure, and address multiple comparisons across the four metrics and three campaigns.
- [§4.2, Eq. (3)] The similarity score s(a,a') is a weighted average with text similarity given three times the weight of image similarity, justified solely by interpretability. Because κ is the only component that distinguishes FA from Baseline, the absence of any sensitivity analysis for this weight means the reported improvement could be an artifact of the chosen similarity scoring rather than a fatigue effect. The post-hoc validation in Section 6 uses the same κ to define fatigue and therefore cannot independently confirm the metric.
- [§5.1 and Table 2] The experiment duration is one week and the significance calculation is not described. The per-campaign CTRs in Table 2 vary widely (e.g., FA 1.21 and Baseline 0.95 for campaign A), and the paper notes campaign A has the least data; without confidence intervals or per-campaign significance tests, the stability claim for FA is not supported.
minor comments (6)
- [§1, Figure 1 caption] The caption reads 'Users jump to login screen of the game when click on these ads'; it should be 'when they click on these ads'.
- [§5.1, footnote 2] The footnote says four campaigns were used but one had only one creative; the results in Table 2 cover three campaigns. Please clarify the discrepancy.
- [§5.2] The sentence 'The results clearly show that the proposed algorithm successfully increased both clicks and conversions' overstates the significance reported in Table 1.
- [§6, Figure 10] Figure 10 uses data from the random algorithm only; the main text should state this explicitly rather than leaving it to the caption.
- [References] Several references contain garbled author names: [16] lists 'Clammer Koby' and 'Shie Monner' (should be Koby Crammer and Shie Mannor), and [20] appears malformed. Please correct the reference list.
- [§5.1] There are typos: 'SGDCliassifier' and 'skleran' should be 'SGDClassifier' and 'scikit-learn', respectively; 'metrices' in Section 6 should be 'metrics'.
Circularity Check
No circularity: the fatigue feature is constructed from exposure history and precomputed similarities, coefficients are fitted, and the main result comes from a live production A/B test.
full rationale
The central comparison is not circular. The fatigue feature kappa_i,t(a_t) = h_i,t · s(a_t) (Eq. 3) is computed from exposure histories and precomputed text/image cosine similarities; it does not use click or conversion outcomes. The reward model (Eq. 2) is the same logistic predictor as the baseline (Eq. 1) plus kappa and kappa^2, with coefficients b1 and b2 estimated by L2-regularized logistic regression; the paper does not present these fitted coefficients as predictions. The claimed CTR/CVR improvements are measured in a week-long production A/B test against the baseline and random selection, so the main claim is empirically grounded rather than derived from its inputs. Section 6's post-hoc analysis uses the same kappa definition to characterize fatigue, and its Figure 10 explicitly uses logs from the random algorithm only, so it does not enter the derivation of the reward predictor. There are no load-bearing self-citations, no imported uniqueness theorems, and no ansatz smuggled in via citation. The manually chosen 3:1 text-to-image weight in Section 4.2 is an unvalidated modeling choice and a potential robustness concern, but it is not a circular step because the weight is not fitted from nor defined in terms of the predicted outcomes.
Assumptions & free parameters
free parameters (7)
- b1 (fatigue linear coefficient) =
not reported
- b2 (fatigue quadratic coefficient) =
not reported
- text_similarity_weight =
3.0
- alpha (Thompson sampling exploration parameter) =
0.01
- negative_sampling_rate =
0.05
- L2_regularization_lambda =
0.0011
- feature_hash_length =
224 (Baseline), 226 (FA)
assumptions (5)
- domain assumption Two-factor model of advertising repetition: repeated exposure first increases then decreases consumer attitude, producing an inverted U.
- ad hoc to paper Fatigue transfers between creatives in proportion to text and image similarity, and kappa = h dot s measures it.
- domain assumption Fatigue enters the log-odds of click as a quadratic function b1*kappa + b2*kappa^2 with coefficients shared across users and campaigns.
- ad hoc to paper Deduplicating impressions within one minute and discarding histories older than 24 hours preserves the fatigue signal.
- standard math The Laplace approximation with a diagonal Hessian gives an accurate posterior covariance for Thompson sampling.
Cite this review
Pith. "Pith review of Fatigue-Aware Ad Creative Selection." pith.science (2026). https://pith.science/paper/TSS564B3
@misc{pith2026190808936,
author = {Pith},
title = {Pith review of: Fatigue-Aware Ad Creative Selection},
year = {2026},
howpublished = {\url{https://pith.science/paper/TSS564B3}},
note = {Machine review of arXiv:1908.08936}
}
read the original abstract
In online display advertising, selecting the most effective ad creative (ad image) for each impression is a crucial task for DSPs (Demand-Side Platforms) to fulfill their goals (click-through rate, number of conversions, revenue, and brand improvement). As widely recognized in the marketing literature, the effect of ad creative changes with the number of repetitive ad exposures. In this study, we propose an efficient and easy-to-implement ad creative selection algorithm that explicitly considers user's psychological status when selecting ad creatives. The proposed system was deployed in a real-world production environment and tested against the baseline algorithms. The results show superiority of the proposed algorithm.
Figures
Figures from the paper (6 more)
Forward citations
Cited by 1 Pith paper
-
Creative4U: MLLMs-based Advertising Creative Image Selector with Comparative Reasoning
A multimodal language model, trained on a new annotated image-pair dataset and a recall-precision reinforcement reward, selects better advertising creatives and explains its choices, with measured offline and online gains.
Reference graph
Works this paper leans on
-
[1]
Zoë Abrams and Erik Vee. 2007. Personalized Ad Delivery When Ads Fatigue: An Approximation Algorithm. In International Workshop on Web and Internet Economics. Springer, 535–540
work page 2007
-
[2]
Deepak Agarwal, Bee-Chung Chen, and Pradheep Elango. 2009. Spatio-Temporal Models for Estimating Click-through Rate. InProceedings of the 18th International Conference on World Wide Web - WWW ’09. ACM Press, Madrid, Spain, 21. https: //doi.org/10.1145/1526709.1526713
arXiv 2009
-
[3]
Alberto Bietti, Alekh Agarwal, and John Langford. 2018. A Contextual Bandit Bake-off. arXiv:1802.04064 [cs, stat] (Feb. 2018). http://arxiv.org/abs/1802.04064 arXiv: 1802.04064
arXiv 2018
-
[4]
Christopher M. Bishop. 2006.Pattern Recognition and Machine Learning. springer
work page 2006
-
[5]
Junyu Cao and Wei Sun. 2019. Dynamic Learning of Sequential Choice Bandit Problem under Marketing Fatigue. Proceedings of the AAAI Conference on Artifi- cial Intelligence 33 (July 2019), 3264–3271. https://doi.org/10.1609/aaai.v33i01. 33013264
-
[6]
Deepayan Chakrabarti, Ravi Kumar, Filip Radlinski, and Eli Upfal. 2009. Mortal Multi-Armed Bandits. In Advances in Neural Information Processing Systems 21 , D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou (Eds.). Curran Associates, Inc., 273–280. http://papers.nips.cc/paper/3580-mortal-multi-armed-bandits.pdf
work page 2009
-
[7]
Olivier Chapelle and Lihong Li. 2011. An Empirical Evaluation of Thompson Sampling. In Advances in neural information processing systems . Granada, Spain, 2249–2257
work page 2011
-
[8]
Patrali Chatterjee, Donna L. Hoffman, and Thomas P. Novak. 2003. Modeling the Clickstream: Implications for Web-Based Advertising Efforts. Marketing Science 22, 4 (Nov. 2003), 520–541. https://doi.org/10.1287/mksc.22.4.520.24906
Show all 31 references
-
[9]
Jiemiao Chen, Xiaojing Yang, and Robert E. Smith. 2016. The effects of creativity on advertising wear-in and wear-out.Journal of the Academy of Marketing Science 44, 3 (May 2016), 334–349. https://doi.org/10.1007/s11747-014-0414-5
2016 doi
-
[10]
François Chollet and others. 2015. Keras. (2015). https://keras.io
2015
-
[11]
Narayanan
Neha Gupta, Abhimanyu Das, Sandeep Pandey, and Vijay K. Narayanan. 2012. Factoring Past Exposure in Display Advertising Targeting. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’12) . ACM, New York, NY, USA, 1204–121...
2012
-
[12]
Junpei Komiyama and Tao Qin. 2014. Time-Decaying Bandits for Non-stationary Systems. In Web and Internet Economics, Tie-Yan Liu, Qi Qi, and Yinyu Ye (Eds.). Vol. 8877. Springer International Publishing, Cham, 460–466. https://doi.org/10. 1007/978-3-319-13129-0_40
2014
-
[13]
Tor Lattimore and Csaba SzepesvÂťari. 2019. Bandit Algorithms. https://tor- lattimore.com/downloads/book/book.pdf
2019
-
[14]
Lakshmanan, Mitul Tiwari, and Sam Shah
Pei Lee, Laks V.S. Lakshmanan, Mitul Tiwari, and Sam Shah. 2014. Modeling Impression Discounting in Large-Scale Recommender Systems. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD ’14 . ACM Press, New York, New York,...
2014
-
[15]
Huitian Lei, Ambuj Tewari, and Susan A. Murphy. 2017. An Actor-Critic Contextual Bandit Algorithm for Personalized Mobile Health Interventions. arXiv:1706.09090 [cs, stat] (June 2017). http://arxiv.org/abs/1706.09090 arXiv: 1706.09090
2017 arXiv
-
[16]
Nir Levine, Clammer Koby, and Shie Monner. 2017. Rotting Bandits. Advances in Neural Information Processing Systems30 (2017). http://papers.nips.cc/paper/6900- rotting-bandits
2017
-
[17]
Randall A Lewis. 2015. Worn-Out or Just Getting Started? The Impact of Fre- quency in Online Display Advertising. Boston, Massachusetts, USA
2015
-
[18]
Schapire
Lihong Li, Wei Chu, John Langford, and Robert E. Schapire. 2010. A Contextual- Bandit Approach to Personalized News Article Recommendation. Proceedings of the 19th international conference on World wide web - WWW ’10 (2010), 661. https://doi.org/10.1145/1772690.1772758 arXiv: ...
2010
-
[19]
Hao Ma, Xueqing Liu, and Zhihong Shen. 2016. User Fatigue in Online News Recommendation. In Proceedings of the 25th International Conference on World Wide Web (WWW ’16) . International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, 1...
2016
-
[20]
Yamashita T
Kitauchi A. Yamashita T. Hirano Y. Matsuda H.-Takaoka K. Asahara M. Mat- sumoto, Y. [n. d.]. Japanese morphological analysis system ChaSen version 2.0 manual. ([n. d.])
-
[21]
Corneilia Pechman and David W. Stewart. 1988. Advertising Repetition: A Critical Review of Wearin and Wearout. Current issues and research in advertising 11, 1-2 (1988), 285–329
1988
-
[22]
Řehůřek and P Sojka
R. Řehůřek and P Sojka. 2010. Software Framework for Topic Modelling with Large Corpora. Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks. http://is.muni.cz/publication/884893/en
2010
-
[23]
Howard A. Zhu M. Zhmoginov A. Chen L.-C Sandler, M. 2018. MobileNetV2: Inverted Residuals and Linear Bottlenecks. arXiv:1801.04381 [cs] (2018)
2018 arXiv
-
[24]
Okumura M Sato, T
Hashimoto T. Okumura M Sato, T. 2017. Implementation of a word segmentation dictionary called mecab-ipadic-NEologd and study on how to use it effectively Moriwaki et al. for information retrieval (in Japanese). In Proceedings of the Twenty-three Annual Meeting of the Associati...
2017
-
[25]
Susanne Schmidt and Martin Eisend. 2015. Advertising Repetition: A Meta- Analysis on Effective Frequency in Advertising. Journal of Advertising 44, 4 (Oct. 2015), 415–428. https://doi.org/10.1080/00913367.2015.1018460
2015
-
[26]
Schumann, Richard E
David W. Schumann, Richard E. Petty, and D. Scott Clemons. 1990. Predicting the Effectiveness of Different Strategies of Advertising Variation: A Test of the Repetition-Variation Hypotheses. Journal of Consumer Research 17, 2 (1990), 192–202
1990
-
[27]
Liang Tang, Yexi Jiang, Lei Li, and Tao Li. 2014. Ensemble Contextual Bandits for Personalized Recommendation. In Proceedings of the 8th ACM Conference on Recommender Systems - RecSys ’14 . ACM Press, Foster City, Silicon Valley, California, USA, 73–80. https://doi.org/10.1145...
2014
-
[28]
Liang Tang, Romer Rosales, Ajit Singh, and Deepak Agarwal. 2013. Auto- matic ad format selection via contextual bandits. In Proceedings of the 22nd ACM international conference on Conference on information & knowledge man- agement - CIKM ’13 . ACM Press, San Francisco, Califor...
2013
-
[29]
Rao Unnava and Robert E
H. Rao Unnava and Robert E. Burnkrant. 1991. Effects of Repeating Varied Ad Executions on Brand Name Memory. Journal of Marketing Research 28, 4 (Nov. 1991), 406–416. https://doi.org/10.1177/002224379102800403
1991 doi
-
[30]
Kilian Weinberger, Anirban Dasgupta, John Langford, Alex Smola, and Josh Attenberg. 2009. Feature hashing for large scale multitask learning. InProceedings of the 26th Annual International Conference on Machine Learning - ICML ’09 . ACM Press, Montreal, Quebec, Canada, 1–8. ht...
2009
-
[31]
Robert B. Zajonc. 1968. Attitudinal effects of mere exposure.Journal of Personality and Social Psychology 9, 2, Pt.2 (1968), 1–27. https://doi.org/10.1037/h0025848
1968 doi
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.