Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

Fatigue-Aware Ad Creative Selection

T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Pith's one-line read Adding a similarity-weighted fatigue score to an ad-selection bandit raised click-through rate and conversions in a one-week production experiment.

desk verdict A genuine production A/B test of a fatigue-aware ad creative selector, with a plausible but modest effect and an under-validated similarity metric. read the letter →

arxiv 1908.08936 v2 pith:TSS564B3 submitted 2019-08-21 cs.CY cs.LGstat.ML

classification cs.CYcs.LGstat.ML
keywords DisplayadvertisingfatiguerepetitionContextualbanditThompsonsamplingAdcreativeselectionReal-timebiddingOnlineexperiment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that online display ads can be chosen more effectively when the selection algorithm takes into account how tired a user has become of seeing an ad. It modifies an existing contextual-logistic selector—a bandit algorithm that balances trying different ads against showing the current best—by adding a fatigue score and its square as two extra features, where the fatigue score counts recent exposures to the same or similar creatives weighted by similarity. The modified algorithm ran for one week in a live production display-advertising system against the unmodified contextual bandit and against random selection. The paper reports that the fatigue-aware version outperformed both on click-through rate and on overall conversion rate, and that it served ads at lower measured fatigue levels. The practical point is that advertisers can counter ad wear-out with a small, low-latency change to the reward model rather than a new system.

What carries the argument

The load-bearing object is the fatigue score $\kappa_{i,t}(a_t)=h_{i,t}\cdot s(a_t)$, where $h_{i,t}$ is the user's exposure-history vector and $s(a_t)$ is a vector of similarity scores between the candidate creative and each creative the user has recently seen. Each similarity score is a weighted average of bag-of-words text cosine similarity and image cosine similarity from a pre-trained image network, with text weighted three times higher. The score enters the logistic reward predictor through the quadratic terms $b_1\kappa + b_2\kappa^2$, so the fitted curve is allowed to be increasing, decreasing, or inverted-U rather than assuming the textbook shape. The model is retrained daily on the previous day's logs and used with Thompson sampling, and the similarity vectors are precomputed offline, leaving only a history lookup and an inner product per candidate at decision time.

What would settle it

Re-run the same one-week A/B test with the fatigue features included but with the history vector $h_{i,t}$ replaced by a scrambled or permuted version that destroys the true exposure signal while keeping the same distribution; if CTR and CVR gains persist, the improvement is not caused by the measured fatigue mechanism. A weaker and cheaper check is to print the fitted $b_1,b_2$ and the implied fatigue-response curve from the random-arm logs: the central claim requires the curve to be measurably negative in the high-fatigue region, not flat or positive.

Watch

Extended reading notes

Core claim

The central claim is that adding two fatigue features to the reward predictor—$b_1\kappa_{i,t}(a_t)+b_2\kappa_{i,t}(a_t)^2$ in the log-odds of a logistic model—produces better ad choices than either the same model without fatigue or random selection. In the one-week production A/B test, normalized click-through rate was 1.08 for the fatigue-aware algorithm versus 1.04 for the baseline (1.00 for random), and normalized post-impression conversion rate was 1.09 versus 1.04, with the CTR and post-impression CVR differences reaching statistical significance; on post-click CVR the baseline was numerically higher but the gap was not significant. The fatigue feature $\kappa_{i,t}(a_t)$ is an inner product of the user's recent exposure-history vector with a vector of precomputed text-and-image similarity scores, which is why it can personalize without adding per-impression computation. The paper reads the result as operational evidence for the two-factor model of advertising repetition: initial repetition helps, fatigue eventually dominates, and a bandit that rotates creatives before that point can raise both clicks and conversions.

Load-bearing premise

The load-bearing premise is that a single number per user and candidate ad—a weighted sum of past exposures, with text similarity counted three times as heavily as image similarity—really captures how tired a user is of an ad, and that one curved relationship between that number and the chance of clicking or converting applies across all users and campaigns.

Editorial extensions

If this is right

  • Advertisers can add fatigue-awareness to an existing bandit by appending two features, with no change to the latency budget.
  • Creatives similar to what a user has already seen are served less often as exposure accumulates, reducing wear-out.
  • The fatigue feature can be tuned for different goals, since the observed relationship between fatigue, clicks, and conversions differs.
  • The daily retraining schedule means the method handles creatives that are added and removed without retraining the similarity network online.
  • The statistically significant production gains imply the fatigue score carries signal beyond the usual contextual features.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An untested extension: the 3:1 text-to-image weighting is a free parameter, and learning it from data could change the size of the gains.
  • The paper leaves these engineering choices untested: the 24-hour history window and one-impression-per-minute cap likely interact with the wear-out curve.
  • The same two-feature correction could apply to any recommender with pairwise-similar items and an exposure log, which would make the production result a proof of concept for content fatigue beyond ads.
  • The differing CTR and CVR fatigue curves visible in the paper's random-arm data suggest that optimizing clicks alone might over-rotate away from creatives that still convert.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a fatigue-aware ad creative selection algorithm for display advertising. The reward predictor in Eq. (2) augments a contextual logistic model with a quadratic term in a fatigue metric κ_i,t(a_t) = h_i,t · s(a_t), where h_i,t is the user's exposure history and s(a_t) is a similarity vector between the candidate creative and past creatives, computed as a weighted combination of text BoW and image MobileNetV2 cosine similarities (Section 4). The algorithm (Algorithm 2) embeds this predictor in a Thompson-sampling bandit. The paper reports a one-week production A/B test across three campaigns (Section 5) in which the fatigue-aware algorithm (FA) numerically outperforms a contextual-bandit baseline and random selection on CTR, overall CVR, and post-impression CVR, although only some differences are statistically significant. Post-hoc analyses in Section 6 examine the relationship between the fatigue measure and CTR/CVR.

Significance. If the result holds, the paper provides a simple, low-latency method for incorporating advertising fatigue into ad creative selection, with a real production deployment and realistic constraints. The derivation of the fatigue feature is non-circular because κ is constructed from exposure histories and precomputed similarities, not from the outcome. The production A/B test is a genuine strength, as are the practical details of implementation. However, the statistical support is partial, and the fatigue metric relies on an unvalidated similarity weighting, so the claim of demonstrated superiority needs further support before it can be accepted as stated.

major comments (3)
  1. [§5.2, Table 1] The statistical evidence for the central claim is weaker than the text suggests. Only CTR (1.08, P<0.1) and post-impression CVR (1.09, P<0.01) are marked significant for FA versus Baseline; overall CVR (1.12) is not significant, and post-click CVR numerically favors Baseline (1.05 vs 1.04). The paper should report confidence intervals, describe the testing procedure, and address multiple comparisons across the four metrics and three campaigns.
  2. [§4.2, Eq. (3)] The similarity score s(a,a') is a weighted average with text similarity given three times the weight of image similarity, justified solely by interpretability. Because κ is the only component that distinguishes FA from Baseline, the absence of any sensitivity analysis for this weight means the reported improvement could be an artifact of the chosen similarity scoring rather than a fatigue effect. The post-hoc validation in Section 6 uses the same κ to define fatigue and therefore cannot independently confirm the metric.
  3. [§5.1 and Table 2] The experiment duration is one week and the significance calculation is not described. The per-campaign CTRs in Table 2 vary widely (e.g., FA 1.21 and Baseline 0.95 for campaign A), and the paper notes campaign A has the least data; without confidence intervals or per-campaign significance tests, the stability claim for FA is not supported.
minor comments (6)
  1. [§1, Figure 1 caption] The caption reads 'Users jump to login screen of the game when click on these ads'; it should be 'when they click on these ads'.
  2. [§5.1, footnote 2] The footnote says four campaigns were used but one had only one creative; the results in Table 2 cover three campaigns. Please clarify the discrepancy.
  3. [§5.2] The sentence 'The results clearly show that the proposed algorithm successfully increased both clicks and conversions' overstates the significance reported in Table 1.
  4. [§6, Figure 10] Figure 10 uses data from the random algorithm only; the main text should state this explicitly rather than leaving it to the caption.
  5. [References] Several references contain garbled author names: [16] lists 'Clammer Koby' and 'Shie Monner' (should be Koby Crammer and Shie Mannor), and [20] appears malformed. Please correct the reference list.
  6. [§5.1] There are typos: 'SGDCliassifier' and 'skleran' should be 'SGDClassifier' and 'scikit-learn', respectively; 'metrices' in Section 6 should be 'metrics'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the fatigue feature is constructed from exposure history and precomputed similarities, coefficients are fitted, and the main result comes from a live production A/B test.

full rationale

The central comparison is not circular. The fatigue feature kappa_i,t(a_t) = h_i,t · s(a_t) (Eq. 3) is computed from exposure histories and precomputed text/image cosine similarities; it does not use click or conversion outcomes. The reward model (Eq. 2) is the same logistic predictor as the baseline (Eq. 1) plus kappa and kappa^2, with coefficients b1 and b2 estimated by L2-regularized logistic regression; the paper does not present these fitted coefficients as predictions. The claimed CTR/CVR improvements are measured in a week-long production A/B test against the baseline and random selection, so the main claim is empirically grounded rather than derived from its inputs. Section 6's post-hoc analysis uses the same kappa definition to characterize fatigue, and its Figure 10 explicitly uses logs from the random algorithm only, so it does not enter the derivation of the reward predictor. There are no load-bearing self-citations, no imported uniqueness theorems, and no ansatz smuggled in via citation. The manually chosen 3:1 text-to-image weight in Section 4.2 is an unvalidated modeling choice and a potential robustness concern, but it is not a circular step because the weight is not fitted from nor defined in terms of the predicted outcomes.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the assumption that psychological ad fatigue can be quantified as a similarity-weighted sum of exposure counts, plus a global quadratic link to click probability. Several hyperparameters and the text-versus-image similarity weight are chosen by hand, and the b1/b2 coefficients are fitted to data. No code or data is shipped, so the numerical results cannot be independently checked.

free parameters (7)
  • b1 (fatigue linear coefficient) = not reported
    Learned from previous-day training data in Eq. (2); the fitted value is not disclosed.
  • b2 (fatigue quadratic coefficient) = not reported
    Learned from previous-day training data in Eq. (2); the fitted value is not disclosed.
  • text_similarity_weight = 3.0
    Manually chosen weight for text similarity relative to image similarity in Section 4.2; directly shapes the fatigue metric.
  • alpha (Thompson sampling exploration parameter) = 0.01
    Set in Section 5.1; controls the degree of exploration and was tuned for the baseline via replay.
  • negative_sampling_rate = 0.05
    5% sampling rate for negative training data, set in Section 5.1.
  • L2_regularization_lambda = 0.0011
    Regularization parameter for logistic regression, set in Section 5.1.
  • feature_hash_length = 224 (Baseline), 226 (FA)
    Hashing-trick feature dimension chosen in Section 5.1; tuned using replay for the baseline and shared by FA.
assumptions (5)
  • domain assumption Two-factor model of advertising repetition: repeated exposure first increases then decreases consumer attitude, producing an inverted U.
    Invoked in Section 1 and Figure 2 as the theoretical motivation; if false, the fatigue feature lacks a clear behavioral basis.
  • ad hoc to paper Fatigue transfers between creatives in proportion to text and image similarity, and kappa = h dot s measures it.
    Stated as a conjecture in Section 4.1; no external calibration or independent validation is provided.
  • domain assumption Fatigue enters the log-odds of click as a quadratic function b1*kappa + b2*kappa^2 with coefficients shared across users and campaigns.
    Eq. (2) in Section 4; the paper says it does not presume the relationship shape, but it still imposes this specific functional form.
  • ad hoc to paper Deduplicating impressions within one minute and discarding histories older than 24 hours preserves the fatigue signal.
    Section 4.3; this data reduction affects up to 40% of users, and no sensitivity analysis is provided.
  • standard math The Laplace approximation with a diagonal Hessian gives an accurate posterior covariance for Thompson sampling.
    Section 3.2 relies on Chapelle and Li (2011); this is a standard approximation, but the diagonal-only Hessian is a simplification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Fatigue-Aware Ad Creative Selection." pith.science (2026). https://pith.science/paper/TSS564B3

@misc{pith2026190808936,
  author       = {Pith},
  title        = {Pith review of: Fatigue-Aware Ad Creative Selection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TSS564B3}},
  note         = {Machine review of arXiv:1908.08936}
}
read the original abstract

In online display advertising, selecting the most effective ad creative (ad image) for each impression is a crucial task for DSPs (Demand-Side Platforms) to fulfill their goals (click-through rate, number of conversions, revenue, and brand improvement). As widely recognized in the marketing literature, the effect of ad creative changes with the number of repetitive ad exposures. In this study, we propose an efficient and easy-to-implement ad creative selection algorithm that explicitly considers user's psychological status when selecting ad creatives. The proposed system was deployed in a real-world production environment and tested against the baseline algorithms. The results show superiority of the proposed algorithm.

Figures

Figures reproduced from arXiv: 1908.08936 by the authors.

Figure 1
Figure 1. Variation of ad creatives for a mobile game title. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. A graphical image of "Two-factor model" for ad [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Number of exposure to same ad creative for each [PITH_FULL_IMAGE:figures/full_fig_p002_3.png] view at source ↗
Figures from the paper (6 more)
Figure 5
Figure 5. Figure 5: An example of similarity matrix. of fatigue for each ad creative candidates can be sum of similarity between each candidate and creatives to which the user are exposed in the past [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 6
Figure 6. Figure 6: Calculation of fatigue. The values of fatigue de [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: Summary of fatigue-aware ad creative selection [PITH_FULL_IMAGE:figures/full_fig_p005_7.png]
Figure 8
Figure 8. Figure 8: The distribution of fatigue level at each impres [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]
Figure 9
Figure 9. Figure 9: The relationship between the number of ad expo [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 10
Figure 10. Figure 10: The relationship between the level of fatigue and [PITH_FULL_IMAGE:figures/full_fig_p007_10.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Creative4U: MLLMs-based Advertising Creative Image Selector with Comparative Reasoning

    cs.CV 2025-08 conditional novelty 6.0 of 10

    A multimodal language model, trained on a new annotated image-pair dataset and a recall-precision reinforcement reward, selects better advertising creatives and explains its choices, with measured offline and online gains.

Reference graph

Works this paper leans on

31 extracted references · 18 canonical work pages · cited by 1 Pith paper

  1. [1]

    Zoë Abrams and Erik Vee. 2007. Personalized Ad Delivery When Ads Fatigue: An Approximation Algorithm. In International Workshop on Web and Internet Economics. Springer, 535–540

  2. [2]

    Deepak Agarwal, Bee-Chung Chen, and Pradheep Elango. 2009. Spatio-Temporal Models for Estimating Click-through Rate. InProceedings of the 18th International Conference on World Wide Web - WWW ’09. ACM Press, Madrid, Spain, 21. https: //doi.org/10.1145/1526709.1526713

  3. [3]

    Alberto Bietti, Alekh Agarwal, and John Langford. 2018. A Contextual Bandit Bake-off. arXiv:1802.04064 [cs, stat] (Feb. 2018). http://arxiv.org/abs/1802.04064 arXiv: 1802.04064

  4. [4]

    Christopher M. Bishop. 2006.Pattern Recognition and Machine Learning. springer

  5. [5]

    Junyu Cao and Wei Sun. 2019. Dynamic Learning of Sequential Choice Bandit Problem under Marketing Fatigue. Proceedings of the AAAI Conference on Artifi- cial Intelligence 33 (July 2019), 3264–3271. https://doi.org/10.1609/aaai.v33i01. 33013264

  6. [6]

    Deepayan Chakrabarti, Ravi Kumar, Filip Radlinski, and Eli Upfal. 2009. Mortal Multi-Armed Bandits. In Advances in Neural Information Processing Systems 21 , D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou (Eds.). Curran Associates, Inc., 273–280. http://papers.nips.cc/paper/3580-mortal-multi-armed-bandits.pdf

  7. [7]

    Olivier Chapelle and Lihong Li. 2011. An Empirical Evaluation of Thompson Sampling. In Advances in neural information processing systems . Granada, Spain, 2249–2257

  8. [8]

    Hoffman, and Thomas P

    Patrali Chatterjee, Donna L. Hoffman, and Thomas P. Novak. 2003. Modeling the Clickstream: Implications for Web-Based Advertising Efforts. Marketing Science 22, 4 (Nov. 2003), 520–541. https://doi.org/10.1287/mksc.22.4.520.24906

Show all 31 references
  1. [9]

    Jiemiao Chen, Xiaojing Yang, and Robert E. Smith. 2016. The effects of creativity on advertising wear-in and wear-out.Journal of the Academy of Marketing Science 44, 3 (May 2016), 334–349. https://doi.org/10.1007/s11747-014-0414-5

  2. [10]

    François Chollet and others. 2015. Keras. (2015). https://keras.io

  3. [11]

    Narayanan

    Neha Gupta, Abhimanyu Das, Sandeep Pandey, and Vijay K. Narayanan. 2012. Factoring Past Exposure in Display Advertising Targeting. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’12) . ACM, New York, NY, USA, 1204–121...

  4. [12]

    Junpei Komiyama and Tao Qin. 2014. Time-Decaying Bandits for Non-stationary Systems. In Web and Internet Economics, Tie-Yan Liu, Qi Qi, and Yinyu Ye (Eds.). Vol. 8877. Springer International Publishing, Cham, 460–466. https://doi.org/10. 1007/978-3-319-13129-0_40

  5. [13]

    Tor Lattimore and Csaba SzepesvÂťari. 2019. Bandit Algorithms. https://tor- lattimore.com/downloads/book/book.pdf

  6. [14]

    Lakshmanan, Mitul Tiwari, and Sam Shah

    Pei Lee, Laks V.S. Lakshmanan, Mitul Tiwari, and Sam Shah. 2014. Modeling Impression Discounting in Large-Scale Recommender Systems. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD ’14 . ACM Press, New York, New York,...

  7. [15]

    Huitian Lei, Ambuj Tewari, and Susan A. Murphy. 2017. An Actor-Critic Contextual Bandit Algorithm for Personalized Mobile Health Interventions. arXiv:1706.09090 [cs, stat] (June 2017). http://arxiv.org/abs/1706.09090 arXiv: 1706.09090

  8. [16]

    Nir Levine, Clammer Koby, and Shie Monner. 2017. Rotting Bandits. Advances in Neural Information Processing Systems30 (2017). http://papers.nips.cc/paper/6900- rotting-bandits

  9. [17]

    Randall A Lewis. 2015. Worn-Out or Just Getting Started? The Impact of Fre- quency in Online Display Advertising. Boston, Massachusetts, USA

  10. [18]

    Schapire

    Lihong Li, Wei Chu, John Langford, and Robert E. Schapire. 2010. A Contextual- Bandit Approach to Personalized News Article Recommendation. Proceedings of the 19th international conference on World wide web - WWW ’10 (2010), 661. https://doi.org/10.1145/1772690.1772758 arXiv: ...

  11. [19]

    Hao Ma, Xueqing Liu, and Zhihong Shen. 2016. User Fatigue in Online News Recommendation. In Proceedings of the 25th International Conference on World Wide Web (WWW ’16) . International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, 1...

  12. [20]

    Yamashita T

    Kitauchi A. Yamashita T. Hirano Y. Matsuda H.-Takaoka K. Asahara M. Mat- sumoto, Y. [n. d.]. Japanese morphological analysis system ChaSen version 2.0 manual. ([n. d.])

  13. [21]

    Corneilia Pechman and David W. Stewart. 1988. Advertising Repetition: A Critical Review of Wearin and Wearout. Current issues and research in advertising 11, 1-2 (1988), 285–329

  14. [22]

    Řehůřek and P Sojka

    R. Řehůřek and P Sojka. 2010. Software Framework for Topic Modelling with Large Corpora. Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks. http://is.muni.cz/publication/884893/en

  15. [23]

    Howard A. Zhu M. Zhmoginov A. Chen L.-C Sandler, M. 2018. MobileNetV2: Inverted Residuals and Linear Bottlenecks. arXiv:1801.04381 [cs] (2018)

  16. [24]

    Okumura M Sato, T

    Hashimoto T. Okumura M Sato, T. 2017. Implementation of a word segmentation dictionary called mecab-ipadic-NEologd and study on how to use it effectively Moriwaki et al. for information retrieval (in Japanese). In Proceedings of the Twenty-three Annual Meeting of the Associati...

  17. [25]

    Susanne Schmidt and Martin Eisend. 2015. Advertising Repetition: A Meta- Analysis on Effective Frequency in Advertising. Journal of Advertising 44, 4 (Oct. 2015), 415–428. https://doi.org/10.1080/00913367.2015.1018460

  18. [26]

    Schumann, Richard E

    David W. Schumann, Richard E. Petty, and D. Scott Clemons. 1990. Predicting the Effectiveness of Different Strategies of Advertising Variation: A Test of the Repetition-Variation Hypotheses. Journal of Consumer Research 17, 2 (1990), 192–202

  19. [27]

    Liang Tang, Yexi Jiang, Lei Li, and Tao Li. 2014. Ensemble Contextual Bandits for Personalized Recommendation. In Proceedings of the 8th ACM Conference on Recommender Systems - RecSys ’14 . ACM Press, Foster City, Silicon Valley, California, USA, 73–80. https://doi.org/10.1145...

  20. [28]

    Liang Tang, Romer Rosales, Ajit Singh, and Deepak Agarwal. 2013. Auto- matic ad format selection via contextual bandits. In Proceedings of the 22nd ACM international conference on Conference on information & knowledge man- agement - CIKM ’13 . ACM Press, San Francisco, Califor...

  21. [29]

    Rao Unnava and Robert E

    H. Rao Unnava and Robert E. Burnkrant. 1991. Effects of Repeating Varied Ad Executions on Brand Name Memory. Journal of Marketing Research 28, 4 (Nov. 1991), 406–416. https://doi.org/10.1177/002224379102800403

  22. [30]

    Kilian Weinberger, Anirban Dasgupta, John Langford, Alex Smola, and Josh Attenberg. 2009. Feature hashing for large scale multitask learning. InProceedings of the 26th Annual International Conference on Machine Learning - ICML ’09 . ACM Press, Montreal, Quebec, Canada, 1–8. ht...

  23. [31]

    Robert B. Zajonc. 1968. Attitudinal effects of mere exposure.Journal of Personality and Social Psychology 9, 2, Pt.2 (1968), 1–27. https://doi.org/10.1037/h0025848

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.