In temporal experiments with treatment habituation, the minimax optimal pulse design is an unbalanced completely randomized design that assigns more units to always-treated and always-control arms and fewer to each pulse arm.
Measuring Long-term Impact of Ads on LinkedIn Feed
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Organic updates (from a member's network) and sponsored updates (or ads, from advertisers) together form the newsfeed on LinkedIn. The newsfeed, the default homepage for members, attracts them to engage, brings them value and helps LinkedIn grow. Engagement and Revenue on feed are two critical, yet often conflicting objectives. Hence, it is important to design a good Revenue-Engagement Tradeoff (RENT) mechanism to blend ads in the feed. In this paper, we design experiments to understand how members' behavior evolve over time given different ads experiences. These experiences vary on ads density, while the quality of ads (ensured by relevance models) is held constant. Our experiments have been conducted on randomized member buckets and we use two experimental designs to measure the short term and long term effects of the various treatments. Based on the first three months' data, we observe that the long term impact is at a much smaller scale than the short term impact in our application. Furthermore, we observe different member cohorts (based on user activity level) adapt and react differently over time.
fields
stat.ME 1years
2019 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Minimax designs for causal effects in temporal experiments with treatment habituation
In temporal experiments with treatment habituation, the minimax optimal pulse design is an unbalanced completely randomized design that assigns more units to always-treated and always-control arms and fewer to each pulse arm.