Pith. sign in

REVIEW 3 major objections 5 minor 146 references

Auditing Differential Visibility of Political Content on TikTok

T0 review · 3 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read TikTok shadow-ban claims dissolve when political reach is analyzed at the account level, not by pooling hourly snapshots.

desk verdict The account-level null is probably right, but the paper overreaches by never using its off-topic videos as a within-account baseline. read the letter →

arxiv 2607.17356 v1 pith:BPZCYEVT submitted 2026-07-19 cs.SI cs.CY

Hazem Ibrahim , Tewoflos Girmay This is my paper
classification cs.SIcs.CY
keywords shadowbanningTikTokdifferentialvisibilitypseudoreplicationunitofanalysisalgorithmicreachpoliticalcontentengagementasymmetry
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to adjudicate whether TikTok suppresses one political side's reach. It tracks 2,753 videos from 67 curated accounts across three contested topics, collecting 556,946 hourly snapshots. A conventional pooled comparison finds an overwhelming reach gap, with p below 1e-140, but when the data are analyzed at the account level—the unit where stance is actually defined—the gap disappears. The authors find no evidence of moderate-to-large reach suppression on any topic, while a separate, robust asymmetry emerges: oppositional content earns more engagement per view. They argue the apparent shadow-ban signal is an artifact of pseudoreplication and stance-aligned confounds such as account size, video age, and content language.

What carries the argument

The account as the independent unit of analysis, with follower-normalized plays as the reach proxy and median video reach collapsed per account. The machinery includes randomization inference with permutation of stance labels, account-bootstrap confidence intervals, a constructed-null simulation that measures how pooling inflates the false-positive rate, and a variance decomposition showing an intraclass correlation of 0.98 within videos and a design effect near 180.

What would settle it

A concrete falsification would come from platform-side exposure logs or a randomized holdout: randomly assign otherwise identical videos or accounts from both stances and compare For You feed impressions. If stance-conditioned impressions differ meaningfully after matching account size, video age, and language, the reach null would fail. Short of platform cooperation, an external audit could test within-account, post-level demotion: for accounts that post both on-topic and off-topic videos, compare peak follower-normalized reach by topic and stance; a stance-dependent drop that survives contro

Watch

Extended reading notes

Core claim

The paper's central claim is that the apparent shadow-ban signal in its corpus is manufactured by the unit of analysis. Pooling tens of thousands of autocorrelated hourly snapshots treats near-identical repeated measurements as independent observations, producing vanishingly small p-values and a false detection. Analyzed at the account level, using median video reach per account and randomization inference, no topic shows a significant reach difference by stance, with Benjamini-Hochberg corrected q values near 0.9 and effect sizes near zero. The same design and sample do detect a clear asymmetry on another outcome: oppositional content receives more likes, shares, and saves per view, which i

Load-bearing premise

The load-bearing assumption is that follower-normalized plays are a valid proxy for algorithmic reach; if follower counts are noisy or the normalization over-penalizes large accounts, the account-level null could mask real suppression, since the proxy cannot separate algorithmic distribution from audience behavior.

Editorial extensions

If this is right

  • If correct, external claims of TikTok shadow banning that rely on pooled post-level or snapshot-level comparisons should be reanalyzed at the account or cluster level, and many reported gaps may dissolve.
  • The reach null is informative and bounded: the design rules out reach differences larger than roughly |Cliff's delta| = 0.5 on the Trump and Israel/Palestine topics at 95% confidence, while smaller effects remain possible.
  • The robust engagement asymmetry, with oppositional content earning more reactions per view, reframes perceived suppression as audience intensity rather than algorithmic demotion.
  • Credible visibility audits should report design effects and effective sample sizes, pre-register the analysis grid, use cluster-aware models, and control for account size, video age, and language before attributing any gap to stance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication left implicit is that the same unit-of-analysis failure could inflate apparent moderation asymmetries in other opaque ranking systems, not only TikTok; audits of search, news feeds, and recommendation surfaces should check design effects before reporting significance.
  • Because stance is nearly collinear with content language on the Israel/Palestine topic, an audit that holds language and audience region fixed could reveal smaller reach effects that this design cannot resolve; the paper's own bounds admit effects below |delta| = 0.5.
  • The engagement asymmetry invites follow-up work on audience composition: whether oppositional content attracts more mobilised followers or whether the For You feed distributes it to users who react more intensely. That distinction has policy consequences but requires platform-side exposure logs.
  • The paper's null should be read as a bounded observational result, not a proof of platform neutrality; a randomized on-platform experiment assigning near-identical content by stance would be the natural next test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper tests allegations that TikTok shadow bans political content by tracking 67 curated political accounts across three contested topics (U.S. immigration enforcement, Trump coverage, Israel/Palestine) and recording hourly play, like, comment, share, and save counters for 2,753 videos, totaling 556,946 hourly observations. A pooled video-hour analysis shows a massive topic-conditional reach gap (p < 10^-140). When the same data are analyzed at the account level—the unit at which accounts are sampled and stance is assigned—every reach contrast is null after Benjamini–Hochberg correction (q ≈ 0.9), with Cliff's delta near zero. The paper attributes the pooled result to pseudoreplication and stance-aligned confounds (account size, video age, language), and reports a secondary, robust engagement-per-view asymmetry favoring oppositional content. It concludes that, at magnitudes the design can resolve, there is no evidence of moderate-to-large differential reach suppression on these topics, and that the apparent shadow-ban signal is better explained by audience engagement than by suppression.

Significance. If the account-level null is accepted, the paper makes a valuable methodological contribution to external platform auditing: it demonstrates on a single corpus that the unit-of-analysis choice manufactures or dissolves a shadow-ban detection, provides a constructed-null calibration showing the pooled test rejects nearly 100% of the time under a true null, and reports a specification curve spanning 24 defensible analyses. The pre-registered grid, randomization inference, cluster-robust and wild-cluster-bootstrap adjustments, and the planned release of de-identified analysis code are exemplary practices. The account-level equivalence bounds and power analysis are also genuinely informative, provided the claims are restricted to the estimand actually identified. The main weakness is that the headline wording sometimes exceeds what the design can identify, and the engagement-per-view 'design check' is not logically independent of the reach hypothesis.

major comments (3)
  1. [Abstract and §3.5, §5.3] The headline claim 'we find no evidence of moderate-to-large reach suppression on any topic' is broader than the estimand. The design never compares on-topic versus off-topic videos within the same account, so content-level demotion that depresses only a subset of an account's videos (e.g., its on-topic posts) need not move account medians and is not identified. The 1,813 off-topic videos already in the panel would permit a within-account baseline comparison that holds follower count and audience composition fixed. The post-level model in §5.3 still contrasts sides across accounts, with account-clustered errors, and does not fix this gap. Either add such an analysis or explicitly restrict the conclusion to 'no evidence of account-level side differences in follower-normalized reach.'
  2. [§4.2, §4.3] The engagement-per-view asymmetry is presented as a design check that 'shows the design can detect effects of this magnitude' and thereby makes the reach null informative. This is not a validity check for the reach null: if suppression reduces views while reactions stay roughly constant, engagement per view rises exactly as observed. The reach power analysis in §4.2 is the appropriate support for the null. The engagement result should be framed as a secondary descriptive finding, not as evidence that the design would have detected a reach effect. The paper acknowledges the logical caveat in one sentence ('Higher engagement per view does not by itself rule out reach suppression') but then proceeds to use it as a design check; this needs revision.
  3. [§5.2, §7] The equivalence bound and power analysis apply to follower-normalized plays, a proxy the paper itself notes 'confounds algorithmic distribution with audience behavior.' The paper also reports that reach is sublinear in followers (slope 0.73 for Trump, near zero for Israel/Palestine), so the normalization may systematically absorb or create side differences. Given the central claim is a null, the claim should be explicitly restricted to the proxy—'no evidence of moderate-to-large differential visibility in follower-normalized plays'—or supplemented with a sensitivity analysis using raw plays or residualized reach controlling for followers. The current wording in the abstract and conclusion risks overstating what the observable data support.
minor comments (5)
  1. [Table 1] The column header 'n_a/n_b' is not defined in the table or its caption. Please label the side counts explicitly (e.g., 'establishment/oppositional').
  2. [Abstract] The phrase '556,946 follower-normalized views' is imprecise; these are hourly snapshot observations, not distinct videos or views. Consider 'hourly observation rows.'
  3. [Appendix A, Table 3] The roster has 67 curated accounts but only 65 posted at least one tracked video. Please clarify the two non-posting accounts explicitly in the text, as the current table note explains the count but not the specific reason.
  4. [Figure 2] The p-value notation is inconsistent: one panel shows 'p<1e-300' while the text and other panels use 'p<1e-300' or 'p < 10^-300'. Use a single standard notation for values below floating-point precision.
  5. [§5.2] The sentence about the collector's default U.S. country setting is confusing because the public counters are global totals. Consider moving this explanation to the Limitations section or clarifying that the U.S. setting affects discoverability/language environment rather than the measured counts.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the account-level reach null is not defined by its inputs, and the few self-citations are background context, not load-bearing.

full rationale

The paper's central claim — no evidence of moderate-to-large account-level reach suppression — is not circular. Stance is assigned from curated account sides fixed before data collection ('Side membership reflects each account's publicly expressed position... the roster was finalized before hourly collection began'), and the reach outcome is follower-normalized plays, an independent proxy. The headline account-level null is not fitted to that outcome or derived from it by construction. The engagement-per-view asymmetry is a separate pre-registered outcome used as a positive control, and the paper explicitly disclaims that it rules out suppression ('Higher engagement per view does not by itself rule out suppression'), so it is not a renamed version of the reach claim. The few self-citations (Ibrahim et al. 2023, 2026; Aldahoul et al. 2026) appear only as background motivation or methodological precedent, e.g. 'TikTok's recommender allegedly favored Republican-aligned content' to motivate the hypothesis; none supplies a uniqueness theorem, an ansatz, or the load-bearing justification for the null. The manuscript's own stated limitations — Section 7: 'Reach is measured by follower-normalized plays... cannot separate algorithmic demotion from audience behavior; only platform-side logs could'; 'That gold set is author-labeled, which bounds the check'; and ICE being too sparse — are validity and measurement caveats, not definitional circularity. The skeptic's concern about the absence of a within-account on-topic/off-topic baseline is an identification limitation, not a reduction of the result to its inputs. Therefore no circular step is exhibited.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

The central claim rests on the choice of account as independent unit and on follower-normalized plays as the reach proxy; both are explicit domain assumptions. No fitted constants enter the primary test, and no new entities are postulated.

assumptions (6)
  • domain assumption Accounts are the independent sampling units, and stance is assigned at the account level from curated side labels.
    Section 3.5: 'Because accounts are the units we sample and the level at which side labels are curated, our primary estimand is an account-level side difference.' The null is defined at this level; if posts were the independent unit, the pooled analysis could be meaningful.
  • domain assumption Follower-normalized plays is a valid reach proxy.
    Section 7: 'Reach is measured by follower-normalized plays, a proxy for distribution rather than a direct read of the ranker, and it cannot separate algorithmic demotion from audience behavior; only platform-side logs could.' If this proxy is biased, the null may not reflect algorithmic reach.
  • domain assumption The LLM ensemble classifier correctly identifies on-topic videos.
    Section 3.4 validates against a human gold set (F1=0.88/0.80), but the filter remains an assumption; if it misclassifies by side, the account-level sample could be skewed.
  • domain assumption The curated convenience sample is representative enough to support a bounded claim.
    Section 3.2: 'The dataset is therefore a curated, stratified convenience sample, not a probability sample of political TikTok, which limits external validity.' The internal null is valid, but generalizing to all political TikTok requires this assumption.
  • domain assumption TikAPI counters reflect true public TikTok counters.
    Section 3.1 describes collection via a third-party wrapper; measurement error in the counters would affect all results.
  • standard math Standard statistical assumptions for Mann-Whitney U, randomization inference, and BH-FDR hold.
    Section 3.6; these are standard and do not need further justification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Auditing Differential Visibility of Political Content on TikTok." pith.science (2026). https://pith.science/paper/BPZCYEVT

@misc{pith2026260717356,
  author       = {Pith},
  title        = {Pith review of: Auditing Differential Visibility of Political Content on TikTok},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BPZCYEVT}},
  note         = {Machine review of arXiv:2607.17356}
}
read the original abstract

Allegations that TikTok shadow bans political content shape what creators post, what advertisers fund, and how regulators act, yet they are hard to adjudicate because platforms do not disclose how content is ranked. We test the claim with a dense hourly panel of 556,946 follower-normalized views across 2,753 videos from 67 accounts curated into pro and anti sides of three contested topics (U.S. immigration enforcement, Trump coverage, and Israel/Palestine). On-topic videos are identified by a multi-step classifier, and stance is taken from each account's curated side. The conventional analysis appears to answer yes. Pooling the hourly snapshots, the topic-conditional reach gap reaches p < 10^-140. Analyzed at the account level, the independent unit at which we sample and assign stance, the gap disappears. Every account-level reach contrast is null after correction (BH-FDR q near 0.9). We find no evidence of moderate-to-large reach suppression on any topic. The null is informative. Account-level confidence intervals and a power analysis rule out such effects. As a design check, the same framework detects a clear asymmetry on a different outcome. Oppositional content (anti-Trump, pro-Palestine) earns more engagement per view rather than less reach (Cliff's delta = -0.51 and -0.64; q < 0.03). Higher engagement does not by itself rule out suppression, but shows the design can detect effects of this magnitude. The apparent reach gap is an artifact of two factors. The first is pseudoreplication, which counts tens of thousands of autocorrelated video-hours as independent observations; the second is confounding, since the side that looks suppressed is larger and, on Israel/Palestine, posts mostly in Arabic. In this corpus, what is taken for a shadow ban is better explained by a more engaged audience than by a suppressed one. We close with what a credible visibility audit requires.

Figures

Figures reproduced from arXiv: 2607.17356 by the authors.

Figure 1
Figure 1. Oppositional content earns more reactions per view, a direction inconsistent with a visibility penalty. Per-account rates [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Inflating the unit of analysis manufactures significance. The same reach comparison at three units; the [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Stance is confounded with (a) account size, (b) video age at first observation, and (c) content language on the Is [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Sign and significance of the reach “effect” flip [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: False-positive rate as a function of the analysis [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Israel/Palestine account-level comparison, all [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

146 extracted references · 2 linked inside Pith

  1. [1]

    V.; Dolan, C

    Aarts, E.; Verhage, M.; Veenvliet, J. V.; Dolan, C. V.; and Van Der Sluis, S. 2014. A solution to dependency: using multilevel analysis to accommodate nested data. Nature neuroscience, 17(4): 491--496

    2014

  2. [3]

    Are, C. 2022. The Shadowban Cycle: an autoethnography of pole dancing, nudity and censorship on Instagram. Feminist Media Studies, 22(8): 2002--2019

    2022

  3. [4]

    Bandy, J. 2021. Problematic machine behavior: A systematic literature review of algorithm audits. Proceedings of the acm on human-computer interaction, 5(CSCW1): 1--34

    2021

  4. [5]

    J.; Levy, R.; Scheepers, C.; and Tily, H

    Barr, D. J.; Levy, R.; Scheepers, C.; and Tily, H. J. 2013. Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of memory and language, 68(3): 255--278

    2013

  5. [6]

    M.; Gebru, T.; McMillan-Major, A.; and Shmitchell, S

    Bender, E. M.; Gebru, T.; McMillan-Major, A.; and Shmitchell, S. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, 610--623

    2021

  6. [7]

    Benjamini, Y.; and Hochberg, Y. 1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal statistical society: series B (Methodological), 57(1): 289--300

    1995

  7. [8]

    Bhandari, A.; and Bimo, S. 2022. Why’s everyone on TikTok now? The algorithmized self and the future of self-making on social media. Social media+ society, 8(1): 20563051221086241

    2022

  8. [9]

    Blasi, D.; Anastasopoulos, A.; and Neubig, G. 2022. Systematic inequalities in language technology performance across the world’s languages. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 5486--5505

    2022

Show all 146 references
  1. [10]

    Boeker, M.; and Urman, A. 2022. An empirical investigation of personalization factors on TikTok. In Proceedings of the ACM web conference 2022, 2298--2309

    2022

  2. [11]

    Colin Cameron, A.; and Miller, D. L. 2015. A practitioner’s guide to cluster-robust inference. Journal of human resources, 50(2): 317--372

    2015

  3. [12]

    Collaboration, O. S. 2015. Estimating the reproducibility of psychological science. Science, 349(6251): aac4716

    2015

  4. [13]

    Cotter, K. 2019. Playing the visibility game: How digital influencers and algorithms negotiate influence on Instagram. New media & society, 21(4): 895--913

    2019

  5. [14]

    Shadowbanning is not a thing

    Cotter, K. 2023. “Shadowbanning is not a thing”: Black box gaslighting and the power to independently know and credibly critique algorithms. Information, Communication & Society, 26(6): 1226--1243

    2023

  6. [15]

    What are you doing, TikTok?

    Delmonaco, D.; Mayworm, S.; Thach, H.; Guberman, J.; Augusta, A.; and Haimson, O. L. 2024. " What are you doing, TikTok?": How Marginalized Social Media Users Perceive, Theorize, and" Prove" Shadowbanning. Proceedings of the ACM on Human-Computer Interaction, 8(CSCW1): 1--39

    2024

  7. [16]

    E.; and Meisner, C

    Duffy, B. E.; and Meisner, C. 2023. Platform governance at the margins: Social media creators’ experiences with algorithmic (in) visibility. Media, Culture & Society, 45(2): 285--304

    2023

  8. [17]

    I always assumed that I wasn't really that close to [her]

    Eslami, M.; Rickman, A.; Vaccaro, K.; Aleyasen, A.; Vuong, A.; Karahalios, K.; Hamilton, K.; and Sandvig, C. 2015. " I always assumed that I wasn't really that close to [her]" Reasoning about invisible algorithms in news feeds. In Proceedings of the 33rd annual ACM conference ...

    2015

  9. [18]

    A.; and Vissel, B

    Galbraith, S.; Daniel, J. A.; and Vissel, B. 2010. A study of clustered data and approaches to its analysis. Journal of Neuroscience, 30(32): 10601--10608

    2010

  10. [19]

    Gelman, A.; and Loken, E. 2016. The statistical crisis in science. The best writing on mathematics (Pitici M, ed), 102: 305--318

    2016

  11. [20]

    Gerlitz, C.; and Helmond, A. 2013. The like economy: Social buttons and the data-intensive web. New Media & Society, 15(8): 1348--1365

    2013

  12. [21]

    Gilardi, F.; Alizadeh, M.; and Kubli, M. 2023. ChatGPT outperforms crowd workers for text-annotation tasks. Proceedings of the National Academy of Sciences, 120(30): e2305016120

    2023

  13. [22]

    Gillespie, T. 2018. Custodians of the Internet: Platforms, content moderation, and the hidden decisions that shape social media. Yale University Press

    2018

  14. [23]

    Gillespie, T. 2022. Do not recommend? Reduction as a form of content moderation. Social Media+ Society, 8(3): 20563051221117552

    2022

  15. [24]

    Goldman, E. 2021. Content moderation remedies

    2021

  16. [25]

    M.; et al

    Gonz \'a lez-Bail \'o n, S.; Lazer, D.; Barber \'a , P.; Zhang, M.; Allcott, H.; Brown, T.; Crespo-Tenorio, A.; Freelon, D.; Gentzkow, M.; Guess, A. M.; et al. 2023. Asymmetric ideological segregation in exposure to political news on Facebook. Science, 381(6656): 392--398

    2023

  17. [26]

    Gorwa, R.; Binns, R.; and Katzenbach, C. 2020. Algorithmic content moderation: Technical and political challenges in the automation of platform governance. Big Data & Society, 7(1): 2053951719897945

    2020

  18. [27]

    M.; Malhotra, N.; Pan, J.; Barber \'a , P.; Allcott, H.; Brown, T.; Crespo-Tenorio, A.; Dimmery, D.; Freelon, D.; Gentzkow, M.; et al

    Guess, A. M.; Malhotra, N.; Pan, J.; Barber \'a , P.; Allcott, H.; Brown, T.; Crespo-Tenorio, A.; Dimmery, D.; Freelon, D.; Gentzkow, M.; et al. 2023. How do social media feed algorithms affect attitudes and behavior in an election campaign? Science, 381(6656): 398--404

    2023

  19. [28]

    Hannak, A.; Sapiezynski, P.; Molavi Kakhki, A.; Krishnamurthy, B.; Lazer, D.; Mislove, A.; and Wilson, C. 2013. Measuring personalization of web search. In Proceedings of the 22nd international conference on World Wide Web, 527--538

    2013

  20. [29]

    Harwell, D. 2023. TikTok was slammed for its pro- Palestinian hashtags. But it's not alone. The Washington Post

    2023

  21. [30]

    M.; and Watts, D

    Hosseinmardi, H.; Ghasemian, A.; Clauset, A.; Mobius, M.; Rothschild, D. M.; and Watts, D. J. 2021. Examining the consumption of radical content on YouTube. Proceedings of the national academy of sciences, 118(32): e2101967118

    2021

  22. [31]

    Human Rights Watch . 2023. Meta's Broken Promises: Systemic Censorship of Palestine Content on Instagram and Facebook . Technical report, Human Rights Watch

    2023

  23. [32]

    Hurlbert, S. H. 1984. Pseudoreplication and the design of ecological field experiments. Ecological monographs, 54(2): 187--211

    1984

  24. [33]

    Hussein, E.; Juneja, P.; and Mitra, T. 2020. Measuring misinformation in video search platforms: An audit study on YouTube. Proceedings of the ACM on human-computer interaction, 4(CSCW1): 1--27

    2020

  25. [34]

    I.; O’Brien, C.; Belli, L.; Schlaikjer, A.; and Hardt, M

    Husz \'a r, F.; Ktena, S. I.; O’Brien, C.; Belli, L.; Schlaikjer, A.; and Hardt, M. 2022. Algorithmic amplification of politics on Twitter. Proceedings of the national academy of sciences, 119(1): e2025334119

    2022

  26. [35]

    Ibrahim, H.; AlDahoul, N.; Lee, S.; Rahwan, T.; and Zaki, Y. 2023. YouTube’s recommendation algorithm is left-leaning in the United States. PNAS nexus, 2(8): pgad264

    2023

  27. [36]

    D.; Aldahoul, N.; Kaufman, A

    Ibrahim, H.; Jang, H. D.; Aldahoul, N.; Kaufman, A. R.; Rahwan, T.; and Zaki, Y. 2026. Systematic partisan content skews in TikTok during the 2024 US elections. Nature, 1--8

    2026

  28. [37]

    Ioannidis, J. P. 2005. Why most published research findings are false. PLoS medicine, 2(8): e124

    2005

  29. [38]

    Jaidka, K.; Mukerjee, S.; and Lelkes, Y. 2023. Silenced on social media: The gatekeeping functions of shadowbans in the American Twitterverse . Journal of Communication, 73(2): 163--178

    2023

  30. [39]

    Joshi, P.; Santy, S.; Budhiraja, A.; Bali, K.; and Choudhury, M. 2020. The state and fate of linguistic diversity and inclusion in the NLP world. In Proceedings of the 58th annual meeting of the association for computational linguistics, 6282--6293

    2020

  31. [40]

    Karizat, N.; Delmonaco, D.; Eslami, M.; and Andalibi, N. 2021. Algorithmic folk theories and identity: How TikTok users co-produce knowledge of identity and engage in algorithmic resistance. Proceedings of the ACM on human-computer interaction, 5(CSCW2): 1--44

    2021

  32. [41]

    Killip, S.; Mahfoud, Z.; and Pearce, K. 2004. What is an intracluster correlation coefficient? Crucial concepts for primary care researchers. The Annals of Family Medicine, 2(3): 204--208

    2004

  33. [42]

    Kish, L. 1965. Survey sampling

    1965

  34. [43]

    Klug, D.; Qin, Y.; Evans, M.; and Kaufman, G. 2021. Trick and please. A mixed-method study on user assumptions about the TikTok algorithm. In Proceedings of the 13th ACM web science conference 2021, 84--92

    2021

  35. [44]

    Kreutzer, J.; Caswell, I.; Wang, L.; et al. 2022. Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets. Transactions of the Association for Computational Linguistics, 10: 50--72

    2022

  36. [45]

    Lazic, S. E. 2010. The problem of pseudoreplication in neuroscientific studies: is it affecting your analysis? BMC neuroscience, 11(1): 5

    2010

  37. [46]

    E.; Clarke-Williams, C

    Lazic, S. E.; Clarke-Williams, C. J.; and Munaf \`o , M. R. 2018. What exactly is ‘N’in cell culture and animal experiments? PLoS biology, 16(4): e2005282

    2018

  38. [47]

    Le Merrer, E.; Morgan, B.; and Tr \'e dan, G. 2021. Setting the record straighter on shadow banning. In IEEE INFOCOM 2021-IEEE conference on computer communications, 1--10. IEEE

    2021

  39. [48]

    Matzko, P. 2024. Lies, Damned Lies, and Statistics: A Misleading Study Compares TikTok and Instagram . Cato at Liberty, Cato Institute

    2024

  40. [49]

    C.; Papakyriakopoulos, O.; and Hegelich, S

    Medina Serrano, J. C.; Papakyriakopoulos, O.; and Hegelich, S. 2020. Dancing to the partisan beat: A first analysis of political communication on TikTok. In Proceedings of the 12th ACM Conference on Web Science, 257--266

    2020

  41. [50]

    S.; Robertson, R

    Metaxa, D.; Park, J. S.; Robertson, R. E.; Karahalios, K.; Wilson, C.; Hancock, J.; and Sandvig, C. 2021. Auditing algorithms: Understanding algorithmic systems from the outside in. Foundations and Trends in Human--Computer Interaction , 14(4): 272--344

    2021

  42. [51]

    Network Contagion Research Institute . 2023. A Tik-Tok-ing Timebomb: How TikTok 's Global Platform Anomalies Align with the Chinese Communist Party 's Geostrategic Objectives. Technical report, Network Contagion Research Institute

    2023

  43. [52]

    Nicholas, G. 2022. Shedding light on shadowbanning. Technical report

    2022

  44. [53]

    A.; Ebersole, C

    Nosek, B. A.; Ebersole, C. R.; DeHaven, A. C.; and Mellor, D. T. 2018. The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11): 2600--2606

    2018

  45. [54]

    Y.; Allcott, H.; Brown, T.; Crespo-Tenorio, A.; Dimmery, D.; et al

    Nyhan, B.; Settle, J.; Thorson, E.; Wojcieszak, M.; Barber \'a , P.; Chen, A. Y.; Allcott, H.; Brown, T.; Crespo-Tenorio, A.; Dimmery, D.; et al. 2023. Like-minded sources on Facebook are prevalent but not polarizing. Nature, 620(7972): 137--144

    2023

  46. [56]

    Rader, E.; and Gray, R. 2015. Understanding user beliefs about algorithmic curation in the Facebook news feed. In Proceedings of the 33rd annual ACM conference on human factors in computing systems, 173--182

    2015

  47. [57]

    Radway, E.; and Edelson, L. 2025. Updating our Israel/Gaza Analysis 2 Years On. Cybersecurity for Democracy

    2025

  48. [59]

    E.; Jiang, S.; Joseph, K.; Friedland, L.; Lazer, D.; and Wilson, C

    Robertson, R. E.; Jiang, S.; Joseph, K.; Friedland, L.; Lazer, D.; and Wilson, C. 2018. Auditing partisan audience bias within google search. Proceedings of the ACM on human-computer interaction, 2(CSCW): 1--22

    2018

  49. [60]

    Sandvig, C.; Hamilton, K.; Karahalios, K.; and Langbort, C. 2014. Auditing algorithms: Research methods for detecting discrimination on internet platforms. Data and discrimination: converting critical concerns into productive inquiry, 22(2014): 4349--4357

    2014

  50. [61]

    Sap, M.; Card, D.; Gabriel, S.; Choi, Y.; and Smith, N. A. 2019. The Risk of Racial Bias in Hate Speech Detection. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), 1668--1678

    2019

  51. [62]

    Savolainen, L. 2022. The shadow banning controversy: perceived governance and algorithmic folklore. Media, culture & society, 44(6): 1091--1109

    2022

  52. [63]

    L.; Martin, D

    Silberzahn, R.; Uhlmann, E. L.; Martin, D. P.; Anselmi, P.; Aust, F.; Awtrey, E.; Bahn \' k, S .; Bai, F.; Bannard, C.; Bonnier, E.; et al. 2018. Many analysts, one data set: Making transparent how variations in analytic choices affect results. Advances in methods and practice...

    2018

  53. [64]

    P.; Nelson, L

    Simmons, J. P.; Nelson, L. D.; and Simonsohn, U. 2011. False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological science, 22(11): 1359--1366

    2011

  54. [65]

    P.; and Nelson, L

    Simonsohn, U.; Simmons, J. P.; and Nelson, L. D. 2020. Specification curve analysis. Nature human behaviour, 4(11): 1208--1214

    2020

  55. [66]

    Simpson, E.; and Semaan, B. 2021. For you, or for" you"? Everyday LGBTQ+ encounters with TikTok. Proceedings of the ACM on human-computer interaction, 4(CSCW3): 1--34

    2021

  56. [67]

    Steegen, S.; Tuerlinckx, F.; Gelman, A.; and Vanpaemel, W. 2016. Increasing transparency through a multiverse analysis. Perspectives on Psychological Science, 11(5): 702--712

    2016

  57. [68]

    Vombatkere, K.; Mousavi, S.; Zannettou, S.; Roesner, F.; and Gummadi, K. P. 2024. Tiktok and the art of personalization: investigating exploration and exploitation on social media feeds. In Proceedings of the ACM Web Conference 2024, 3789--3797

    2024

  58. [69]

    Yarchi, M.; and Boxman-Shabtai, L. 2025. The Image War Moves to TikTok : Evidence from the May 2021 Round of the Israeli--Palestinian Conflict. Digital Journalism, 13(1): 115--135

    2025

  59. [70]

    Ye, J.; Luceri, L.; and Ferrara, E. 2025. Auditing Political Exposure Bias: Algorithmic Amplification on Twitter/X During the 2024 U.S. Presidential Election. In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency

    2025

  60. [71]

    Zeng, J.; and Kaye, D. B. V. 2022. From content moderation to visibility moderation: A case study of platform governance on TikTok. Policy & Internet, 14(1): 79--95

    2022

  61. [72]

    Ziems, C.; Held, W.; Shaikh, O.; Chen, J.; Zhang, Z.; and Yang, D. 2024. Can Large Language Models Transform Computational Social Science? Computational Linguistics, 50(1): 237--291

    2024

  62. [73]

    Zuckerberg, M. 2018. A Blueprint for Content Governance and Enforcement. Facebook note, 15 November 2018

    2018

  63. [74]

    IEEE INFOCOM 2021-IEEE conference on computer communications , pages=

    Setting the record straighter on shadow banning , author=. IEEE INFOCOM 2021-IEEE conference on computer communications , pages=. 2021 , organization=

    2021

  64. [75]

    Media, culture & society , volume=

    The shadow banning controversy: perceived governance and algorithmic folklore , author=. Media, culture & society , volume=. 2022 , publisher=

    2022

  65. [76]

    New media & society , volume=

    Playing the visibility game: How digital influencers and algorithms negotiate influence on Instagram , author=. New media & society , volume=. 2019 , publisher=

    2019

  66. [77]

    Shadowbanning is not a thing

    “Shadowbanning is not a thing”: Black box gaslighting and the power to independently know and credibly critique algorithms , author=. Information, Communication & Society , volume=. 2023 , publisher=

    2023

  67. [78]

    New media & society , volume=

    Want to be on the top? Algorithmic power and the threat of invisibility on Facebook , author=. New media & society , volume=. 2012 , publisher=

    2012

  68. [79]

    2018 , publisher=

    Custodians of the Internet: Platforms, content moderation, and the hidden decisions that shape social media , author=. 2018 , publisher=

    2018

  69. [80]

    Social Media+ Society , volume=

    Do not recommend? Reduction as a form of content moderation , author=. Social Media+ Society , volume=. 2022 , publisher=

    2022

  70. [81]

    Big Data & Society , volume=

    Algorithmic content moderation: Technical and political challenges in the automation of platform governance , author=. Big Data & Society , volume=. 2020 , publisher=

    2020

  71. [82]

    Feminist Media Studies , volume=

    The Shadowban Cycle: an autoethnography of pole dancing, nudity and censorship on Instagram , author=. Feminist Media Studies , volume=. 2022 , publisher=

    2022

  72. [83]

    Media, Culture & Society , volume=

    Platform governance at the margins: Social media creators’ experiences with algorithmic (in) visibility , author=. Media, Culture & Society , volume=. 2023 , publisher=

    2023

  73. [84]

    What are you doing, TikTok?

    " What are you doing, TikTok?": How Marginalized Social Media Users Perceive, Theorize, and" Prove" Shadowbanning , author=. Proceedings of the ACM on Human-Computer Interaction , volume=. 2024 , publisher=

    2024

  74. [85]

    Center for Democracy & Technology

    Shedding light on shadowbanning , author=. Center for Democracy & Technology. https://cdt. org/insights/shedding-light-on-shadowbanning , year=

  75. [86]

    Policy & Internet , volume=

    From content moderation to visibility moderation: A case study of platform governance on TikTok , author=. Policy & Internet , volume=. 2022 , publisher=

    2022

  76. [87]

    Content moderation remedies , author=

  77. [88]

    2018 , howpublished =

    Zuckerberg, Mark , title =. 2018 , howpublished =

    2018

  78. [89]

    Data and discrimination: converting critical concerns into productive inquiry , volume=

    Auditing algorithms: Research methods for detecting discrimination on internet platforms , author=. Data and discrimination: converting critical concerns into productive inquiry , volume=

  79. [90]

    Foundations and Trends

    Auditing algorithms: Understanding algorithmic systems from the outside in , author=. Foundations and Trends. 2021 , publisher=

    2021

  80. [91]

    Proceedings of the acm on human-computer interaction , volume=

    Problematic machine behavior: A systematic literature review of algorithm audits , author=. Proceedings of the acm on human-computer interaction , volume=. 2021 , publisher=

    2021

  81. [92]

    Proceedings of the 22nd international conference on World Wide Web , pages=

    Measuring personalization of web search , author=. Proceedings of the 22nd international conference on World Wide Web , pages=

  82. [93]

    Proceedings of the ACM on human-computer interaction , volume=

    Measuring misinformation in video search platforms: An audit study on YouTube , author=. Proceedings of the ACM on human-computer interaction , volume=. 2020 , publisher=

    2020

  83. [94]

    Proceedings of the national academy of sciences , volume=

    Examining the consumption of radical content on YouTube , author=. Proceedings of the national academy of sciences , volume=. 2021 , publisher=

    2021

  84. [95]

    Proceedings of the ACM on human-computer interaction , volume=

    Auditing partisan audience bias within google search , author=. Proceedings of the ACM on human-computer interaction , volume=. 2018 , publisher=

    2018

  85. [96]

    Proceedings of the ACM web conference 2022 , pages=

    An empirical investigation of personalization factors on TikTok , author=. Proceedings of the ACM web conference 2022 , pages=

    2022

  86. [97]

    Proceedings of the ACM Web Conference 2024 , pages=

    Tiktok and the art of personalization: investigating exploration and exploitation on social media feeds , author=. Proceedings of the ACM Web Conference 2024 , pages=

    2024

  87. [98]

    Proceedings of the 12th ACM Conference on Web Science , pages=

    Dancing to the partisan beat: A first analysis of political communication on TikTok , author=. Proceedings of the 12th ACM Conference on Web Science , pages=

  88. [99]

    A mixed-method study on user assumptions about the TikTok algorithm , author=

    Trick and please. A mixed-method study on user assumptions about the TikTok algorithm , author=. Proceedings of the 13th ACM web science conference 2021 , pages=

    2021

  89. [100]

    Social media+ society , volume=

    Why’s everyone on TikTok now? The algorithmized self and the future of self-making on social media , author=. Social media+ society , volume=. 2022 , publisher=

    2022

  90. [101]

    Proceedings of the ACM on human-computer interaction , volume=

    Algorithmic folk theories and identity: How TikTok users co-produce knowledge of identity and engage in algorithmic resistance , author=. Proceedings of the ACM on human-computer interaction , volume=. 2021 , publisher=

    2021

  91. [102]

    Proceedings of the ACM on human-computer interaction , volume=

    For you, or for" you"? Everyday LGBTQ+ encounters with TikTok , author=. Proceedings of the ACM on human-computer interaction , volume=. 2021 , publisher=

    2021

  92. [103]

    I always assumed that I wasn't really that close to [her]

    " I always assumed that I wasn't really that close to [her]" Reasoning about invisible algorithms in news feeds , author=. Proceedings of the 33rd annual ACM conference on human factors in computing systems , pages=

  93. [104]

    Proceedings of the 33rd annual ACM conference on human factors in computing systems , pages=

    Understanding user beliefs about algorithmic curation in the Facebook news feed , author=. Proceedings of the 33rd annual ACM conference on human factors in computing systems , pages=

  94. [105]

    Nature , pages=

    Systematic partisan content skews in TikTok during the 2024 US elections , author=. Nature , pages=. 2026 , publisher=

    2024

  95. [106]

    Proceedings of the national academy of sciences , volume=

    Algorithmic amplification of politics on Twitter , author=. Proceedings of the national academy of sciences , volume=. 2022 , publisher=

    2022

  96. [107]

    Science , volume=

    How do social media feed algorithms affect attitudes and behavior in an election campaign? , author=. Science , volume=. 2023 , publisher=

    2023

  97. [108]

    Nature , volume=

    Like-minded sources on Facebook are prevalent but not polarizing , author=. Nature , volume=. 2023 , publisher=

    2023

  98. [109]

    PNAS nexus , volume=

    YouTube’s recommendation algorithm is left-leaning in the United States , author=. PNAS nexus , volume=. 2023 , publisher=

    2023

  99. [110]

    arXiv preprint arXiv:2601.20413 , year=

    Schadenfreude in the Digital Public Sphere: A cross-national and decade-long analysis of Facebook news engagement , author=. arXiv preprint arXiv:2601.20413 , year=

  100. [111]

    Science , volume=

    Asymmetric ideological segregation in exposure to political news on Facebook , author=. Science , volume=. 2023 , publisher=

    2023

  101. [112]

    Ecological monographs , volume=

    Pseudoreplication and the design of ecological field experiments , author=. Ecological monographs , volume=. 1984 , publisher=

    1984

  102. [113]

    BMC neuroscience , volume=

    The problem of pseudoreplication in neuroscientific studies: is it affecting your analysis? , author=. BMC neuroscience , volume=. 2010 , publisher=

    2010

  103. [114]

    PLoS biology , volume=

    What exactly is ‘N’in cell culture and animal experiments? , author=. PLoS biology , volume=. 2018 , publisher=

    2018

  104. [115]

    Nature neuroscience , volume=

    A solution to dependency: using multilevel analysis to accommodate nested data , author=. Nature neuroscience , volume=. 2014 , publisher=

    2014

  105. [116]

    Journal of Neuroscience , volume=

    A study of clustered data and approaches to its analysis , author=. Journal of Neuroscience , volume=. 2010 , publisher=

    2010

  106. [117]

    The Annals of Family Medicine , volume=

    What is an intracluster correlation coefficient? Crucial concepts for primary care researchers , author=. The Annals of Family Medicine , volume=. 2004 , publisher=

    2004

  107. [118]

    , author=

    Survey sampling. , author=. 1965 , publisher=

    1965

  108. [119]

    Journal of memory and language , volume=

    Random effects structure for confirmatory hypothesis testing: Keep it maximal , author=. Journal of memory and language , volume=. 2013 , publisher=

    2013

  109. [120]

    Journal of human resources , volume=

    A practitioner’s guide to cluster-robust inference , author=. Journal of human resources , volume=. 2015 , publisher=

    2015

  110. [121]

    The best writing on mathematics (Pitici M, ed) , volume=

    The statistical crisis in science , author=. The best writing on mathematics (Pitici M, ed) , volume=

  111. [122]

    Psychological science , volume=

    False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant , author=. Psychological science , volume=. 2011 , publisher=

    2011

  112. [123]

    Nature human behaviour , volume=

    Specification curve analysis , author=. Nature human behaviour , volume=. 2020 , publisher=

    2020

  113. [124]

    Perspectives on Psychological Science , volume=

    Increasing transparency through a multiverse analysis , author=. Perspectives on Psychological Science , volume=. 2016 , publisher=

    2016

  114. [125]

    Journal of the Royal statistical society: series B (Methodological) , volume=

    Controlling the false discovery rate: a practical and powerful approach to multiple testing , author=. Journal of the Royal statistical society: series B (Methodological) , volume=. 1995 , publisher=

    1995

  115. [126]

    PLoS medicine , volume=

    Why most published research findings are false , author=. PLoS medicine , volume=. 2005 , publisher=

    2005

  116. [127]

    Science , volume=

    Estimating the reproducibility of psychological science , author=. Science , volume=. 2015 , publisher=

    2015

  117. [128]

    Proceedings of the National Academy of Sciences , volume=

    The preregistration revolution , author=. Proceedings of the National Academy of Sciences , volume=. 2018 , publisher=

    2018

  118. [129]

    Advances in methods and practices in psychological science , volume=

    Many analysts, one data set: Making transparent how variations in analytic choices affect results , author=. Advances in methods and practices in psychological science , volume=. 2018 , publisher=

    2018

  119. [130]

    Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Systematic inequalities in language technology performance across the world’s languages , author=. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  120. [131]

    Proceedings of the 58th annual meeting of the association for computational linguistics , pages=

    The state and fate of linguistic diversity and inclusion in the NLP world , author=. Proceedings of the 58th annual meeting of the association for computational linguistics , pages=

  121. [132]

    , title =

    Sap, Maarten and Card, Dallas and Gabriel, Saadia and Choi, Yejin and Smith, Noah A. , title =. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL) , pages =. 2019 , doi =

    2019

  122. [133]

    Proceedings of the 2021 ACM conference on fairness, accountability, and transparency , pages=

    On the dangers of stochastic parrots: Can language models be too big? , author=. Proceedings of the 2021 ACM conference on fairness, accountability, and transparency , pages=

    2021

  123. [134]

    Transactions of the Association for Computational Linguistics , volume =

    Kreutzer, Julia and Caswell, Isaac and Wang, Lisa and others , title =. Transactions of the Association for Computational Linguistics , volume =. 2022 , doi =

    2022

  124. [135]

    ChatGPT outperforms crowd workers for text-annotation tasks , journal =

    Gilardi, Fabrizio and Alizadeh, Meysam and Kubli, Ma. ChatGPT outperforms crowd workers for text-annotation tasks , journal =. 2023 , doi =

    2023

  125. [136]

    Computational Linguistics , volume =

    Ziems, Caleb and Held, William and Shaikh, Omar and Chen, Jiaao and Zhang, Zhehao and Yang, Diyi , title =. Computational Linguistics , volume =. 2024 , doi =

    2024

  126. [137]

    , title =

    Reiss, Michael V. , title =. 2023 , howpublished =. 2304.11085 , archivePrefix =

    2023 arXiv

  127. [138]

    2023 , howpublished =

    Pangakis, Nicholas and Wolken, Samuel and Fasching, Neil , title =. 2023 , howpublished =. 2306.00176 , archivePrefix =

    2023 arXiv

  128. [139]

    New Media & Society , volume =

    Gerlitz, Carolin and Helmond, Anne , title =. New Media & Society , volume =. 2013 , doi =

    2013

  129. [140]

    Digital Journalism , volume =

    Yarchi, Moran and Boxman-Shabtai, Lillian , title =. Digital Journalism , volume =. 2025 , doi =

    2025

  130. [141]

    Journal of Communication , volume =

    Jaidka, Kokil and Mukerjee, Subhayan and Lelkes, Yphtach , title =. Journal of Communication , volume =. 2023 , doi =

    2023

  131. [142]

    Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency , year =

    Ye, Jinyi and Luceri, Luca and Ferrara, Emilio , title =. Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency , year =

    2025

  132. [143]

    2023 , url =

    A Tik-Tok-ing Timebomb: How. 2023 , url =

    2023

  133. [144]

    2023 , url =

    Meta's Broken Promises: Systemic Censorship of. 2023 , url =

    2023

  134. [145]

    2025 , url =

    Radway, Elizabeth and Edelson, Laura , title =. 2025 , url =

    2025

  135. [146]

    2024 , url =

    Matzko, Paul , title =. 2024 , url =

    2024

  136. [147]

    2023 , url =

    Harwell, Drew , title =. 2023 , url =

    2023

  137. [148]

    Wilkinson, Mark D. and Dumontier, Michel and Aalbersberg, IJsbrand Jan and Appleton, Gabrielle and Axton, Myles and Baak, Arie and Blomberg, Niklas and Boiten, Jan-Willem and da Silva Santos, Luiz Bonino and Bourne, Philip E. and others , journal=. The. 2016 , publisher=

    2016

  138. [149]

    Communications of the ACM , volume=

    Datasheets for datasets , author=. Communications of the ACM , volume=. 2021 , publisher=

    2021

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.