REVIEW 5 major objections 6 minor 1 cited by
Coordinated Inauthentic Behavior on TikTok: Challenges and Opportunities for Detection in a Video-First Ecosystem
T0 review · 5 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Network-based similarity signals can surface coordinated inauthentic behavior on TikTok, while duets, stitches, and video replies mostly reflect organic engagement.
desk verdict The first serious adaptation of network-based CIB detection to TikTok, with real descriptive payoff, but the 'effective detection' claim outruns the validation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is a user-user similarity network built from seven behavioral traces: hashtag sequences, synchronized posting, shared external domains, duet and stitch interactions, video replies, speech similarity, and video similarity. For the text- and metadata-based traces, users are linked to behavioral entities in a bipartite graph, edges are weighted by TF-IDF to down-weight common hashtags and domains, and the graph is projected onto users so that cosine similarity scores how much two users behave alike. For speech and video, direct embedding similarity is computed instead, with deliberately strict filters: only user pairs that share identical or near-identical content at exactly the same time, repeatedly, are linked. Dense subnetworks are then isolated by keeping high-centrality users and, in the stricter variant, removing low-weight edges before centrality pruning. The interpretive claim doing the work is that anomalously high collective similarity, concentrated in a few dense clusters, marks inauthentic coordination rather than coincidence or shared trends.
What would settle it
A decisive test would be to apply the same pruning pipeline to a large set of ordinary fan or trend communities on TikTok and check whether dense clusters with identical hashtag sequences, synchronized posting, and unified username patterns appear at similar rates; if they do, the signal does not separate CIB from organic collective behavior. A second test is to inspect infrastructure metadata—account creation bursts, device or IP overlap—for the 68-user August hashtag cluster; if those accounts are run by many unrelated people, the coordination interpretation fails.
Extended reading notes
Core claim
The paper's central discovery is that behavioral similarity networks built from content and timing traces can expose coordinated inauthentic accounts on TikTok. Applying the pipeline to 1.35 million videos posted between August and October 2024, it finds dense clusters of accounts that share identical hashtag sequences, post within minutes of each other, link to the same external websites, reuse identical audio tracks, or post near-duplicate videos with zero-second gaps. Manual inspection of these clusters shows strongly patterned usernames, repeated watermark reuse, synthetic voiceovers that mimic a presidential candidate, and split-screen videos that imitate duets to hide duplicated political content. The paper also finds that duet, stitch, and video-reply traces do not produce suspicious clusters, and interprets this as evidence that those features are used organically on TikTok. The conclusion is that CIB detection on TikTok is feasible, but only when detection combines multimodal similarity signals rather than relying on platform-native interaction features.
Load-bearing premise
The whole interpretation rests on the assumption that dense clusters of accounts with unusually high behavioral similarity are coordinated inauthentic actors, rather than organic fan groups, users who share templates, or people swept up in the same viral trend; the study has no ground-truth labels and validates clusters through manual inspection and later suspension.
Editorial extensions
If this is right
- Hashtag-sequence overlap, synchronized posting, shared domains, speech similarity, and video similarity each surface plausible CIB clusters in election-related TikTok data, so these signals generalize from text platforms to TikTok.
- Duet, stitch, and video-reply behaviors, and textual similarity of comments, should not be treated as CIB indicators on TikTok without further evidence, because they reflect organic participation in political conversation.
- CIB on TikTok now routinely uses multimodal and generative-AI tactics—synthetic voiceovers, split-screen formats, watermarks, and cross-modal content reuse—so detection must include audio and visual similarity.
- Because different signals reveal largely disjoint account sets even inside one campaign, operational CIB detection requires integrating several weak signals rather than thresholding one indicator.
- Most of the accounts surfaced by the method were still active at the time of analysis, implying that platform enforcement lags behind what computational detection can find.
Reading between the lines
- The paper's election-specific setup leaves open whether the same signals separate CIB from organic behavior in non-political contexts, such as fan communities or viral challenges, where synchronized posting and hashtag reuse are common.
- The negative result for duets, stitches, and replies is tied to this dataset and time window; coordinated campaigns could still use those features in smaller or more targeted operations that the aggregate analysis would miss.
- The strict similarity thresholds mean the method probably catches only tight, centrally controlled operations; looser coordination by human-run networks may need fusion models or temporal pattern mining to surface.
- A direct testable extension is to run the same pipeline on a control sample of trending non-political TikTok content and measure how many dense clusters appear; that baseline would quantify the false-positive rate the current study does not report.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper presents an unsupervised, network-based framework for detecting coordinated inauthentic behavior (CIB) on TikTok. It builds user-user similarity networks from seven behavioral traces—hashtag sequences, synchronized posting, co-domain links, duet/stitch interactions, video replies, speech similarity, and video similarity—and prunes these networks using eigenvector centrality and edge-weight thresholds. The pipeline is applied to roughly 1.35 million election-related TikTok videos from August through October 2024, and the authors report clusters that they attribute to candidate-support campaigns, fundraising spam, synthetic-voice networks, and content-duplication operations. They argue that several signals (hashtag sequences, co-domains, synchronized posting, speech similarity, video similarity) are effective for CIB detection, while TikTok-native interaction signals (duet, stitch, video reply) mostly reflect organic engagement. The paper also reports robustness simulations under simulated data loss and manual annotation agreement for cluster inspection.
Significance. If the effectiveness claims held up, this would be a valuable first empirical mapping of CIB on a video-first platform and a meaningful extension of network-based coordination detection beyond text-centric sites. The paper has clear strengths: a detailed and reproducible pipeline with public code, a large dataset, explicit negative results, robustness checks under simulated data loss, and documented inter-annotator agreement. However, the headline claim that several indicators 'effectively detect CIB' is not currently supported by the evidence presented. The evaluation relies on post-hoc inspection of clusters produced by the method itself, with no ground-truth labels, no baseline or null-model comparison, and asymmetric use of account-suspension status. The contribution is therefore best read as an exploratory case study until validation is added or the claims are appropriately scoped.
major comments (5)
- [Methodology (3); Results, Hashtag Sequence and Co-Domain] The central claim that indicators 'effectively detect CIB' is not established because clusters are selected using the same behavioral signal that defines them. For example, the hashtag-sequence cluster is presented as evidence of coordination because its accounts share identical hashtag sequences, but that is exactly the criterion used to build the network; no comparison is made to organic accounts that also share hashtag sequences, such as participants in viral challenges or template-driven creators. The manuscript also states that the authors 'report the most significant and interpretable coordination patterns uncovered' (Methodology, Section 3), which makes the effectiveness claim unfalsifiable. I recommend adding a null-model or baseline evaluation: apply the identical pipeline to matched non-election corpora, compare detected clusters to organic user groups matched on activity volume, and report precision with respect to a randomly sampled, independently annotated set of clusters rather than only showcased ones.
- [Results, Co-Domain and Discussion] Account-suspension status cannot serve as validation in the way it is currently used. When a reported cluster is suspended, the paper treats this as confirmation ('all accounts previously involved ... have been removed or suspended'); when it is not, it is treated as an enforcement gap ('only a small fraction of the suspicious accounts ... were suspended or removed'). Both interpretations are consistent with any outcome, so this evidence does not test the detection method. The paper should either obtain independent labels, such as platform takedown disclosures or known campaign assets, and report detection precision/recall against them, or explicitly scope the contribution as an exploratory case study and avoid effectiveness language.
- [Methodology (2); Limitations] Several load-bearing thresholds are introduced without sensitivity analysis: the 98th and 95th percentile centrality cutoffs, the 99.5th percentile edge-weight cutoff, the 0-second posting gap, the minimum of two exact matches, the ViSiL 0.9 similarity threshold, and the four-word minimum speech segment. The Limitations section concedes that 'no single pruning or filtering strategy was consistently effective across all indicators' and that trace-specific tuning is required. Because the effectiveness claim depends on these choices, the paper should report how cluster composition and the set of 'effective' signals change over a reasonable range of thresholds, or justify the choices with a parameter-free argument.
- [Results, Video Similarity, Figure 9] The baseline of 'organic (non-coordinated) accounts' used in Figure 9 is not defined. If it is simply the complement of detected accounts, the comparison is partly circular; if it is a hand-picked set, the selection criteria must be described. The temporal claims, such as the sharp peak at zero hours and the 6pm-8pm UTC concentration, also lack confidence intervals or significance tests, so their evidentiary weight is unclear.
- [Appendix, Impact of Data Loss] The data-loss robustness assessment computes retention of 'coordinated accounts' relative to the clusters already produced by the original pipeline. This demonstrates stability under random deletion but does not validate that those accounts are coordinated, and it cannot, by itself, support the claim that the method is robust to API data loss in an externally meaningful sense. Please either validate against an external ground truth or clearly label this as a stability check rather than a detection-accuracy result.
minor comments (6)
- [Abstract vs. Introduction and Methodology] The abstract reports 793K videos, while the Introduction and Methodology report approximately 1.35M videos; please reconcile these numbers.
- [Results and Appendix, NMI values] The paper states that NMI scores among the four candidate-support networks are 'consistently low' but only one value (NMI = 0.1) appears in the Appendix; please report the full set of values and specify the interpretation threshold.
- [Related Work and Discussion] The phrase 'first empirical foundation' should be qualified to 'first empirical foundation for CIB detection specifically,' since prior TikTok studies on misinformation and conspiracy content are cited in the Related Work section.
- [Code availability] The code link is a shortened URL; for reproducibility, please provide a permanent DOI or archived repository.
- [Methodology, Speech Similarity] The DBSCAN clustering of mel-spectrograms uses hyperparameters selected via the k-distance graph, but the specific choice of epsilon and min_samples is not reported; please provide these details or a reference to the exact procedure.
- [Results, Hashtag Sequence] Figure 1 reports that only usernames with more than five AI-voiceover videos are displayed, but the procedure for identifying AI-generated voiceovers from spectrogram clusters is described only at a high level; please include the classification criteria.
Circularity Check
Pipeline is mostly self-contained, but the effectiveness claim rests partly on a self-referential validation loop in which the trace used to build clusters is also cited as evidence of coordination.
-
self definitional
[Methodology §'Pruning Similarity Networks'; Results §'CIB Detection via Hashtag Sequence' (after Fig. 1)]
"We apply both pruning strategies across all behavioral indicators and report the most significant and interpretable coordination patterns uncovered. ... Such uniformity reflects a high degree of collective similarity, a pattern associated with CIB as demonstrated by (Luceri et al. 2024)."
The clusters are produced by pruning a similarity network built from a specific behavioral trace, e.g., identical hashtag sequences. The same uniformity is then cited as evidence that the cluster is coordinated, backed by the authors' own prior work. Because the reported clusters are selected as the 'most significant and interpretable,' a positive result is partly built into the selection criterion rather than tested against an external label. Independent checks (manual annotation, watermark and AI-voice inspection, later suspensions) limit but do not eliminate this self-referential validation loop.
full rationale
This is not a formal derivation paper; it is an unsupervised, exploratory case study. The pipeline constructs similarity networks from behavioral traces and prunes them, and no fitted parameter is renamed as an independent prediction. No equation makes an output definitionally equal to an input. The main circularity-adjacent issue is validation: clusters are selected as the 'most significant and interpretable coordination patterns uncovered,' and the uniformity of the defining trace is then cited as evidence of CIB. This loop is partially broken by independent manual annotation of usernames and watermarks, the ECAPA-TDNN audio-forensic comparison against verified recordings, and later account suspensions. The self-citations (Luceri et al. 2024; Cinus et al. 2025) are methodological lineage and are not used as a uniqueness theorem; they do not by themselves force the paper's conclusions. The lack of a labeled baseline is a correctness and generalizability limitation, not a circularity per se. Score 2 reflects a partially self-referential validation loop while acknowledging the independent evidence that keeps the central claim from being vacuous.
Assumptions & free parameters
free parameters (6)
- Eigenvector centrality percentile threshold =
98th percentile for hashtag and co-domain; 95th percentile for synchronous posting after edge filter
- Edge cosine similarity percentile threshold =
99.5th percentile for synchronous posting
- Time synchronization window =
0-second gap for speech and video similarity; 5-minute bins for synchronous posting
- Minimum repeated exact matches =
2
- ViSiL video similarity threshold =
greater than 0.9
- Minimum speech segment length =
4 words
assumptions (4)
- domain assumption Anomalously high behavioral similarity across the selected traces indicates underlying coordination.
- domain assumption Clusters surviving centrality pruning and edge filtering are unlikely to arise organically.
- domain assumption Manual inspection can reliably distinguish synthetic from authentic voices and coordinated from organic accounts.
- domain assumption The TikTok Research API dataset and third-party crawler provide a sufficiently representative sample.
Cite this review
Pith. "Pith review of Coordinated Inauthentic Behavior on TikTok: Challenges and Opportunities for Detection in a Video-First Ecosystem." pith.science (2026). https://pith.science/paper/VBOHMDHB
@misc{pith2026250510867,
author = {Pith},
title = {Pith review of: Coordinated Inauthentic Behavior on TikTok: Challenges and Opportunities for Detection in a Video-First Ecosystem},
year = {2026},
howpublished = {\url{https://pith.science/paper/VBOHMDHB}},
note = {Machine review of arXiv:2505.10867}
}
read the original abstract
Detecting coordinated inauthentic behavior (CIB) is central to the study of online influence operations. However, most methods focus on text-centric platforms, leaving video-first ecosystems like TikTok largely unexplored. To address this gap, we develop and evaluate a computational framework for detecting CIB on TikTok, leveraging a network-based approach adapted to the platform's unique content and interaction structures. Building on existing approaches, we construct user similarity networks based on shared behaviors, including synchronized posting, repeated use of similar captions, multimedia content reuse, and hashtag sequence overlap, and apply graph pruning techniques to identify dense networks of likely coordinated accounts. Analyzing a dataset of 793K TikTok videos related to the 2024 U.S. Presidential Election, we uncover a range of coordinated activities, from synchronized amplification of political narratives to semi-automated content replication using AI-generated voiceovers and split-screen video formats. Our findings show that while traditional coordination indicators generalize well to TikTok, other signals, such as those based on textual similarity of video transcripts or Duet and Stitch interactions, prove ineffective, highlighting the platform's distinct content norms and interaction mechanics. This work provides the first empirical foundation for studying and detecting CIB on TikTok, paving the way for future research into influence operations in short-form video platforms.
Figures
Figures from the paper (12 more)
Forward citations
Cited by 1 Pith paper
-
Emergent Coordinated Behaviors in Networked LLM Agents: Modeling the Strategic Dynamics of Information Operations
In networked LLM agents, simply informing influence-operation agents of their teammates' identities produces coordination nearly as strong as collective deliberation and voting.
Reference graph
Works this paper leans on
-
[4]
Entrena-Serrano, C.; Degeling, M.; Romano, S.; and C ¸ etin, R
Ecapa-tdnn: Emphasized channel attention, propagation and aggregation in tdnn based speaker verification.arXiv preprint arXiv:2005.07143. Entrena-Serrano, C.; Degeling, M.; Romano, S.; and C ¸ etin, R. B
arXiv 2005
-
[5]
Ezzeddine, F.; Luceri, L.; Ayoub, O.; Sbeity, I.; Nogara, G.; Ferrara, E.; and Giordano, S
TikTok’s Research API: Problems Without Ex- planations.arXiv preprint arXiv:2506.09746. Ezzeddine, F.; Luceri, L.; Ayoub, O.; Sbeity, I.; Nogara, G.; Ferrara, E.; and Giordano, S
-
[6]
Kordopatis-Zilos, G.; Papadopoulos, S.; Patras, I.; and Kom- patsiaris, Y
It takes a village to manipulate the media: coordinated link sharing behavior during 2018 and 2019 Italian elections.In- formation, Communication & Society, 23(6): 867–891. Kordopatis-Zilos, G.; Papadopoulos, S.; Patras, I.; and Kom- patsiaris, Y
work page 2018
-
[7]
Leveraging Large Language Models to Detect Influence Campaigns on Social Media. InCompanion Proc. ACM on Web Conference 2024, 1459–1467. Luceri, L.; Giordano, S.; and Ferrara, E
work page 2024
-
[8]
Detecting troll behavior via inverse reinforcement learning: A case study of russian trolls in the 2016 us election. InProc. in- ternational AAAI conference on web and social media, 417–
work page 2016
-
[9]
Unmasking the web of deceit: Uncovering coordinated ac- tivity to expose information operations on twitter. InProc. ACM on Web Conference 2024, 2530–2541. Magelinski, T.; Ng, L.; and Carley, K
work page 2024
-
[10]
Mariconti, E.; Onaolapo, J.; Ahmad, S
Detection and characterization of coordi- nated online behavior: A survey.arXiv:2408.01257. Mariconti, E.; Onaolapo, J.; Ahmad, S. S.; Nikiforou, N.; Egele, M.; Nikiforakis, N.; and Stringhini, G
-
[11]
Minici, M.; Luceri, L.; Fabbri, F.; and Ferrara, E
Uncovering coordinated cross-platform information opera- tions: Threatening the integrity of the 2024 US presidential election.First Monday. Minici, M.; Luceri, L.; Fabbri, F.; and Ferrara, E
work page 2024
Show all 18 references
-
[14]
Web Conference
Tracking the 2024 US Presidential Election Chatter on Tiktok: A Public Multimodal Dataset.Companion Proc. Web Conference. Shang, L.; Kou, Z.; Zhang, Y .; and Wang, D
2024
-
[15]
Characterizing online engagement with disinformation and conspiracies in the 2020 US presidential election. InProc. international AAAI conference on web and social media, 908–919. Tardelli, S.; Nizzoli, L.; Avvenuti, M.; Cresci, S.; and Tesconi, M. 2024a. Multifaceted online c...
2020
-
[16]
Suscepti- bility to unreliable information sources: Swift adoption with minimal exposure. InProc. ACM Web Conference 2024, 4674–4685. Zannettou, S.; Caulfield, T.; De Cristofaro, E.; Sirivianos, M.; Stringhini, G.; and Blackburn, J
2024
-
[18]
Can- didateX
Using data from a short video social media platform to identify emergent monkey- pox conspiracy theories.JAMA Network Open. Appendix Impact of Data Loss on Detection Strategies: A Robustness Assessment We assessed the robustness of our detection strategies un- der simulated da...
2024
-
[2019]
InCompanion Proc
Disinformation warfare: Understanding state-sponsored trolls on Twitter and their influence on the web. InCompanion Proc. 2019 world wide web conference, 218–226. Zeng, J.; and Kaye, D. B. V
2019
-
[2020]
Barnekow, V .; Binder, D.; Kromrey, N.; Munaretto, P.; Schaad, A.; and Schmieder, F
# tulsaflop: A case study of algorithmically-influenced collective action on tik- tok.arXiv preprint arXiv:2012.07716. Barnekow, V .; Binder, D.; Kromrey, N.; Munaretto, P.; Schaad, A.; and Schmieder, F
2012 arXiv
-
[2021]
Coordinated behavior on social media in 2019 UK general election. InProc. International AAAI Conference on Web and Social Media, 443–454. Nwala, A. C.; Flammini, A.; and Menczer, F
2019
-
[2022]
Online coordination: methods and comparative case studies of coordinated groups across four events in the united states. InProc. 14th ACM Web Science Conference 2022, 12–21. Ng, L. H. X.; Cruickshank, I. J.; and Carley, K. M
2022
-
[2024]
arXiv:2407.12545
Con- spiracy theories and where to find them on TikTok. arXiv:2407.12545. Cresci, S.; Di Pietro, R.; Petrocchi, M.; Spognardi, A.; and Tesconi, M
-
[2025]
Ex- posing cross-platform coordinated inauthentic activity in the run-up to the 2024 us election. InProc. ACM on Web Con- ference 2025, 541–559. Corso, F.; Pierri, F.; and De Francisci Morales, G
2024
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.