pith. sign in

arxiv: 2401.06205 · v1 · submitted 2024-01-11 · 💻 cs.SI

Unsupervised detection of coordinated information operations in the wild

Pith reviewed 2026-05-24 04:22 UTC · model grok-4.3

classification 💻 cs.SI
keywords coordinated information operationsunsupervised detectionBayesian inferencesocial media analysisTwitterinauthentic accountsvariational inferenceinformation operations
0
0 comments X

The pith

An unsupervised Bayesian method detects coordinated inauthentic accounts by grouping those with similar characteristics and narratives.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper develops an unsupervised technique to find coordinated information operations on social media. It applies Bayesian inference to cluster accounts that share account-level traits and push overlapping narratives, using amortized variational inference to handle millions of accounts efficiently. Tests on five real operations from Twitter show the method raises detection power substantially over naive baselines and over using flags or narratives separately. The gains hold when observations are sparse, markers of inauthenticity are weak, and the coordinated accounts form only a tiny share of the data. The approach comes close to supervised performance without needing labeled examples.

Core claim

The paper introduces an unsupervised method that uses Bayesian inference to identify groups of accounts sharing similar account-level characteristics and targeting similar narratives, solved via amortized variational inference for efficiency with millions of accounts. Validation on five CIOs from three countries on four topics shows the approach increases area under the precision-recall curve by 76 to 580 times over a naive baseline, 1.3 to 4.8 times over flags or narratives alone, and approaches supervised performance. The method is robust to small shares of messaging, weak inauthenticity markers, and CIOs comprising tiny fractions of the data.

What carries the argument

Bayesian group inference model solved with amortized variational inference to cluster accounts by shared characteristics and narratives.

If this is right

  • The method scales inference to millions of accounts without supervision.
  • It identifies novel operations without prior labels or examples.
  • Detection power stays high with only a small share of messages observed.
  • Performance approaches supervised benchmarks while remaining unsupervised.
  • The framework applies to many social-media platforms beyond the tested setting.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Platforms could use this to monitor emerging campaigns without first building labeled training sets.
  • The clustering step might combine with interaction networks to capture additional coordination signals.
  • Applying the same model to different languages or platforms would test its claimed generality.
  • Incremental updates to the variational inference could support tracking of evolving operations over time.

Load-bearing premise

Coordinated accounts will reliably share similar account-level characteristics and target similar narratives that the unsupervised Bayesian model can recover even in noisy settings with limited observations.

What would settle it

A dataset of known coordinated accounts that exhibit highly diverse characteristics and unrelated narratives, where the model's area under the precision-recall curve drops to the level of the naive baseline.

read the original abstract

This paper introduces and tests an unsupervised method for detecting novel coordinated inauthentic information operations (CIOs) in realistic settings. This method uses Bayesian inference to identify groups of accounts that share similar account-level characteristics and target similar narratives. We solve the inferential problem using amortized variational inference, allowing us to efficiently infer group identities for millions of accounts. We validate this method using a set of five CIOs from three countries discussing four topics on Twitter. Our unsupervised approach increases detection power (area under the precision-recall curve) relative to a naive baseline (by a factor of 76 to 580), relative to the use of simple flags or narratives on their own (by a factor of 1.3 to 4.8), and comes quite close to a supervised benchmark. Our method is robust to observing only a small share of messaging on the topic, having only weak markers of inauthenticity, and to the CIO accounts making up a tiny share of messages and accounts on the topic. Although we evaluate the results on Twitter, the method is general enough to be applied in many social-media settings.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript presents an unsupervised Bayesian generative model, solved via amortized variational inference, to detect coordinated inauthentic information operations (CIOs) by clustering accounts that share account-level features and target similar narratives. It reports AUPRC gains of 76-580x over a naive baseline, 1.3-4.8x over simple flags or narratives alone, and near-supervised performance when evaluated on five known CIOs spanning three countries and four topics on Twitter; the method is claimed robust to small CIO prevalence, weak markers, and limited observations.

Significance. If the unsupervised recovery of novel groups can be demonstrated on unlabeled data, the work would offer a scalable, label-free tool for identifying coordinated campaigns in social media, with the amortized VI approach providing a clear computational advantage for large datasets. The robustness claims, if substantiated, would strengthen applicability in realistic noisy settings.

major comments (2)
  1. [Abstract] Abstract and validation description: performance is measured on five pre-known CIOs whose accounts and narratives are already identified; this setup does not isolate whether the model recovers novel clusters when no ground-truth labels are supplied during inference, which is required to support the central claim of unsupervised detection of novel CIOs in the wild.
  2. [Abstract] Validation procedure (throughout): the abstract and manuscript provide no details on model specification, data processing steps, exact definition of positive examples for AUPRC, or error analysis, making it impossible to assess whether the reported gains (e.g., 76-580x) are driven by the unsupervised clustering or by post-hoc use of known labels.
minor comments (1)
  1. [Abstract] The abstract does not define the naive baseline or the precise account-level characteristics and narrative features used in the generative model.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We address each major comment below, clarifying that our inference procedure uses no labels and committing to expanded details on validation.

read point-by-point responses
  1. Referee: [Abstract] Abstract and validation description: performance is measured on five pre-known CIOs whose accounts and narratives are already identified; this setup does not isolate whether the model recovers novel clusters when no ground-truth labels are supplied during inference, which is required to support the central claim of unsupervised detection of novel CIOs in the wild.

    Authors: The inference is fully unsupervised: the Bayesian model and amortized variational inference receive no ground-truth labels, known CIO identities, or supervision of any kind during fitting or cluster assignment. The five known CIOs are used solely for post-inference evaluation of the discovered clusters via AUPRC. This follows standard practice for validating unsupervised methods when benchmark labels exist. The results show the model recovers the coordinated groups without being told their membership, supporting the unsupervised claim. We do not claim to have identified previously unknown CIOs in this study; external validation of novel detections is noted as future work. revision: no

  2. Referee: [Abstract] Validation procedure (throughout): the abstract and manuscript provide no details on model specification, data processing steps, exact definition of positive examples for AUPRC, or error analysis, making it impossible to assess whether the reported gains (e.g., 76-580x) are driven by the unsupervised clustering or by post-hoc use of known labels.

    Authors: The full manuscript details the generative model (account traits plus shared narratives), amortized VI solver, Twitter data collection and preprocessing, positive-example definition (accounts belonging to the five known CIOs), and error/robustness analyses. Ablations confirm gains arise from joint clustering rather than post-hoc labeling. We will revise the abstract to summarize these elements and add a dedicated validation subsection for clarity. revision: yes

Circularity Check

0 steps flagged

No significant circularity; validation uses external labels post-inference

full rationale

The paper describes an unsupervised Bayesian model with amortized variational inference to cluster accounts by shared features and narratives. It validates performance on five known CIOs using AUPRC against baselines, which is standard evaluation for unsupervised methods and does not reduce the claimed detection power to fitted inputs or self-definitions by construction. No equations, self-citations, or ansatzes are shown to make the central result equivalent to its inputs. The method remains self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review provides no information on specific free parameters, axioms, or invented entities used in the model.

pith-pipeline@v0.9.0 · 5723 in / 1120 out tokens · 22683 ms · 2026-05-24T04:22:50.892165+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

38 extracted references · 38 canonical work pages · 1 internal anchor

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 if FUNCTIO...

  2. [2]

    & Ferrara, E

    Addawood, A., Badawy, A., Lerman, K. & Ferrara, E. (2019). Linguistic cues to deception: Identifying political trolls on social media. In Proceedings of the international AAAI conference on web and social media , Volume 13 (pp.\ 15--25)

  3. [3]

    & Domke, J

    Agrawal, A. & Domke, J. (2021). Amortized variational inference for simple hierarchical models. Advances in Neural Information Processing Systems , 34 , 21388--21399

  4. [4]

    Alam, F., Cresci, S., Chakraborty, T., Silvestri, F., Dimitrov, D., Martino, G. D. S., Shaar, S., Firooz, H. & Nakov, P. (2021). A survey on multimodal disinformation detection. arXiv preprint arXiv:2103.12541

  5. [5]

    N., Buntain, C

    Alizadeh, M., Shapiro, J. N., Buntain, C. & Tucker, J. A. (2020). Content-based features predict social media influence operations. Science Advances , 6(30) , eabb5824

  6. [6]

    & Widmer, P

    Ash, E., Gauthier, G. & Widmer, P. (2024). Relatio: Text semantics capture political and economic narratives. Political Analysis , 32(1) , 115--132

  7. [7]

    Bishop, C. M. (2013). Model-based machine learning. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences , 371(1984) , 20120222

  8. [8]

    M., Kucukelbir, A

    Blei, D. M., Kucukelbir, A. & McAuliffe, J. D. (2017). Variational inference: A review for statisticians. Journal of the American statistical Association , 112(518) , 859--877

  9. [9]

    Bushwick, S. (2022). Russia's information war is being waged on social media platforms. Scientific American

  10. [10]

    & Palow, C

    Cao, Q., Yang, X., Yu, J. & Palow, C. (2014). Uncovering large groups of active malicious accounts in online social networks. In Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security (pp.\ 477--488)

  11. [11]

    Dal Cin , S., Zanna, M. P. & Fong, G. T. (2004). Narrative persuasion and overcoming resistance. In E. S. Knowles & J. A. Linn (Eds.), Resistance and persuasion (pp.\ 175--191). Lawrence Erlbaum Associates

  12. [12]

    Donath, J. S. et al. (1999). Identity and deception in the virtual community. Communities in cyberspace , 1996 , 29--59

  13. [13]

    Edgitt, S. (2017). Opening remarks from twitter general counsel. In US Senate Committee on the Judiciary, Subcommittee on Crime and Terrorism

  14. [14]

    Goldstein, J. A. & Grossman, S. (2021). How disinformation evolved in 2020. Brookings

  15. [15]

    & Chakraborty, T

    Gupta, S., Kumaraguru, P. & Chakraborty, T. (2019). Malreg: Detecting and analyzing malicious retweeter groups. In Proceedings of the ACM India Joint International Conference on Data Science and Management of Data (pp.\ 61--69)

  16. [16]

    & Gilbert, E

    Im, J., Chandrasekharan, E., Sargent, J., Lighthammer, P., Denby, T., Bhargava, A., Hemphill, L., Jurgens, D. & Gilbert, E. (2020). Still out there: Modeling and identifying russian troll accounts on twitter. In 12th ACM Conference on Web Science (pp.\ 1--10)

  17. [17]

    & Szegedy, C

    Ioffe, S. & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning (pp.\ 448--456)

  18. [18]

    & Ferrara, E

    Luceri, L., Giordano, S. & Ferrara, E. (2020). Detecting troll behavior via inverse reinforcement learning: A case study of russian trolls in the 2016 us election. In Proceedings of the International AAAI Conference on Web and Social Media , Volume 14 (pp.\ 417--427)

  19. [19]

    & Royesh, A

    Miller, C., Kheradpir, T., DiResta, R. & Royesh, A. (2020). Hacked and hoaxed: Tactics of an iran-linked operation to influence black lives matter narratives on twitter. Available at: https://cyber.fsi.stanford.edu/io/news/twitter-takedown-iran-october-2020 (Archived: https://archive.ph/ikDJh)

  20. [20]

    T., Flammini, A

    Pacheco, D., Hui, P.-M., Torres-Lugo, C., Truong, B. T., Flammini, A. & Menczer, F. (2021). Uncovering coordinated networks on social media: methods and case studies. In Proceedings of the international AAAI conference on web and social media , Volume 15 (pp.\ 455--466)

  21. [21]

    & Balluff, P

    Righetti, N. & Balluff, P. (2023). CooRTweet: Coordinated Networks Detection on Social Media . R package version 1.3.3

  22. [22]

    & Liu, Y

    Ruchansky, N., Seo, S. & Liu, Y. (2017). Csi: A hybrid deep model for fake news detection. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management (pp.\ 797--806)

  23. [23]

    & Liu, Y

    Sharma, K., Zhang, Y., Ferrara, E. & Liu, Y. (2020). Identifying coordinated accounts on social media through hidden influence and group behaviours. arXiv preprint arXiv:2008.11308

  24. [24]

    & Liu, Y

    Sharma, K., Zhang, Y. & Liu, Y. (2021). Covid-19 vaccines: characterizing misinformation campaigns and vaccine hesitancy on twitter. arXiv preprint arXiv:2106.08423

  25. [25]

    & Liu, H

    Shu, K., Sliva, A., Wang, S., Tang, J. & Liu, H. (2017). Fake news detection on social media: A data mining perspective. ACM SIGKDD explorations newsletter , 19(1) , 22--36

  26. [26]

    T., Kao, E

    Smith, S. T., Kao, E. K., Mackin, E. D., Shah, D. C., Simek, O. & Rubin, D. B. (2021). Automatic detection of influential actors in disinformation networks. Proceedings of the National Academy of Sciences , 118(4)

  27. [27]

    Autoencoding Variational Inference For Topic Models

    Srivastava, A. & Sutton, C. (2017). Autoencoding variational inference for topic models. arXiv preprint arXiv:1703.01488

  28. [28]

    & Salakhutdinov, R

    Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I. & Salakhutdinov, R. (2014). Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research , 15(1) , 1929--1958

  29. [29]

    & Zhao, B

    Wang, G., Zhang, X., Tang, S., Zheng, H. & Zhao, B. Y. (2016). Unsupervised clickstream clustering for user behavior analysis. In Proceedings of the 2016 CHI conference on human factors in computing systems (pp.\ 225--236)

  30. [30]

    & Gao, J

    Wang, Y., Yang, W., Ma, F., Xu, J., Zhong, B., Deng, Q. & Gao, J. (2020). Weak supervision for fake news detection via reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence , Volume 34(01) (pp.\ 516--523)

  31. [31]

    & Liu, Y

    Yu, R., He, X. & Liu, Y. (2015). Glad: group anomaly detection in social media analysis. ACM Transactions on Knowledge Discovery from Data (TKDD) , 10(2) , 1--22

  32. [32]

    u tepage, J., Kjellstr \

    Zhang, C., B \"u tepage, J., Kjellstr \"o m, H. & Mandt, S. (2018). Advances in variational inference. IEEE transactions on pattern analysis and machine intelligence , 41(8) , 2008--2026

  33. [33]

    & Liu, Y

    Zhang, Y., Sharma, K. & Liu, Y. (2021). Vigdet: Knowledge informed neural temporal point process for coordination detection on social media. Advances in Neural Information Processing Systems , 34

  34. [34]

    & Zafarani, R

    Zhou, X. & Zafarani, R. (2020). A survey of fake news: Fundamental theories, detection methods, and opportunities. ACM Computing Surveys (CSUR) , 53(5) , 1--40

  35. [35]

    , " * write output.state after.block = add.period write newline

    ENTRY address author booktitle chapter doi edition editor eid howpublished institution isbn journal key month note number organization pages publisher school series title type url volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid...

  36. [36]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize ":" * " " *...

  37. [37]

    , " * write output.state after.block = add.period write newline

    ENTRY address archive author booktitle chapter doi edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type url volume year label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid.sentence := #2 'af...

  38. [38]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...