REVIEW 5 major objections 5 minor 28 references
Propaganda and Information Dissemination in the Russo-Ukrainian War: Natural Language Processing of Russian and Western Twitter Narratives
T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read This paper claims that Russian propaganda and Western trusted accounts on Twitter pursued measurably different narrative strategies during the first months of the Ukraine war: propaganda leaned on fear and disinformation, while trusted…
desk verdict A modest descriptive NLP case study with a real dataset, but the central comparison is partly circular because the two account groups are defined by the very behaviors the paper claims to measure. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The analysis is built on a two-class dataset of tweets from accounts manually classified as Russian propaganda or Western trusted, and it processes that dataset with Latent Dirichlet Allocation (LDA), a probabilistic topic model that groups co-occurring words into themes, and a Gaussian Naive Bayes (GNB) classifier, which assigns sentiment labels. Human-in-the-loop review and t-SNE clustering of account-level feature vectors are used to interpret the results, with LDA supplying thematic fingerprints, GNB supplying sentiment tone, and clustering supplying the suspected coordination signal.
What would settle it
Reclassify the same 40,000 tweets using independently validated account labels, such as expert adjudication or a pre-registered list of state-affiliated outlets, and rerun the LDA and GNB pipeline; if the sentiment and topic distributions no longer separate into the two described patterns, the central claim fails.
Extended reading notes
Core claim
The paper reports that, in its dataset of 40,000 tweets, Russian propaganda accounts and Western trusted accounts diverged systematically in both sentiment and content. Russian-class tweets were predominantly negative, emotionally charged, and drew on conspiracy motifs, historical figures, and derogatory language, while Western-class tweets emphasized factual reporting, geopolitical themes, and humanitarian support. The paper interprets this divergence as evidence of distinct information strategies rather than random variation, and it notes that clustering of account-level behavior suggests possible coordinated amplification among Russian accounts, although network-level coordination was not directly analyzed.
Load-bearing premise
Everything depends on the initial sorting of accounts into 'propaganda' and 'trusted', which the paper describes qualitatively without formal criteria; if that sorting is biased, the measured differences in tone and topics would reflect the sorting rather than real strategies.
Editorial extensions
If this is right
- If the claim holds, audiences of Russian propaganda and Western trusted accounts received systematically different portraits of the same war, one fear-driven and one fact-driven.
- The LDA topic clusters could serve as measurable narrative fingerprints for future monitoring of influence operations on social media.
- The GNB classifier's 78% accuracy implies that automated labeling of propaganda versus trusted content is feasible but error-prone, so human review remains necessary.
- The observed behavioral clusters among Russian accounts, if real, point to possible coordinated amplification networks that merit direct network analysis.
- The method demonstrated on this conflict could be transferred to other geopolitical crises to compare whether similar narrative asymmetries appear.
Reading between the lines
- If the pattern is real, an automated early-warning system could flag coordinated emotionally charged messaging by comparing an account's topic-sentiment profile against the two clusters, though this extension is not tested in the paper.
- The appearance of German-language terms in Russian propaganda suggests targeted cross-border messaging; a testable follow-up would check whether locally translated propaganda outperforms generic Russian-to-Western translation.
- Because the data stops in May 2022, the observed strategies may be time-bound; running the same pipeline on later periods would test whether Russian and Western narratives converge or shift as the war evolves.
- The clustering hints at coordination, but explicit analysis of retweet and reply graphs would be needed to confirm that accounts with similar behavior actually form coordinated networks rather than merely sharing a style.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This manuscript analyzes 40,000 tweets collected from February to May 2022, comparing Russian propaganda accounts with Western trusted accounts on Twitter/X. After preprocessing and BoW/TF-IDF feature extraction, the authors apply LDA topic modeling and a Gaussian Naive Bayes sentiment classifier, followed by t-SNE clustering. They report that Russian propaganda tweets are characterized by emotionally charged language, conspiracy theories, disinformation, and fear-evoking narratives, while Western accounts focus on factual reporting and humanitarian concerns. The paper also suggests that clustering reveals coordinated behavior among Russian accounts. The research question is timely, and the authors frame their work as a contribution to understanding information warfare through NLP.
Significance. If the findings were fully supported, the paper would offer a useful early-window case study of information-warfare strategies on Twitter during a major conflict, and it would demonstrate an accessible NLP pipeline combining sentiment, topic, and clustering methods. The authors deserve credit for addressing a consequential research question, for collecting a sizable tweet corpus from the first months of the invasion, and for being transparent that the 78% model accuracy implies a large number of misclassifications. However, the current evidence is not sufficient for the central claim as stated: part of the conclusion is built into the definition of the two account groups, and the remaining non-tautological comparisons lack the methodological detail needed to rule out annotator or interpretation bias. The non-tautological components could be made credible with blind annotation, inter-rater reliability, statistical testing, and quantitative topic validation.
major comments (5)
- [Section 1, account classification] Part of the headline result is true by construction. The authors state that propaganda accounts were classified based on 'association with state-backed media and a history of spreading disinformation and misinformation,' while trusted accounts were selected for 'reputation in factual reporting.' The abstract's claim that propaganda accounts 'employ... disinformation' while trusted accounts 'focus on factual reporting' is therefore not an empirical discovery on those specific dimensions. I do not regard this as a fatal flaw, because sentiment and topic differences are not definitional, but the central claim must be reframed, and the non-definitional dimensions need to be validated independently.
- [Section 1.2 and Figure 5] The sentiment analysis lacks the methodological detail required to support the reported contrast. It is not stated how many tweets were manually labeled, whether the annotator was blind to account class, whether multiple annotators were used, or what inter-rater reliability was obtained. If annotators knew the class labels, the sentiment comparison could encode their prior expectations. In addition, Figure 5 reports '78% accuracy' without specifying the prediction target (e.g., three-class sentiment versus binary class), the class-wise precision/recall, or the test split. These details are essential for interpreting the sentiment results.
- [Section 1.3 and Section 2.1] The LDA topic selection is under-specified. The text says the optimal number of topics was found to be 'between 5 and 10,' but no final number of topics, coherence score, held-out likelihood, or top-word lists are provided. Because the thematic interpretation is central to the paper's claims about distinct narrative strategies, the reader cannot assess whether the reported themes are stable or selectively chosen. Please report a quantitative model-selection criterion and show the top words per topic for both classes.
- [Section 2.2] The paper asserts 'substantial differences between Russian and Western tweets' in sentiment, but no statistical tests, confidence intervals, or effect sizes are reported. With n=40,000, even numerically small differences could be statistically significant, so the reader needs to see the magnitude of the differences and their uncertainty. Since the conclusions are explicitly comparative, an inferential analysis is necessary to distinguish meaningful contrasts from noise.
- [Section 1, data availability] The manuscript states that the dataset is openly available on Kaggle, but no link, dataset identifier, or access instructions are given, and no analysis code or preprocessing scripts are included. This prevents replication of the reported pipeline. Given that other load-bearing components (account classification, manual labeling) are not externally validated, providing the dataset link and code is especially important for the manuscript's credibility.
minor comments (5)
- [Section 1.4] The Elbow method is mentioned for choosing the number of clusters, but no elbow plot or cluster-validation measure is shown; please add the supporting plot or an alternative validation metric.
- [Figures 2 and 3] The figure captions and in-text references for the common-word plots appear to be in reverse order; the text refers to Figure 2 for Russian words and Figure 3 for Western words, but the placement of the captions is confusing. Please reorder and add axis labels.
- [Section 1.2] The sentence 'we chose these for its effectiveness in text classification tasks' has a subject-verb agreement error and should be rewritten, and the target of the reported accuracy in Figure 5 should be stated in the caption.
- [Table 1] It would be useful to clarify whether the examples are original tweets or retweets, and whether any personally identifying information was considered in the presentation of user-associated content.
- [Conclusion] The acknowledgments of limitations in the Conclusion are welcome and could be expanded into a dedicated limitations subsection covering the temporal range, the binary account classification, and the known misclassification rate.
Circularity Check
Central claim is partially circular: propaganda and trusted labels are defined by disinformation history and factual-reporting reputation, and the paper then reports those very behaviors as findings.
-
self definitional
[Section 1 (account classification), with Abstract, Section 2.2, and Section 3 (findings).]
"We classified propaganda accounts as those based on their association with state-backed media and a history of spreading disinformation and misinformation, while trusted accounts were selected for their reputation in factual reporting, often verified through independent sources and mainstream media. ... Propaganda accounts frequently employ emotionally charged language and disinformation to evoke fear and distrust, whereas other accounts, primarily Western tend to focus on factual reporting and humanitarian aspects of the conflict."
The propaganda and trusted classes are constructed from the very behaviors that the paper later reports as empirical findings. The propaganda label includes 'a history of spreading disinformation and misinformation,' and the trusted label includes 'reputation in factual reporting.' The abstract and conclusion then state, as results, that propaganda accounts use disinformation and that Western or trusted accounts focus on factual reporting. On those dimensions the comparison is true by definition rather than by measurement, so the headline claim reduces to its own sampling criteria. The sentiment, topic, and humanitarian-aspect findings are not rendered circular by this step, but they are bundled with definitional claims in the central statement.
full rationale
The paper contains no self-citations, no imported uniqueness theorem, and no ansatz smuggled through prior work, so those circularity patterns are absent. The one concrete reduction is the account-classification step: propaganda is defined by a history of disinformation and trusted accounts by factual-reporting reputation, and the paper's abstract and conclusion report exactly these attributes as discovered differences. This makes part of the central claim true by construction. The remaining evidence (sentiment distributions, LDA topics, example tweets, clustering) is not itself circular, though the sentiment analysis would be stronger with blind annotation and inter-rater reliability, which the paper does not report; that is a validity limitation rather than a circular reduction. Overall, the central claim is partially circular but retains independent empirical content in the sentiment and topic analyses, warranting a score of 6.
Assumptions & free parameters
free parameters (3)
- Number of LDA topics =
not reported, between 5 and 10
- Number of t-SNE clusters =
not reported
- Manual sentiment labels =
subjective labels (positive, negative, neutral)
assumptions (6)
- standard math Latent Dirichlet Allocation assumes documents are mixtures of topics and words are generated from topic distributions.
- standard math Gaussian Naive Bayes assumes features are independent and normally distributed.
- standard math t-SNE approximately preserves local structure in high-dimensional data.
- domain assumption The classification of accounts as propaganda or trusted is correct and meaningful.
- domain assumption Manual sentiment labels are accurate and unbiased.
- domain assumption The tweets from Feb-May 2022 are representative of the broader information war.
Cite this review
Pith. "Pith review of Propaganda and Information Dissemination in the Russo-Ukrainian War: Natural Language Processing of Russian and Western Twitter Narratives." pith.science (2026). https://pith.science/paper/AGWGZEYD
@misc{pith2026250601807,
author = {Pith},
title = {Pith review of: Propaganda and Information Dissemination in the Russo-Ukrainian War: Natural Language Processing of Russian and Western Twitter Narratives},
year = {2026},
howpublished = {\url{https://pith.science/paper/AGWGZEYD}},
note = {Machine review of arXiv:2506.01807}
}
read the original abstract
The conflict in Ukraine has been not only characterised by military engagement but also by a significant information war, with social media platforms like X, formerly known as Twitter playing an important role in shaping public perception. This article provides an analysis of tweets from propaganda accounts and trusted accounts collected from the onset of the war, February 2022 until the middle of May 2022 with n=40,000 total tweets. We utilise natural language processing and machine learning algorithms to assess the sentiment and identify key themes, topics and narratives across the dataset with human-in-the-loop (HITL) analysis throughout. Our findings indicate distinct strategies in how information is created, spread, and targeted at different audiences by both sides. Propaganda accounts frequently employ emotionally charged language and disinformation to evoke fear and distrust, whereas other accounts, primarily Western tend to focus on factual reporting and humanitarian aspects of the conflict. Clustering analysis reveals groups of accounts with similar behaviours, which we suspect indicates the presence of coordinated efforts. This research attempts to contribute to our understanding of the dynamics of information warfare and offers techniques for future studies on social media influence in military conflicts.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
Samah M. Alzanin, Aqil M. Azmi, and Hatim A. Aboalsamh. Short text classification for arabic social media tweets. Journal of King Saud University - Computer and Information Sciences, 34(9):6595–6604, October 2022. Received 26 September 2021, Revised 26 February 2022, Accepted 21 March 2022, Available online 23 March 2022, Version of Record 30 September 20...
-
[2]
A multi-faceted nlp analysis of misinformation spreaders in twitter
Dimosthenis Antypas, Alun Preece, and Jose Camacho-Collados. A multi-faceted nlp analysis of misinformation spreaders in twitter. In Proceedings of the 14th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pages 71–83, Cardiff, United Kingdom, August 2024. Association for Computational Linguistics, Cardiff NLP, Sch...
work page 2024
-
[3]
Abayomi Arowosegbe and Tope Oyelade. Application of natural language processing (nlp) in detecting and preventing suicide ideation: A systematic review. International Journal of Environmental Research and Public Health, 20(2):1514, January 2023. doi:10.3390/ijerph20021514
-
[4]
Hybrid warfare and disinformation: A ukraine war perspective
Sascha-Dominik Dov Bachmann, Dries Putter, and Guy Duczynski. Hybrid warfare and disinformation: A ukraine war perspective. Global Policy, August 2023. First published: 25 August 2023. doi:10.1111/1758-5899. 13257
-
[5]
Schmitt, Judit Szakács, and Erik Uszkiewicz
Judit Bayer, Bernd Holznagel, Katarzyna Lubianiec, Adela Pintea, Josephine B. Schmitt, Judit Szakács, and Erik Uszkiewicz. Disinformation and propaganda: Impact on the functioning of the rule of law and democratic processes in the eu and its member states - 2021 update. Report PE 653.633, European Parliament, April 2021. Policy Department for External Rel...
work page 2021
-
[6]
BBC News. Crimea profile, 2018. January 17, 2018, sec. Europe. URL: https://www.bbc.com/news/ world-europe-18287223
work page 2018
-
[7]
David M. Blei, Andrew Y . Ng, and Michael I. Jordan. Latent dirichlet allocation.Journal of Machine Learning Research, 3:993–1022, January 2003. Submitted 2/02; Published 1/03. URL: http://www.jmlr.org/papers/ volume3/blei03a/blei03a.pdf
work page 2003
-
[8]
Analyzing the worldwide perception of the russia-ukraine conflict through twitter
Bernardo Breve, Loredana Caruccio, Stefano Cirillo, Vincenzo Deufemia, and Giuseppe Polese. Analyzing the worldwide perception of the russia-ukraine conflict through twitter. Journal of Big Data, 11:Article number: 76, 2024
work page 2024
Show all 28 references
-
[9]
Conflict in ukraine’s donbas: A visual explainer, 2021
Crisis Group. Conflict in ukraine’s donbas: A visual explainer, 2021. June 29, 2021. URL: https://www. crisisgroup.org/content/conflict-ukraines-donbas-visual-explainer
2021
-
[10]
A topic modeling comparison between lda, nmf, top2vec, and bertopic to demystify twitter posts
Roman Egger and Joanne Yu. A topic modeling comparison between lda, nmf, top2vec, and bertopic to demystify twitter posts. Frontiers in Sociology, 7, May 2022. This article is part of the Research Topic Big Data and Machine Learning in Sociology. doi:10.3389/fsoc.2022.886498
2022
-
[11]
Disguised propaganda from digital to social media
Johan Farkas and Christina Neumayer. Disguised propaganda from digital to social media. In Jeremy Hunsinger, Lisbeth Klastrup, and Matthew Allen, editors, Second International Handbook of Internet Research. Springer, Dordrecht, 2018. doi:10.1007/978-94-024-1202-4_33-1 . 7 A PR...
2018 doi
-
[12]
Disinformation as political communication
Deen Freelon and Chris Wells. Disinformation as political communication. Political Communication, 37(2):145– 146, 2020
2020
-
[13]
Machine learning and natural language processing of social media data for event detection in smart cities
Andrei Hodorog, Ioan Petri, and Yacine Rezgui. Machine learning and natural language processing of social media data for event detection in smart cities. Sustainable Cities and Society, 85:104026, 2022. Received 10 February 2022, Revised 16 May 2022, Accepted 22 June 2022, Ava...
2022
-
[14]
Huang, G
D. Huang, G. Guan, J. Zhou, et al. Network-based naive bayes model for social network. Science China Mathematics, 61:627–640, 2018. doi:10.1007/s11425-017-9209-6
2018 doi
-
[15]
fake news
Dean Jackson. Distinguishing disinformation from propaganda, misinformation, and “fake news”. Technical report, International Forum for Democratic Studies, 2018. Prepared by the Interna- tional Forum for Democratic Studies. URL: https://www.ned.org/wp-content/uploads/2018/06/ ...
2018
-
[16]
Propaganda and Persuasion
Garth S Jowett and Victoria O’Donnell. Propaganda and Persuasion. Sage Publications, 2019
2019
-
[17]
D. H. Lan and T. M. Tung. Exploring fake news awareness and trust in the age of social media among university student tiktok users. Cogent Social Sciences, 10(1), 2024. doi:10.1080/23311886.2024.2302216
2024
-
[18]
Organizational propaganda on the internet: A systematic review
Irina Lock and Ramona Ludolph. Organizational propaganda on the internet: A systematic review. Inter- national Journal of Communication , 9(1), 2019. First published online September 6, 2019. doi:10.1177/ 2046147X19870844
2019
-
[19]
Ukraine: Conflict at the crossroads of europe and russia, 2023
Jonathan Masters. Ukraine: Conflict at the crossroads of europe and russia, 2023. Last updated February 14, 2023. URL: https://www.cfr.org/backgrounder/ukraine-conflict-crossroads-europe-and-russia
2023
-
[20]
Social media’s role in ‘hybrid strategies’
Thomas Elkjer Nissen. Social media’s role in ‘hybrid strategies’. Technical report, NATO Strategic Communica- tions Centre of Excellence, 2023. Prepared by the NATO Strategic Communications Centre of Excellence
2023
-
[21]
The role of social media in contemporary conflicts
Maria Ojala. The role of social media in contemporary conflicts. Media, War & Conflict, 12(4):433–448, 2019
2019
-
[22]
New Narratives: Digital Diplomacy and the Evolution of Statecraft
James Pamment. New Narratives: Digital Diplomacy and the Evolution of Statecraft. Routledge, 2020
2020
-
[23]
Information warfare in russia’s war in ukraine, 2022
Christian Perez. Information warfare in russia’s war in ukraine, 2022. August 22, 2022. URL: https:// foreignpolicy.com/2022/08/22/information-warfare-in-russias-war-in-ukraine/
2022
-
[24]
Commanding the trend: Social media as information warfare
Jarred Prier. Commanding the trend: Social media as information warfare. Strategic Studies Quarterly, 11(4):50– 85, 2017. Winter 2017. URL: https://www.jstor.org/stable/26271634
2017
-
[25]
Deplatforming: Following extreme internet celebrities to telegram and alternative social media
Richard Rogers. Deplatforming: Following extreme internet celebrities to telegram and alternative social media. European Journal of Communication, 36(4):356–372, 2021
2021
-
[26]
Computational propaganda: Political parties, politicians, and political manipulation on social media
Samuel C Woolley and Philip N Howard. Computational propaganda: Political parties, politicians, and political manipulation on social media. Oxford University Press, 2017
2017
-
[27]
Understanding online disinformation on social media: An empirical study based on twitter during the covid-19 pandemic
Yang Zhou, Zhiqi Zhang, Yingjie Zhao, and Wei Wang. Understanding online disinformation on social media: An empirical study based on twitter during the covid-19 pandemic. Journal of Information Science, 48(2):253–265, 2022
2022
-
[28]
Political effects of the internet and social media
Ekaterina Zhuravskaya, Maria Petrova, and Ruben Enikolopov. Political effects of the internet and social media. Annual Review of Economics, 2024. 8
2024
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.