Pith. sign in

REVIEW 5 major objections 5 minor 28 references

Propaganda and Information Dissemination in the Russo-Ukrainian War: Natural Language Processing of Russian and Western Twitter Narratives

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read This paper claims that Russian propaganda and Western trusted accounts on Twitter pursued measurably different narrative strategies during the first months of the Ukraine war: propaganda leaned on fear and disinformation, while trusted…

desk verdict A modest descriptive NLP case study with a real dataset, but the central comparison is partly circular because the two account groups are defined by the very behaviors the paper claims to measure. read the letter →

arxiv 2506.01807 v2 pith:AGWGZEYD submitted 2025-06-02 cs.CL

classification cs.CL
keywords propagandaRusso-UkrainianwarTwitter/XsentimentanalysistopicmodellingLatentDirichletAllocationGaussianNaiveBayesdisinformation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that the Russian and Western sides of the early-war Twitter conversation did not merely disagree but used different machinery of persuasion. Russian propaganda accounts are said to rely on negative emotion, conspiracy themes, historical analogies, and trolling, whereas Western trusted accounts are said to focus on factual reporting, geopolitical framing, and humanitarian support. The stakes are practical: if true, platforms like Twitter/X can be read as a measurable battlefield where influence operations show up in aggregate text patterns. The analysis works from 40,000 tweets collected from the start of the invasion in February 2022 to mid-May 2022, using topic modeling, sentiment classification, and clustering with human review.

What carries the argument

The analysis is built on a two-class dataset of tweets from accounts manually classified as Russian propaganda or Western trusted, and it processes that dataset with Latent Dirichlet Allocation (LDA), a probabilistic topic model that groups co-occurring words into themes, and a Gaussian Naive Bayes (GNB) classifier, which assigns sentiment labels. Human-in-the-loop review and t-SNE clustering of account-level feature vectors are used to interpret the results, with LDA supplying thematic fingerprints, GNB supplying sentiment tone, and clustering supplying the suspected coordination signal.

What would settle it

Reclassify the same 40,000 tweets using independently validated account labels, such as expert adjudication or a pre-registered list of state-affiliated outlets, and rerun the LDA and GNB pipeline; if the sentiment and topic distributions no longer separate into the two described patterns, the central claim fails.

Watch

Extended reading notes

Core claim

The paper reports that, in its dataset of 40,000 tweets, Russian propaganda accounts and Western trusted accounts diverged systematically in both sentiment and content. Russian-class tweets were predominantly negative, emotionally charged, and drew on conspiracy motifs, historical figures, and derogatory language, while Western-class tweets emphasized factual reporting, geopolitical themes, and humanitarian support. The paper interprets this divergence as evidence of distinct information strategies rather than random variation, and it notes that clustering of account-level behavior suggests possible coordinated amplification among Russian accounts, although network-level coordination was not directly analyzed.

Load-bearing premise

Everything depends on the initial sorting of accounts into 'propaganda' and 'trusted', which the paper describes qualitatively without formal criteria; if that sorting is biased, the measured differences in tone and topics would reflect the sorting rather than real strategies.

Editorial extensions

If this is right

  • If the claim holds, audiences of Russian propaganda and Western trusted accounts received systematically different portraits of the same war, one fear-driven and one fact-driven.
  • The LDA topic clusters could serve as measurable narrative fingerprints for future monitoring of influence operations on social media.
  • The GNB classifier's 78% accuracy implies that automated labeling of propaganda versus trusted content is feasible but error-prone, so human review remains necessary.
  • The observed behavioral clusters among Russian accounts, if real, point to possible coordinated amplification networks that merit direct network analysis.
  • The method demonstrated on this conflict could be transferred to other geopolitical crises to compare whether similar narrative asymmetries appear.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the pattern is real, an automated early-warning system could flag coordinated emotionally charged messaging by comparing an account's topic-sentiment profile against the two clusters, though this extension is not tested in the paper.
  • The appearance of German-language terms in Russian propaganda suggests targeted cross-border messaging; a testable follow-up would check whether locally translated propaganda outperforms generic Russian-to-Western translation.
  • Because the data stops in May 2022, the observed strategies may be time-bound; running the same pipeline on later periods would test whether Russian and Western narratives converge or shift as the war evolves.
  • The clustering hints at coordination, but explicit analysis of retweet and reply graphs would be needed to confirm that accounts with similar behavior actually form coordinated networks rather than merely sharing a style.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. This manuscript analyzes 40,000 tweets collected from February to May 2022, comparing Russian propaganda accounts with Western trusted accounts on Twitter/X. After preprocessing and BoW/TF-IDF feature extraction, the authors apply LDA topic modeling and a Gaussian Naive Bayes sentiment classifier, followed by t-SNE clustering. They report that Russian propaganda tweets are characterized by emotionally charged language, conspiracy theories, disinformation, and fear-evoking narratives, while Western accounts focus on factual reporting and humanitarian concerns. The paper also suggests that clustering reveals coordinated behavior among Russian accounts. The research question is timely, and the authors frame their work as a contribution to understanding information warfare through NLP.

Significance. If the findings were fully supported, the paper would offer a useful early-window case study of information-warfare strategies on Twitter during a major conflict, and it would demonstrate an accessible NLP pipeline combining sentiment, topic, and clustering methods. The authors deserve credit for addressing a consequential research question, for collecting a sizable tweet corpus from the first months of the invasion, and for being transparent that the 78% model accuracy implies a large number of misclassifications. However, the current evidence is not sufficient for the central claim as stated: part of the conclusion is built into the definition of the two account groups, and the remaining non-tautological comparisons lack the methodological detail needed to rule out annotator or interpretation bias. The non-tautological components could be made credible with blind annotation, inter-rater reliability, statistical testing, and quantitative topic validation.

major comments (5)
  1. [Section 1, account classification] Part of the headline result is true by construction. The authors state that propaganda accounts were classified based on 'association with state-backed media and a history of spreading disinformation and misinformation,' while trusted accounts were selected for 'reputation in factual reporting.' The abstract's claim that propaganda accounts 'employ... disinformation' while trusted accounts 'focus on factual reporting' is therefore not an empirical discovery on those specific dimensions. I do not regard this as a fatal flaw, because sentiment and topic differences are not definitional, but the central claim must be reframed, and the non-definitional dimensions need to be validated independently.
  2. [Section 1.2 and Figure 5] The sentiment analysis lacks the methodological detail required to support the reported contrast. It is not stated how many tweets were manually labeled, whether the annotator was blind to account class, whether multiple annotators were used, or what inter-rater reliability was obtained. If annotators knew the class labels, the sentiment comparison could encode their prior expectations. In addition, Figure 5 reports '78% accuracy' without specifying the prediction target (e.g., three-class sentiment versus binary class), the class-wise precision/recall, or the test split. These details are essential for interpreting the sentiment results.
  3. [Section 1.3 and Section 2.1] The LDA topic selection is under-specified. The text says the optimal number of topics was found to be 'between 5 and 10,' but no final number of topics, coherence score, held-out likelihood, or top-word lists are provided. Because the thematic interpretation is central to the paper's claims about distinct narrative strategies, the reader cannot assess whether the reported themes are stable or selectively chosen. Please report a quantitative model-selection criterion and show the top words per topic for both classes.
  4. [Section 2.2] The paper asserts 'substantial differences between Russian and Western tweets' in sentiment, but no statistical tests, confidence intervals, or effect sizes are reported. With n=40,000, even numerically small differences could be statistically significant, so the reader needs to see the magnitude of the differences and their uncertainty. Since the conclusions are explicitly comparative, an inferential analysis is necessary to distinguish meaningful contrasts from noise.
  5. [Section 1, data availability] The manuscript states that the dataset is openly available on Kaggle, but no link, dataset identifier, or access instructions are given, and no analysis code or preprocessing scripts are included. This prevents replication of the reported pipeline. Given that other load-bearing components (account classification, manual labeling) are not externally validated, providing the dataset link and code is especially important for the manuscript's credibility.
minor comments (5)
  1. [Section 1.4] The Elbow method is mentioned for choosing the number of clusters, but no elbow plot or cluster-validation measure is shown; please add the supporting plot or an alternative validation metric.
  2. [Figures 2 and 3] The figure captions and in-text references for the common-word plots appear to be in reverse order; the text refers to Figure 2 for Russian words and Figure 3 for Western words, but the placement of the captions is confusing. Please reorder and add axis labels.
  3. [Section 1.2] The sentence 'we chose these for its effectiveness in text classification tasks' has a subject-verb agreement error and should be rewritten, and the target of the reported accuracy in Figure 5 should be stated in the caption.
  4. [Table 1] It would be useful to clarify whether the examples are original tweets or retweets, and whether any personally identifying information was considered in the presentation of user-associated content.
  5. [Conclusion] The acknowledgments of limitations in the Conclusion are welcome and could be expanded into a dedicated limitations subsection covering the temporal range, the binary account classification, and the known misclassification rate.

Circularity Check

1 steps flagged · score 6.0 of 10

Central claim is partially circular: propaganda and trusted labels are defined by disinformation history and factual-reporting reputation, and the paper then reports those very behaviors as findings.

  1. self definitional [Section 1 (account classification), with Abstract, Section 2.2, and Section 3 (findings).]
    "We classified propaganda accounts as those based on their association with state-backed media and a history of spreading disinformation and misinformation, while trusted accounts were selected for their reputation in factual reporting, often verified through independent sources and mainstream media. ... Propaganda accounts frequently employ emotionally charged language and disinformation to evoke fear and distrust, whereas other accounts, primarily Western tend to focus on factual reporting and humanitarian aspects of the conflict."

    The propaganda and trusted classes are constructed from the very behaviors that the paper later reports as empirical findings. The propaganda label includes 'a history of spreading disinformation and misinformation,' and the trusted label includes 'reputation in factual reporting.' The abstract and conclusion then state, as results, that propaganda accounts use disinformation and that Western or trusted accounts focus on factual reporting. On those dimensions the comparison is true by definition rather than by measurement, so the headline claim reduces to its own sampling criteria. The sentiment, topic, and humanitarian-aspect findings are not rendered circular by this step, but they are bundled with definitional claims in the central statement.

full rationale

The paper contains no self-citations, no imported uniqueness theorem, and no ansatz smuggled through prior work, so those circularity patterns are absent. The one concrete reduction is the account-classification step: propaganda is defined by a history of disinformation and trusted accounts by factual-reporting reputation, and the paper's abstract and conclusion report exactly these attributes as discovered differences. This makes part of the central claim true by construction. The remaining evidence (sentiment distributions, LDA topics, example tweets, clustering) is not itself circular, though the sentiment analysis would be stronger with blind annotation and inter-rater reliability, which the paper does not report; that is a validity limitation rather than a circular reduction. Overall, the central claim is partially circular but retains independent empirical content in the sentiment and topic analyses, warranting a score of 6.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The analysis depends on several hand-chosen settings (topic count, cluster count) and subjective labels. The account classification is the most consequential assumption, since all comparative findings follow from it.

free parameters (3)
  • Number of LDA topics = not reported, between 5 and 10
    Chosen by iterative testing rather than a model selection criterion; directly affects the topic interpretations.
  • Number of t-SNE clusters = not reported
    Chosen using the Elbow method; cluster interpretation feeds the coordinated behavior claim.
  • Manual sentiment labels = subjective labels (positive, negative, neutral)
    The GNB classifier is trained on these hand labels; no annotation protocol or reliability measure is provided.
assumptions (6)
  • standard math Latent Dirichlet Allocation assumes documents are mixtures of topics and words are generated from topic distributions.
    Used in Section 1.3 for thematic analysis.
  • standard math Gaussian Naive Bayes assumes features are independent and normally distributed.
    Used in Section 1.2 for sentiment classification.
  • standard math t-SNE approximately preserves local structure in high-dimensional data.
    Used in Section 1.4 for visualization.
  • domain assumption The classification of accounts as propaganda or trusted is correct and meaningful.
    Section 1, no validation.
  • domain assumption Manual sentiment labels are accurate and unbiased.
    Section 1.2, no inter-rater reliability.
  • domain assumption The tweets from Feb-May 2022 are representative of the broader information war.
    The authors acknowledge this limitation in Section 3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Propaganda and Information Dissemination in the Russo-Ukrainian War: Natural Language Processing of Russian and Western Twitter Narratives." pith.science (2026). https://pith.science/paper/AGWGZEYD

@misc{pith2026250601807,
  author       = {Pith},
  title        = {Pith review of: Propaganda and Information Dissemination in the Russo-Ukrainian War: Natural Language Processing of Russian and Western Twitter Narratives},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AGWGZEYD}},
  note         = {Machine review of arXiv:2506.01807}
}
read the original abstract

The conflict in Ukraine has been not only characterised by military engagement but also by a significant information war, with social media platforms like X, formerly known as Twitter playing an important role in shaping public perception. This article provides an analysis of tweets from propaganda accounts and trusted accounts collected from the onset of the war, February 2022 until the middle of May 2022 with n=40,000 total tweets. We utilise natural language processing and machine learning algorithms to assess the sentiment and identify key themes, topics and narratives across the dataset with human-in-the-loop (HITL) analysis throughout. Our findings indicate distinct strategies in how information is created, spread, and targeted at different audiences by both sides. Propaganda accounts frequently employ emotionally charged language and disinformation to evoke fear and distrust, whereas other accounts, primarily Western tend to focus on factual reporting and humanitarian aspects of the conflict. Clustering analysis reveals groups of accounts with similar behaviours, which we suspect indicates the presence of coordinated efforts. This research attempts to contribute to our understanding of the dynamics of information warfare and offers techniques for future studies on social media influence in military conflicts.

Figures

Figures reproduced from arXiv: 2506.01807 by the authors.

Figure 1
Figure 1. shows our class balance, while [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Common Words found in Western Tweets 1.2 Feature Extraction and Sentiment Analysis Following preprocessing, feature extraction was performed using two primary methods: • Bag-of-Words (BoW): Created a matrix where each word represented a feature, capturing its frequency across the dataset. • TF-IDF: Applied a weighting scheme to adjust word frequency by its importance across the dataset, reducing the impact of common… view at source ↗
Figure 3
Figure 3. Common Words found in Russian Tweets 3 [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: LDA topic modeling results showing key themes. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: The model has a relatively high accuracy at 78%, classifying 6400+ correctly out of a total of 8300 predictions. [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: t-SNE of Clusters of GNB In contrast, Russian propaganda tweets show a markedly different set of themes and narrative strategies. The use of German words like "eingriff," "erweiterung," and "reichstag" indicates efforts to influence German-speaking audiences, reflectin…
Figure 7
Figure 7. Figure 7: t-SNE of Clusters of LDA Model 2.2 Sentiment Analysis and Narrative Strategies Sentiment analysis revealed substantial differences between Russian and Western tweets. Russian propaganda is characterised by a predominantly negative sentiment, using emotionally charged l…
Figure 8
Figure 8. Figure 8: Sentiment analysis results across different accounts. [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

28 extracted references · 26 canonical work pages

  1. [1]

    Alzanin, Aqil M

    Samah M. Alzanin, Aqil M. Azmi, and Hatim A. Aboalsamh. Short text classification for arabic social media tweets. Journal of King Saud University - Computer and Information Sciences, 34(9):6595–6604, October 2022. Received 26 September 2021, Revised 26 February 2022, Accepted 21 March 2022, Available online 23 March 2022, Version of Record 30 September 20...

  2. [2]

    A multi-faceted nlp analysis of misinformation spreaders in twitter

    Dimosthenis Antypas, Alun Preece, and Jose Camacho-Collados. A multi-faceted nlp analysis of misinformation spreaders in twitter. In Proceedings of the 14th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pages 71–83, Cardiff, United Kingdom, August 2024. Association for Computational Linguistics, Cardiff NLP, Sch...

  3. [3]

    Application of natural language processing (nlp) in detecting and preventing suicide ideation: A systematic review

    Abayomi Arowosegbe and Tope Oyelade. Application of natural language processing (nlp) in detecting and preventing suicide ideation: A systematic review. International Journal of Environmental Research and Public Health, 20(2):1514, January 2023. doi:10.3390/ijerph20021514

  4. [4]

    Hybrid warfare and disinformation: A ukraine war perspective

    Sascha-Dominik Dov Bachmann, Dries Putter, and Guy Duczynski. Hybrid warfare and disinformation: A ukraine war perspective. Global Policy, August 2023. First published: 25 August 2023. doi:10.1111/1758-5899. 13257

  5. [5]

    Schmitt, Judit Szakács, and Erik Uszkiewicz

    Judit Bayer, Bernd Holznagel, Katarzyna Lubianiec, Adela Pintea, Josephine B. Schmitt, Judit Szakács, and Erik Uszkiewicz. Disinformation and propaganda: Impact on the functioning of the rule of law and democratic processes in the eu and its member states - 2021 update. Report PE 653.633, European Parliament, April 2021. Policy Department for External Rel...

  6. [6]

    Crimea profile, 2018

    BBC News. Crimea profile, 2018. January 17, 2018, sec. Europe. URL: https://www.bbc.com/news/ world-europe-18287223

  7. [7]

    Blei, Andrew Y

    David M. Blei, Andrew Y . Ng, and Michael I. Jordan. Latent dirichlet allocation.Journal of Machine Learning Research, 3:993–1022, January 2003. Submitted 2/02; Published 1/03. URL: http://www.jmlr.org/papers/ volume3/blei03a/blei03a.pdf

  8. [8]

    Analyzing the worldwide perception of the russia-ukraine conflict through twitter

    Bernardo Breve, Loredana Caruccio, Stefano Cirillo, Vincenzo Deufemia, and Giuseppe Polese. Analyzing the worldwide perception of the russia-ukraine conflict through twitter. Journal of Big Data, 11:Article number: 76, 2024

Show all 28 references
  1. [9]

    Conflict in ukraine’s donbas: A visual explainer, 2021

    Crisis Group. Conflict in ukraine’s donbas: A visual explainer, 2021. June 29, 2021. URL: https://www. crisisgroup.org/content/conflict-ukraines-donbas-visual-explainer

  2. [10]

    A topic modeling comparison between lda, nmf, top2vec, and bertopic to demystify twitter posts

    Roman Egger and Joanne Yu. A topic modeling comparison between lda, nmf, top2vec, and bertopic to demystify twitter posts. Frontiers in Sociology, 7, May 2022. This article is part of the Research Topic Big Data and Machine Learning in Sociology. doi:10.3389/fsoc.2022.886498

  3. [11]

    Disguised propaganda from digital to social media

    Johan Farkas and Christina Neumayer. Disguised propaganda from digital to social media. In Jeremy Hunsinger, Lisbeth Klastrup, and Matthew Allen, editors, Second International Handbook of Internet Research. Springer, Dordrecht, 2018. doi:10.1007/978-94-024-1202-4_33-1 . 7 A PR...

  4. [12]

    Disinformation as political communication

    Deen Freelon and Chris Wells. Disinformation as political communication. Political Communication, 37(2):145– 146, 2020

  5. [13]

    Machine learning and natural language processing of social media data for event detection in smart cities

    Andrei Hodorog, Ioan Petri, and Yacine Rezgui. Machine learning and natural language processing of social media data for event detection in smart cities. Sustainable Cities and Society, 85:104026, 2022. Received 10 February 2022, Revised 16 May 2022, Accepted 22 June 2022, Ava...

  6. [14]

    Huang, G

    D. Huang, G. Guan, J. Zhou, et al. Network-based naive bayes model for social network. Science China Mathematics, 61:627–640, 2018. doi:10.1007/s11425-017-9209-6

  7. [15]

    fake news

    Dean Jackson. Distinguishing disinformation from propaganda, misinformation, and “fake news”. Technical report, International Forum for Democratic Studies, 2018. Prepared by the Interna- tional Forum for Democratic Studies. URL: https://www.ned.org/wp-content/uploads/2018/06/ ...

  8. [16]

    Propaganda and Persuasion

    Garth S Jowett and Victoria O’Donnell. Propaganda and Persuasion. Sage Publications, 2019

  9. [17]

    D. H. Lan and T. M. Tung. Exploring fake news awareness and trust in the age of social media among university student tiktok users. Cogent Social Sciences, 10(1), 2024. doi:10.1080/23311886.2024.2302216

  10. [18]

    Organizational propaganda on the internet: A systematic review

    Irina Lock and Ramona Ludolph. Organizational propaganda on the internet: A systematic review. Inter- national Journal of Communication , 9(1), 2019. First published online September 6, 2019. doi:10.1177/ 2046147X19870844

  11. [19]

    Ukraine: Conflict at the crossroads of europe and russia, 2023

    Jonathan Masters. Ukraine: Conflict at the crossroads of europe and russia, 2023. Last updated February 14, 2023. URL: https://www.cfr.org/backgrounder/ukraine-conflict-crossroads-europe-and-russia

  12. [20]

    Social media’s role in ‘hybrid strategies’

    Thomas Elkjer Nissen. Social media’s role in ‘hybrid strategies’. Technical report, NATO Strategic Communica- tions Centre of Excellence, 2023. Prepared by the NATO Strategic Communications Centre of Excellence

  13. [21]

    The role of social media in contemporary conflicts

    Maria Ojala. The role of social media in contemporary conflicts. Media, War & Conflict, 12(4):433–448, 2019

  14. [22]

    New Narratives: Digital Diplomacy and the Evolution of Statecraft

    James Pamment. New Narratives: Digital Diplomacy and the Evolution of Statecraft. Routledge, 2020

  15. [23]

    Information warfare in russia’s war in ukraine, 2022

    Christian Perez. Information warfare in russia’s war in ukraine, 2022. August 22, 2022. URL: https:// foreignpolicy.com/2022/08/22/information-warfare-in-russias-war-in-ukraine/

  16. [24]

    Commanding the trend: Social media as information warfare

    Jarred Prier. Commanding the trend: Social media as information warfare. Strategic Studies Quarterly, 11(4):50– 85, 2017. Winter 2017. URL: https://www.jstor.org/stable/26271634

  17. [25]

    Deplatforming: Following extreme internet celebrities to telegram and alternative social media

    Richard Rogers. Deplatforming: Following extreme internet celebrities to telegram and alternative social media. European Journal of Communication, 36(4):356–372, 2021

  18. [26]

    Computational propaganda: Political parties, politicians, and political manipulation on social media

    Samuel C Woolley and Philip N Howard. Computational propaganda: Political parties, politicians, and political manipulation on social media. Oxford University Press, 2017

  19. [27]

    Understanding online disinformation on social media: An empirical study based on twitter during the covid-19 pandemic

    Yang Zhou, Zhiqi Zhang, Yingjie Zhao, and Wei Wang. Understanding online disinformation on social media: An empirical study based on twitter during the covid-19 pandemic. Journal of Information Science, 48(2):253–265, 2022

  20. [28]

    Political effects of the internet and social media

    Ekaterina Zhuravskaya, Maria Petrova, and Ruben Enikolopov. Political effects of the internet and social media. Annual Review of Economics, 2024. 8

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.