Pith. sign in

REVIEW 5 major objections 5 minor 60 references

Studying Disinformation Narratives on Social Media with LLMs and Semantic Similarity

T0 review · 5 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A similarity score computed from a single target narrative sentence can detect, trace, and characterize disinformation across large social media datasets, and does so in general agreement with human similarity judgments, subject to…

desk verdict A cleanly written thesis with a genuinely useful dashboard and honest limitations, but the central detection claim is undercut by stance-blind cosine similarity and circular case-study validation; worth sending out, but only with major revisions. read the letter →

arxiv 2507.20066 v1 pith:M36MAO5R submitted 2025-07-26 cs.SI cs.CYcs.ET

Chaytan Inman This is my paper · ORCID
classification cs.SIcs.CYcs.ET
keywords disinformationdetectionsemanticsimilaritycosinesentencetransformersnarrativetracingsocialmediaanalysiscontinuousmeasurementSTS-Bbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a continuous similarity score — computed by embedding a tweet and a target narrative sentence, then taking their cosine similarity — can detect, trace, and characterize disinformation in large social media datasets without time-intensive qualitative coding. The claim matters because disinformation typically mixes falsehoods with partial truths, so a binary in-or-out label misses the gray area; a continuous measure of how close a post is to a known narrative would let researchers see the strength and evolution of alignment. The author builds two tools on this idea: a tracing tool that scores every tweet against a target narrative and graphs the scores as a timeline, and a narrative synthesis tool that clusters high-scoring tweets and summarizes their dominant themes. The scoring model is validated on the STS-B benchmark, where its scores correlate at $0.8696$ with human similarity judgments, and both tools are demonstrated on two cases: the "2020 election was stolen" narrative in Donald Trump's tweets, and the "transgender people are harmful to society" narrative across four news outlets. The author's stated finding is that this form of detection is generally aligned with human similarity judgments, although some misclassifications occur.

What carries the argument

The load-bearing mechanism is a sentence-transformer similarity model that encodes each tweet and the target narrative into 384-dimensional numerical vectors (embeddings), so that "how close is this tweet to the narrative" reduces to the cosine similarity of two vectors. Because the resulting score is continuous, every tweet receives a degree of alignment rather than a yes-or-no label; a similarity threshold (set to $0.45$ for the election case and $0.38$ for the anti-trans case) selects the tweets that form a traceable timeline. The same embeddings feed a K-means clustering step, and the tweets in each cluster are summarized by an instruction-tuned generation model into "dominant narratives," which is how the tool characterizes what the detected disinformation actually says. Validation of the similarity model against human judgments on the STS-B benchmark is what licenses the whole pipeline: the $0.8696$ correlation is the evidence that the continuous scores mean something to humans.

What would settle it

Count the failure class directly: take a few thousand labeled tweets on a contested topic in which half endorse a narrative and half oppose it using overlapping vocabulary, run them through the tracing tool, and measure the false-positive rate among the opposing half at the paper's thresholds; the paper already exhibits one instance (a pro-trans tweet scoring $0.590$ against the anti-trans narrative), but whether the method's detection claim stands depends on how often such inversions occur — a number the paper does not report.

Watch

Extended reading notes

Core claim

The paper's central claim is that disinformation is better measured as a continuous quantity than classified as a binary label: the cosine similarity between a tweet's embedding and the embedding of a known disinformation narrative tells you how strongly that tweet aligns with the narrative, even when the tweet shares only part of the narrative's meaning. On this basis, the author claims, researchers can detect, trace, and characterize subtle patterns of disinformation spread across datasets too large for qualitative coding, and the resulting scores will generally agree with human judgments about textual similarity. The evidence offered has two parts. The similarity model achieves a $0.8696$ Pearson correlation with human ratings on the STS-B benchmark, with error concentrated on pairs humans rate as most dissimilar, which the model overestimates. The two case studies then demonstrate the claim in the wild: the election-hoax timeline spikes in November 2020 but also shows doubt-sowing months before the election, and Fox News produces anti-trans narrative content at rates far exceeding its share of total tweets, with high-similarity tweets numerous only for the two right-leaning outlets. The author's summary finding is that this continuous form of detection aligns with human similarity judgments in general, subject to documented misclassifications, most notably same-topic tweets with opposite sentiment.

Load-bearing premise

The load-bearing premise is that the cosine similarity between a tweet's embedding and a single target-narrative sentence measures how much that tweet actually endorses the narrative; the paper's own results show this premise fails for same-topic, opposite-sentiment tweets, which can score as high as $0.590$ (a pro-trans New York Times tweet against "Transgender people are harmful to society"), and if that failure is frequent, the timelines and thresholds are not measuring narrative alignment as claimed.

Editorial extensions

If this is right

  • Whole-sentence target narratives replace keyword lists: the tool catches tweets such as "I won the election!" (scoring $0.489$) or mentions of "unsecured" ballots that a keyword search would likely miss.
  • The timelines expose temporal structure that frequency counts flatten — Case Study 1 shows pre-election "seeds of doubt" followed by a post-election spike, consistent with prior findings on the election-hoax narrative.
  • Cross-outlet comparison becomes quantitative: Fox News exceeds a $0.38$ similarity threshold at $2.0\times$ the rate of The Gateway Pundit, $7.1\times$ the New York Times, and $14.1\times$ The Guardian — far beyond its $1.6$–$2.5\times$ share of total tweets — and tweets above $0.5$ similarity are numerous only for the two right-leaning outlets.
  • The method transfers to any textual data — other platforms, longer documents, video transcripts — because the pipeline only requires a body of text and a target narrative.
  • Detection thresholds must absorb the model's documented bias: it overestimates similarity on low-similarity pairs (signed error $0.1276$ at human score $0$), so thresholded detection will over-flag rather than under-flag, and same-topic opposite-sentiment tweets near $0.5$ are the known failure mode.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the metric works as claimed, a monitoring use follows that the paper does not develop: track a target narrative's mean or median similarity across a corpus over time and flag anomalous rises, turning the tracing tool into an early-warning signal rather than a post-hoc analysis.
  • A direct testable extension is the paper's own hinted polarity flip: attach a stance classifier to each high-scoring tweet and invert the similarity score when stance opposes the narrative; the open question is whether the anti-trans case study's New York Times false positives drop below threshold without losing true positives.
  • Because the score measures similarity to one chosen sentence, results will depend on how the target narrative is phrased; a practical consequence is that cross-study comparisons will need canonical narrative phrasings, a standardization problem the paper does not address.
  • The same continuous framing transfers naturally to other "partial truth" domains — climate, vaccine, or migration narratives — where the main validation burden is to confirm that the embedding model's similarity judgments match the distinctions a domain expert would draw.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper develops a continuous semantic-similarity-based method for detecting, tracing, and characterizing disinformation narratives in social media text. A sentence-transformer embedding model is used to compute cosine similarity between tweets and a user-supplied target narrative; a tracing tool plots similarity over time, and a narrative synthesis tool clusters high-similarity tweets and generates summaries using an LLM. The similarity model is validated on GLUE STS-B (Pearson r = 0.8696, MAE = 0.1383), and the combined dashboard is demonstrated on two case studies: tracing the '2020 election was stolen' narrative in Donald Trump's tweets, and tracing 'Transgender people are harmful to society' across four news outlets. The paper claims that the continuous metric can detect, trace, and characterize disinformation without time-intensive qualitative analysis, with the caveat that some misclassifications occur.

Significance. If the central claim were established, the paper would contribute a practical, repeatable pipeline that complements keyword-based disinformation detection: the STS-B validation is a genuine external check on the similarity model, and the open dashboard and documented code are useful resources. The two case studies also demonstrate an accessible way to visualize narrative alignment over time and across sources. However, the strongest claim—that cosine similarity to a single target sentence measures disinformation alignment—is not supported by the current evidence, because the metric is stance-blind and the case studies lack ground truth labels for narrative agreement. The paper itself acknowledges this limitation but does not quantify its impact. The contribution is therefore promising but the empirical validation needs substantial rework before the detection/tracing claims can be accepted.

major comments (5)
  1. [Section 5.4, Figure 8] The similarity metric is stance-blind: the paper's own example shows a New York Times tweet supporting transgender people, but containing the word 'harmful', scoring 0.590 against 'Transgender people are harmful to society'. This score lies above the 0.38 threshold used in Case Study 2 and inside the 0.5–0.75 bracket that Table 7 explicitly warns 'often' contains opposite-sentiment texts. Consequently, the timelines in Section 4.4 (e.g., 10 Guardian and 20 NYT tweets above threshold) and the Fox-vs-left ratios in Table 8 are not established measures of disinformation spread; they may include critical or neutral mentions of the narrative. The claim that such false positives are 'rare' is unsupported because no opposition-labeled evaluation is reported, and the STS-B validation in Section 4.1 measures general semantic equivalence, not stance agreement. The authors should either add a stance-separated evaluation (e.g., an annotated dataset of same-topic opposing-sentiment pairs) or re-scope the claims to 'topical similarity' rather than 'disinformation alignment'.
  2. [Section 4.3 and Section 5.1] The Case Study 1 validation is circular. Tweets are selected by their similarity to the target narrative, the narrative synthesis tool summarizes the selected tweets, and the high similarity of the generated narratives to the target narrative is then cited as 'useful validation of the accuracy of the trace' (Section 4.3). Because both the selection and the confirmation use the same cosine-similarity function, resemblance is built into the procedure and does not provide independent evidence that the trace captures disinformation. An external ground truth is needed—for example, expert-coded labels of whether each tweet endorses the stolen-election claim, or a comparison against a pre-existing hand-validated timeline—before the tool can be said to detect disinformation as opposed to topical mention.
  3. [Section 3.3] The similarity thresholds are chosen post hoc without a principled or reproducible criterion. The text states that the 0.38 threshold was selected because it 'reduced the unrelated tweets while retaining the greatest amount of related tweets during testing', but no testing details, stability analysis, or separate validation set are provided. The 0.45 threshold in Case Study 1 is not justified at all. Since every quantitative result in Section 4.4 (Figures 4–6, Table 8) depends directly on these thresholds, and since the STS-B error analysis in Table 1 shows the largest errors occur in the 0.2–0.4 human-score range where these thresholds lie, the threshold choice is load-bearing. The authors should report results across a range of thresholds or derive thresholds from a labeled development set.
  4. [Section 5.1 and Section 5.4] The narrative synthesis tool is not validated for the claimed characterization contribution. The paper relies on the base model's performance on unrelated commonsense benchmarks and asserts that hallucination 'did not appear to occur', but no systematic evaluation of the generated narratives against human expert coding is reported. Section 5.4 acknowledges this gap and calls for expert comparison as future work. For a paper whose stated contributions include characterization, this is a load-bearing omission; a small manual evaluation comparing, say, the six generated narratives in Case Study 1 and Table 13 with independent human summaries would materially strengthen the claim.
  5. [Section 4.1, Table 1] The STS-B validation is a useful external benchmark, but it does not address the operational regime of the case studies. The model's absolute error is highest for sentence pairs humans rate as dissimilar (average absolute error 0.1424 at human score 0.0 and 0.2043 at 0.25), which is exactly the region where the case-study thresholds (0.38 and 0.45) are set. Reporting only the aggregate Pearson r and MAE obscures this. The authors should report performance separately for the low-similarity bracket and discuss how the measured overestimation bias affects threshold-based detection.
minor comments (5)
  1. [Section 4.2 and Section 4.3] The figure numbering is inconsistent: Figure 3 is used both for the dashboard features in Section 4.2 and for the Case Study 1 timeline in Section 4.3. Please renumber the figures throughout.
  2. [Table 8] The column headers 'Ratio of Total Tweets to Fox News' and 'Tweets>0.38 Similarity Ratio to Fox News' are reversed relative to the interpretation in the text (the text says Fox tweets are at a 1.6/1 ratio to The Gateway Pundit, i.e., Fox/outlet, not outlet/Fox). Please clarify the direction of the ratios.
  3. [Section 3.1] The specific similarity model is not identified (only described as a distilled sentence transformer with 384-dimensional embeddings). Please name the model and version, since STS-B results are model-specific.
  4. [Section 5.4] The sentence 'Don't do that. I won't grant you access if you plan to do that.' in the ethics discussion is informal for a journal article; consider rewording to a professional statement of the access-control policy.
  5. [Section 3.3] The paper does not specify the exact dates for which Trump's tweets were retrieved beyond '01/01/2020 to 01/01/2021', but the text in Section 5.2 refers to a tweet from 05/01/2020; please ensure the date format is consistent.

Circularity Check

2 steps flagged · score 6.0 of 10

Case-study validation is circular: tweets are pre-selected by similarity to the target narrative, then LLM-generated narratives that restate the target are cited as confirming the trace; the Case Study 2 threshold is also tuned on the data it later reports.

  1. self definitional [Section 4.3, Case Study 1 (reiterated in Section 5.2)]
    "Generating three narratives in this case confirms the accuracy of the tracing tool, as the six different themes for the three separate narratives are different phrasings of the original target narrative. The similarity of the generated narratives to the target narrative is useful validation of the accuracy of the trace."

    The narrative synthesis tool receives only tweets already above the 0.45 similarity threshold to 'The 2020 election was stolen' (Section 3.3/4.3, 'the tweets in the timeline'). An LLM summarizing that pre-filtered set will naturally produce phrasings of the same target. Confirming the trace by observing that these generated narratives resemble the target is therefore comparing the output to the selection criterion; it is a self-consistency check, not independent evidence that the timeline contains disinformation. The same move is repeated in Section 5.2 as a check against model bias, but the input set already guarantees target-like summaries.

  2. fitted input called prediction [Section 3.3 (threshold choice) with Section 4.4/Table 8 and Section 5.3 (reported counts)]
    "The similarity threshold is set to 0.38. I selected this threshold because it reduced the unrelated tweets while retaining the greatest amount of related tweets during testing."

    The 0.38 threshold is tuned with the researcher's 'related/unrelated' judgment on the same Case Study 2 tweet streams, with no reported held-out set. Table 8 then reports 'Tweets>0.38 Similarity' for each outlet (e.g., Fox News 141; 14.1:1 vs The Guardian) and Section 5.3 presents these ratios as empirical findings about anti-trans disinformation spread. The counts are a function of the fitted threshold: choosing the threshold to maximize 'related' tweets on these data means the subsequent above-threshold ratios are partly constructed by that choice, so they are not independent predictions.

full rationale

The GLUE STS-B validation is genuinely external and non-circular: it supports the claim that the embedding model's cosine similarity tracks human textual similarity, but not the further claim that similarity to a target narrative detects disinformation alignment. The circularity is in the case-study validation loop: tweets are threshold-selected by similarity to the target, then LLM-generated narratives over those tweets are cited as confirming the trace because they restate the target. The Case Study 2 threshold is tuned on the same data whose above-threshold counts are later reported as findings. Section 5.4 explicitly concedes a same-topic, opposite-sentiment New York Times tweet scored 0.590 against 'Transgender people are harmful to society' (above the 0.38 threshold); this is a validity counterexample rather than a circularity, but it further undermines the claim that the reported timelines measure disinformation alignment. No load-bearing self-citation or uniqueness-imported-from-authors pattern is present; the only self-citation ([15]) is to the thesis website. Overall score 6: one central validation step reduces by construction and one empirical count set is partly threshold-fitted, while the underlying embedding benchmark remains independent.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper introduces software tools, not new physical or theoretical entities. The main auxiliary assumptions are the validity of embedding similarity as a disinformation signal and the representativeness of single-sentence target narratives.

free parameters (4)
  • Case Study 1 similarity threshold = 0.45
    Hand-selected in Section 3.3; no principled criterion or sensitivity analysis is provided.
  • Case Study 2 similarity threshold = 0.38
    Selected post hoc in Section 3.3 to reduce unrelated tweets while retaining related tweets during testing.
  • Number of clusters for narrative synthesis = 3
    User-selected in Section 3.1; the paper notes that 'the number of clusters significantly varies the results'.
  • Narrative generation temperature = 0.9
    Chosen from initial experimentation in Section 3.1; not rigorously optimized.
assumptions (4)
  • domain assumption Cosine similarity between sentence embeddings is a meaningful measure of narrative alignment for disinformation detection.
    Central to the tracing tool; Section 3.1. The paper's own Figure 8 shows a same-topic opposite-sentiment tweet scoring 0.590, indicating this assumption is imperfect.
  • standard math The GLUE STS-B human similarity labels provide a valid external benchmark for the similarity model's behavior.
    Used in Section 3.2 and 4.1 to validate the model; this is a standard NLP benchmark assumption.
  • domain assumption Tweets above the similarity threshold can be clustered with K-means and summarized by an LLM to yield dominant narratives.
    Underlies the narrative synthesis tool; Section 3.1. The paper acknowledges the tool is not validated against expert coding (Section 5.4).
  • domain assumption The target disinformation narrative can be adequately represented as a single natural-language sentence.
    Both case studies use a single sentence as the target; Section 3.3. Complex narratives may not compress to one sentence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Studying Disinformation Narratives on Social Media with LLMs and Semantic Similarity." pith.science (2026). https://pith.science/paper/M36MAO5R

@misc{pith2026250720066,
  author       = {Pith},
  title        = {Pith review of: Studying Disinformation Narratives on Social Media with LLMs and Semantic Similarity},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/M36MAO5R}},
  note         = {Machine review of arXiv:2507.20066}
}
read the original abstract

This thesis develops a continuous scale measurement of similarity to disinformation narratives that can serve to detect disinformation and capture the nuanced, partial truths that are characteristic of it. To do so, two tools are developed and their methodologies are documented. The tracing tool takes tweets and a target narrative, rates the similarities of each to the target narrative, and graphs it as a timeline. The second narrative synthesis tool clusters tweets above a similarity threshold and generates the dominant narratives within each cluster. These tools are combined into a Tweet Narrative Analysis Dashboard. The tracing tool is validated on the GLUE STS-B benchmark, and then the two tools are used to analyze two case studies for further empirical validation. The first case study uses the target narrative "The 2020 election was stolen" and analyzes a dataset of Donald Trump's tweets during 2020. The second case study uses the target narrative, "Transgender people are harmful to society" and analyzes tens of thousands of tweets from the media outlets The New York Times, The Guardian, The Gateway Pundit, and Fox News. Together, the empirical findings from these case studies demonstrate semantic similarity for nuanced disinformation detection, tracing, and characterization. The tools developed in this thesis are hosted and can be accessed through the permission of the author. Please explain your use case in your request. The HTML friendly version of this paper is at https://chaytanc.github.io/projects/disinfo-research (Inman, 2025).

Figures

Figures reproduced from arXiv: 2507.20066 by the authors.

Figure 3
Figure 3. Analysis of the Election Hoax Narrative. Results from the Tweet Narrative Anal [PITH_FULL_IMAGE:figures/full_fig_p024_3.png] view at source ↗
Figure 4
Figure 4. A combined timeline of all four news outlets compared in Case Study 2, graphed [PITH_FULL_IMAGE:figures/full_fig_p025_4.png] view at source ↗
Figure 8
Figure 8. High Similarity of a Tweet with Opposite Sentiment to the Target Narrative. The [PITH_FULL_IMAGE:figures/full_fig_p034_8.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 47 canonical work pages

  1. [1]

    Ad Observer. (n.d.). Chrome Web Store; Google

    work page

  2. [2]

    Alkaissi, H., & McFarlane, S. (2023). Artificial hallucinations in ChatGPT: Implications in scientific writing.Cureus,15(2).https://doi.org/10.7759/cureus.35179 35

    doi:10.7759/cureus.35179 2023

  3. [3]

    Arcos, I., Rosso, P., & Salaverr´ ıa, R. (2025). Divergent Emotional Patterns in Disin- formation on Social Media? An Analysis of Tweets and TikToks about the DANA in Valencia

    work page 2025

  4. [4]

    Banerjee, S., Agarwal, A., & Singla, S. (2024). LLMs Will Always Hallucinate, and We Need to Live With This. ArXiv.org.https://arxiv.org/abs/2409.05746

    arXiv 2024

  5. [5]

    Barnett, S., Brannelly, Z., Kurniawan, S., & Wong, S. (2023). Fine-Tuning or Fine- Failing? Debunking Performance Myths in Large Language Models. Arxiv.org.https: //arxiv.org/html/2406.11201v1

    arXiv 2023

  6. [6]

    Benzoni, P. (2024). Social Data Search. Alliance for Securing Democracy.https:// securingdemocracy.gmfus.org/social-data-search/?q=

    work page 2024

  7. [7]

    Berger, A., Della, V., & Della, S. (1996). A Maximum Entropy Approach to Natural Language Processing.https://aclanthology.org/J96-1002.pdf

    work page 1996

  8. [8]

    M., Fariss, C

    Bond, R. M., Fariss, C. J., Jones, J. J., Kramer, A. D. I., Marlow, C., Settle, J. E., & Fowler, J. H. (2012). A 61-million-person Experiment in Social Influence and Political Mobilization.Nature,489(7415), 295–298.https://doi.org/10.1038/nature11421

    doi:10.1038/nature11421 2012

Show all 60 references
  1. [9]

    N., & Loh, W

    Bossert, L. N., & Loh, W. (2025). Why the carbon footprint of generative large language models alone will not help us assess their sustainability.Nature Machine Intelligence. https://doi.org/10.1038/s42256-025-00979-y

    2025 doi

  2. [10]

    Bot Sentinel. (2024). Botsentinel.com.https://botsentinel.com/

    2024

  3. [11]

    Bowler, S., Carreras, M., & Merolla, J. L. (2022). Trump Tweets and Demo- cratic Attitudes: Evidence from a Survey Experiment.Political Research Quarterly, 106591292211373.https://doi.org/10.1177/10659129221137348

    2022 doi

  4. [12]

    Brown, B. (2016). Trump Twitter Archive. Www.thetrumparchive.com.https://www. thetrumparchive.com/

    2016

  5. [13]

    Cao, B., Cai, D., & Lam, W. (2025). InfiniteICL: Breaking the Limit of Context Window Size via Long Short-term Memory Transformation

    2025

  6. [14]

    Chandrasekaran, D., & Mago, V. (2021). Evolution of Semantic Similarity—A Survey. ACM Computing Surveys,54(2), 1–37.https://doi.org/10.1145/3440755

    2021 doi

  7. [15]

    Inman, C. (2025). Master’s Thesis: Tracing Disinformation with NLP.https:// chaytanc.github.io/projects/disinfo-research/

    2025

  8. [16]

    Cho, K., Merrienboer, van, Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. ArXiv.org.https://arxiv.org/abs/1406.1078

    2014 arXiv

  9. [17]

    Dong, X., & Lian, Y. (2021). A review of social media-based public opinion analyses: Challenges and recommendations.Technology in Society,67, 101724.https://doi. org/10.1016/j.techsoc.2021.101724 36

    2021

  10. [18]

    El Barachi, M., AlKhatib, M., Mathew, S., & Oroumchian, F. (2021). A novel sentiment analysis framework for monitoring the evolving public opinion in real-time: Case study on climate change.Journal of Cleaner Production,312, 127820.https://doi.org/10. 1016/j.jclepro.2021.127820

    2021

  11. [19]

    European External Action Service. (n.d.). EU vs DISINFORMATION. Retrieved 2025, fromhttps://euvsdisinfo.eu/

    2025

  12. [20]

    European Parliament. (2021). BRIEFING European Parliament Liaison Office in Washington DC.https://www.europarl.europa.eu/RegData/etudes/BRIE/2021/ 679076/EPRS_BRI(2021)679076_EN.pdf

    2021

  13. [21]

    Field, A., Kliger, D., Wintner, S., Pan, J., Jurafsky, D., & Tsvetkov, Y. (2018). Framing and Agenda-setting in Russian News: a Computational Analysis of Intricate Political Strategies

    2018

  14. [22]

    Freelon, D., McIlwain, C., & Clark, M. (2016). Quantifying the power and consequences of social media protest.New Media & Society,20(3), 990–1011.https://doi.org/10. 1177/1461444816676646

    2016

  15. [23]

    Fried, A., & Harris, D. B. (2020). In Suspense: Donald Trump’s Efforts to Under- mine Public Trust in Democracy.Society,57(5), 527–533.https://doi.org/10.1007/ s12115-020-00526-y

    2020

  16. [24]

    GLUE Benchmark. (n.d.). Gluebenchmark.com.https://gluebenchmark.com/ leaderboard/

  17. [25]

    Gotfredsen, S. G. (2023, December 6). Q&A: What happened to academic re- search on Twitter?Columbia Journalism Review.https://www.cjr.org/tow_center/ qa-what-happened-to-academic-research-on-twitter.php

    2023

  18. [26]

    Heersmink, R., Barend de Rooij, Clavel, J., & Colombo, M. (2024). A phenomenol- ogy and epistemology of large language models: transparency, trust, and trust- worthiness.Ethics and Information Technology,26(3).https://doi.org/10.1007/ s10676-024-09777-3

    2024

  19. [27]

    (2023, February 2)

    Hu, K. (2023, February 2). ChatGPT sets record for fastest- growing user base.Reuters.https://www.reuters.com/technology/ chatgpt-sets-record-fastest-growing-user-base-analyst-note-2023-02-01/

    2023

  20. [28]

    Information Laundromat. (2025). Informationlaundromat.com.https:// informationlaundromat.com/

    2025

  21. [29]

    Jos´ e´Angel Alc´ antara-Liz´ arraga, & Jima-Gonz´ alez, A. (2024). Digital manipulation and mass mobilization over the long run: evidence from Latin America.Frontiers in Political Science,6.https://doi.org/10.3389/fpos.2024.1296004

    2024

  22. [30]

    Junkipedia. (2024). Junkipedia.org.https://www.junkipedia.org/ 37

    2024

  23. [31]

    S., Garcia-Camargo, I., Spiro, E

    Kennedy, I., Wack, M., Beers, A., Schafer, J. S., Garcia-Camargo, I., Spiro, E. S., & Starbird, K. (2022, September 15). Repeat Spreaders and Election Delegit- imization: A Comprehensive Dataset of Misinformation Tweets from the 2020 U.S. Election. Washington.edu.https://digit...

    2022

  24. [32]

    Kotek, H., Dockum, R., & Sun, D. (2023). Gender bias and stereotypes in Large Language Models.Proceedings of the ACM Collective Intelligence Conference.https: //doi.org/10.1145/3582269.3615599

    2023

  25. [33]

    Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., K¨ uttler, H., Lewis, M., Yih, W.-T., Rockt¨ aschel, T., Riedel, S., Kiela, D., Facebook, & Research, A. (2021). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

    2021

  26. [34]

    Li, C., Liu, Z., Xiao, S., & Shao, Y. (2023). Making Large Language Models A Better Foundation For Dense Retrieval

    2023

  27. [35]

    (2019, July 26)

    Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettle- moyer, L., & Stoyanov, V. (2019, July 26). RoBERTa: A Robustly Optimized BERT Pretraining Approach. ArXiv.org.https://arxiv.org/abs/1907.11692

    2019 arXiv

  28. [36]

    Meta. (2025). Meta Ad Library API. Facebook.com.https://www.facebook.com/ads/ library/api/?source=nav-header

    2025

  29. [37]

    (2023, September 27)

    Mistral AI. (2023, September 27). Mistral 7B. Mistral.ai.https://mistral.ai/news/ announcing-mistral-7b

    2023

  30. [38]

    Mistral AI. (2024). Mistral NeMo.https://mistral.ai/en/news/mistral-nemo

    2024

  31. [39]

    Mu˜ noz, P., D´ ıez, F., & Bellog´ ın, A. (2024). Modeling disinformation networks on Twitter: structure, behavior, and impact.Applied Network Science,9(1).https: //doi.org/10.1007/s41109-024-00610-w

    2024 doi

  32. [40]

    National Center for Supercomputing Applications. (2025). SMILE: social media intelli- gence & learning environment. Illinois.edu.https://smile.smm.ncsa.illinois.edu/

    2025

  33. [41]

    Open AI. (n.d.). OpenAI Platform Tokenizer. Platform.openai.com. Retrieved May 2025, fromhttps://platform.openai.com/tokenizer

    2025

  34. [42]

    B., Bechmann, A., & Petersen, M

    Osmundsen, M., Bor, A., Vahlstrup, P. B., Bechmann, A., & Petersen, M. B. (2021). Partisan Polarization Is the Primary Psychological Motivation behind Political Fake News Sharing on Twitter.American Political Science Review,115(3), 1–17.https: //doi.org/10.1017/s0003055421000290

    2021 doi

  35. [43]

    (n.d.-a)

    OSoMe. (n.d.-a). Botometer X. Botometer.iuni.iu.edu.https://botometer.osome.iu. edu/

  36. [44]

    (n.d.-b)

    OSoMe. (n.d.-b). Hoaxy2 beta. Hoaxy.iuni.iu.edu.https://hoaxy.osome.iu.edu/ 38

  37. [45]

    OSoMe. (2025). OSoMeNet. OSoMeNet.https://osome.iu.edu/tools/osomenet/

    2025

  38. [46]

    Park, C., Mendelsohn, J., Field, A., & Tsvetkov, Y. (2022). Challenges and Oppor- tunities in Information Manipulation Detection: An Examination of Wartime Russian Media (pp. 5238–5264).https://aclanthology.org/2022.findings-emnlp.382.pdf

    2022

  39. [47]

    RAND Corporation. (2022). Tools That Fight Disinformation Online. Www.rand.org.https://www.rand.org/research/projects/truth-decay/ fighting-disinformation/search.html

    2022

  40. [48]

    (2021).ACTIVE MEASURES : the secret history of disinformation and political warfare

    Rid, T. (2021).ACTIVE MEASURES : the secret history of disinformation and political warfare. Picador. (Original work published 2020)

    2021

  41. [49]

    E., Hinton, G

    Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors.Nature,323(6088), 533–536.https://doi.org/10.1038/ 323533a0

    1986

  42. [50]

    Schmidt, C., Reddy, V., Zhang, H., Alameddine, A., Uzan, O., Pinter, Y., Tanner, C., & Technologies, K. (2024). Tokenization Is More Than Compression.https://arxiv. org/pdf/2402.18376

    2024 arXiv

  43. [51]

    (2019, November 25)

    Schwartz, O. (2019, November 25). In 2016, Microsoft’s Racist Chatbot Revealed the Dangers of Online Conversation.IEEE Spectrum.https://spectrum.ieee.org/ in-2016-microsofts-racist-chatbot-revealed-the-dangers-of-online-conversation

    2019

  44. [52]

    Starbird, K., Arif, A., & Wilson, T. (2019). Disinformation as Collaborative Work. Proceedings of the ACM on Human-Computer Interaction,3(CSCW), 1–26.https: //doi.org/10.1145/3359229

    2019 doi

  45. [53]

    Starbird, K., DiResta, R., & DeButts, M. (2023). Influence and Improvisation: Partic- ipatory Disinformation during the 2020 US Election.Social Media and Society,9(2). https://doi.org/10.1177/20563051231177943

    2023 doi

  46. [54]

    Congress

    U.S. Congress. (2021). DISINFORMATION IN THE GRAY ZONE: OPPOR- TUNITIES, LIMITATIONS, AND CHALLENGES HEARING BEFORE THE SUBCOMMITTEE ON INTELLIGENCE AND SPECIAL OPERATIONS OF THE COMMITTEE ON ARMED SERVICES HOUSE OF REPRESENTATIVES ONE HUNDRED SEVENTEENTH CONGRESS FIRST SESSIO...

    2021

  47. [55]

    K., Aslett, K., Godel, W

    Tokita, C. K., Aslett, K., Godel, W. P., Sanderson, Z., Tucker, J. A., Nagler, J., Persily, N., & Bonneau, R. (2024). Measuring receptivity to misinformation at scale on a social media platform.PNAS Nexus,3(10).https://doi.org/10.1093/pnasnexus/pgae396

    2024 doi

  48. [56]

    Tuparova, E., Tagarev, A., Tulechki, N., & Boytcheva, S. (2022). Analyzing the Evo- lution of Disinformation Content on Facebook -a Pilot Study.https://ceur-ws.org/ Vol-3372/paper05.pdf 39

    2022

  49. [57]

    Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2019). GLUE: A MULTI-TASK BENCHMARK AND ANALYSIS PLATFORM FOR NATURAL LAN- GUAGE UNDERSTAND- ING.https://openreview.net/pdf?id=rJ4km2R5t7

    2019

  50. [58]

    Wardle, C., & Derakhshan, H. (2017). INFORMATION DISORDER : Toward an interdisciplinary framework for research and policy making Information Dis- order. ResearchGate.https://www.researchgate.net/publication/339031969_ INFORMATION_DISORDER_Toward_an_interdisciplinary_framework_...

    2017

  51. [59]

    Who Targets Me. (2018). Whotargets.me.https://whotargets.me/en/

    2018

  52. [60]

    outputs from sources that we can directly attribute to the Russian, Chinese, or Iranian governments or their various news and infor- mation channels

    Yang, D., Zhang, Z., & Zhao, H. (2023). Learning Better Masking for Better Language Model Pre-training. 8 Appendix 8.1 Disinformation Research Tool Analysis The RAND Corporation assembled a database of disinformation tools that have been released publicly (RAND Corporation, 20...

    2023

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.