arxiv: 2605.14354 · v1 · submitted 2026-05-14 · 💻 cs.CL

Recognition: no theorem link

LLM-based Detection of Manipulative Political Narratives

Sinclair Schneider , Florian Steuber , Gabi Dreo Rodosek

Authors on Pith no claims yet

Pith reviewed 2026-05-15 02:41 UTC · model grok-4.3

classification 💻 cs.CL

keywords manipulative narrativessocial mediaLLMfew-shot promptingunsupervised clusteringHDBSCANUMAPpolitical narratives

0 comments

The pith

An LLM few-shot prompt filters manipulative posts before unsupervised clustering identifies 41 distinct narrative clusters from 1.2 million social media posts.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes a framework that uses a detailed few-shot prompt in a reasoning model to filter social media posts for manipulative political narratives, separating them from legitimate critiques and event reframings. The filtered posts are embedded, reduced with UMAP, and clustered with HDBSCAN to find groups without any predefined categories. A reasoning model then interprets the narrative for each cluster. Tested on more than 1.2 million posts, the method uncovered 41 such clusters. This approach matters because political discussion has moved online, where spotting manipulation at scale is difficult without fixed lists of known tactics.

Core claim

We present a new computational framework for detecting and structuring manipulative political narratives. To achieve good clustering results, we filter manipulative posts beforehand using a detailed few-shot prompt that combines documented campaign narratives with legitimate criticisms to differentiate them. The remaining posts are subsequently embedded and dimensionality-reduced using UMAP, before HDBSCAN is applied to uncover narrative groups. Finally, a reasoning model is employed to uncover the narrative behind each cluster. This approach, applied to over 1.2 million social media posts, effectively identified 41 distinct manipulative narrative clusters by integrating prompt-based filter

What carries the argument

The integration of a few-shot LLM prompt for pre-filtering manipulative content with UMAP dimensionality reduction and HDBSCAN clustering on embeddings to discover narrative groups unsupervised.

If this is right

The method discovers narrative clusters independently of any predefined list of target categories.
Each identified cluster receives an interpretation from a reasoning model describing its narrative.
The pipeline scales effectively to datasets exceeding one million social media posts.
It handles the differentiation between manipulative reframings of real events and straightforward legitimate criticism through the prompt step.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Applying the same pipeline to time-stamped data could track the emergence and evolution of specific narrative clusters over time.
Extending the approach to other languages or additional social platforms could reveal cross-cultural patterns in manipulative discourse.
Pairing the cluster outputs with user engagement metrics might identify which narratives gain the most traction.

Load-bearing premise

The few-shot prompt can reliably separate manipulative political narratives from legitimate critiques and reframings of real events without systematic bias or high false-positive rates.

What would settle it

A human evaluation study annotating a representative sample of posts flagged as manipulative by the prompt, where the precision falls significantly below expected levels, would indicate the filter does not perform reliably.

Figures

Figures reproduced from arXiv: 2605.14354 by Florian Steuber, Gabi Dreo Rodosek, Sinclair Schneider.

**Figure 1.** Figure 1: The way from a FIMI campaign to a behavior change at the audience During a FIMI campaign, manipulative content, such as disinformation, is disseminated through channels such as Telegram, X, and Reddit to shape audience behavior. For example, a campaign might falsely claim that Ukraine is trafficking children to the West, reinforcing negative perceptions of Ukrainian corruption and depicting children as vi… view at source ↗

**Figure 2.** Figure 2: provides an overview of the data flow from raw data to narrative labels. All individual steps are described in Section 4. Raw Data Classification and Filtering Embedding Reduction Clustering Labeling [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** Figure 3: Row-normalized alignment matrix, highlighting the model’s high recall (91.7%) and its tendency to be stricter than human raters. cluded, and replacement samples were drawn until the balanced 200-post corpus was fully restored. In the second stage, a secondary evaluation of reasoning coherence was conducted. The rater was presented with the model’s final label alongside its generated reasoning to assess w… view at source ↗

read the original abstract

We present a new computational framework for detecting and structuring manipulative political narratives. A task that became more important due to the shift of political discussions to social media. One of the primary challenges thereby is differentiating between manipulative political narratives and legitimate critiques. Some posts may also reframe actual events within a manipulative context. To achieve good clustering results, we filter manipulative posts beforehand using a detailed few-shot prompt that combines documented campaign narratives with legitimate criticisms to differentiate them. This prompt enables a reasoning model to assign labels, retaining only manipulative narrative posts for further processing. The remaining posts are subsequently embedded and dimensionality-reduced using UMAP, before HDBSCAN is applied to uncover narrative groups. A key advantage of this unsupervised approach is its independence from a predefined list of target categories, enabling it to uncover new narrative clusters. Finally, a reasoning model is employed to uncover the narrative behind each cluster. This approach, applied to over 1.2 million social media posts, effectively identified 41 distinct manipulative narrative clusters by integrating prompt-based filtering with unsupervised clustering.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The pipeline combines LLM few-shot filtering with UMAP+HDBSCAN clustering on 1.2 million posts to surface 41 narrative groups without preset categories, but the filter step has no reported validation so the clusters are hard to trust.

read the letter

The main thing to know is that this paper puts together a practical pipeline: a detailed few-shot prompt filters manipulative political posts from 1.2 million social media items, the survivors get embedded and reduced with UMAP, HDBSCAN finds clusters, and a second LLM labels the resulting 41 groups. It avoids needing a fixed list of narrative types, which lets it discover whatever patterns are present rather than matching against known ones.

Referee Report

2 major / 1 minor

Summary. The manuscript proposes an LLM-based pipeline for detecting manipulative political narratives on social media. It first applies a detailed few-shot prompt to filter manipulative posts from a corpus of over 1.2 million posts (distinguishing them from legitimate critiques and reframed events), embeds the retained posts, reduces dimensionality with UMAP, runs HDBSCAN to discover 41 narrative clusters, and finally uses a reasoning model to interpret the narrative in each cluster. The central claim is that this unsupervised approach successfully uncovers distinct manipulative narrative groups without relying on predefined categories.

Significance. If the filtering and clustering stages can be shown to be reliable, the work would offer a scalable, open-ended method for surfacing emerging manipulative narratives at social-media scale. This would be a useful contribution to computational social science and misinformation research, particularly because it avoids fixed taxonomies and leverages LLMs for both filtering and interpretation.

major comments (2)

[Abstract] Abstract (pipeline description): no precision, recall, confusion matrix, or human-evaluation results are reported for the few-shot prompt filter on any held-out or annotated set. This is load-bearing for the central claim, because the 41 clusters are only interpretable as manipulative narratives if the filter reliably excludes legitimate critiques and reframed events; without these metrics the downstream HDBSCAN output cannot be validated.
[Clustering stage] Clustering and interpretation stages: the manuscript supplies no details on UMAP/HDBSCAN hyper-parameters, cluster-quality metrics (e.g., silhouette scores, stability across runs), or any manual validation of the 41 clusters. Consequently it is impossible to determine whether the discovered groups reflect genuine narrative structure or artifacts of the preceding LLM filter.

minor comments (1)

[Abstract] The abstract states that the prompt 'combines documented campaign narratives with legitimate criticisms' but does not reproduce the actual prompt text or the exact label schema used by the reasoning model; including the prompt in an appendix would improve reproducibility.

Simulated Author's Rebuttal

2 responses · 0 unresolved

Thank you for the constructive feedback on our manuscript. We have carefully considered the major comments and provide point-by-point responses below. We plan to revise the manuscript to address the concerns regarding validation of the filtering and clustering stages.

read point-by-point responses

Referee: [Abstract] Abstract (pipeline description): no precision, recall, confusion matrix, or human-evaluation results are reported for the few-shot prompt filter on any held-out or annotated set. This is load-bearing for the central claim, because the 41 clusters are only interpretable as manipulative narratives if the filter reliably excludes legitimate critiques and reframed events; without these metrics the downstream HDBSCAN output cannot be validated.

Authors: We agree that quantitative evaluation of the few-shot filtering prompt is essential to validate the pipeline. The original manuscript focused on the novel unsupervised clustering approach and omitted detailed metrics for the filter due to space constraints and emphasis on the discovery aspect. In the revised version, we will include precision, recall, and F1 scores based on a human-annotated held-out set, along with a confusion matrix and details of the annotation process. This will strengthen the claim that the retained posts are indeed manipulative narratives. revision: yes
Referee: [Clustering stage] Clustering and interpretation stages: the manuscript supplies no details on UMAP/HDBSCAN hyper-parameters, cluster-quality metrics (e.g., silhouette scores, stability across runs), or any manual validation of the 41 clusters. Consequently it is impossible to determine whether the discovered groups reflect genuine narrative structure or artifacts of the preceding LLM filter.

Authors: We acknowledge the lack of hyperparameter details and validation metrics in the current version. To address this, the revised manuscript will report the specific UMAP parameters (e.g., n_neighbors, min_dist) and HDBSCAN settings (e.g., min_cluster_size, min_samples), along with cluster quality metrics such as silhouette scores and Davies-Bouldin index. Additionally, we will include results from stability analysis across multiple runs and a summary of manual inspection of the 41 clusters to confirm they represent coherent narrative structures rather than artifacts. revision: yes

Circularity Check

0 steps flagged

No circularity: sequential pipeline of independent standard components

full rationale

The paper presents a linear pipeline consisting of a few-shot LLM prompt for filtering manipulative posts, followed by UMAP embedding, HDBSCAN clustering, and a second LLM step for cluster interpretation. No equations, fitted parameters, or self-referential definitions appear in the derivation; the filtering prompt is described as an external input combining documented narratives with criticisms, and clustering is performed with off-the-shelf unsupervised methods. No self-citations are invoked to justify uniqueness or load-bearing premises, and no predictions are constructed by renaming fitted inputs. The absence of validation metrics for the filter is a limitation of empirical support rather than a circular reduction of the claimed output to its inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The framework rests on the assumption that current reasoning LLMs can perform reliable binary classification of manipulative versus legitimate content from few-shot examples and that semantic embeddings plus density-based clustering will produce coherent narrative groups.

axioms (2)

domain assumption Large language models can accurately distinguish manipulative political narratives from legitimate critiques when given documented examples in a few-shot prompt.
This is invoked in the filtering stage described in the abstract.
domain assumption UMAP-reduced embeddings preserve enough semantic structure for HDBSCAN to recover meaningful narrative clusters.
This is the basis for the unsupervised grouping step.

pith-pipeline@v0.9.0 · 5478 in / 1436 out tokens · 35569 ms · 2026-05-15T02:41:49.936450+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

31 extracted references · 31 canonical work pages · 3 internal anchors

[1]

Alaphilippe, A., Machado, G., Miguel, R., Poldi, F.: Doppelganger: Me- dia clones serving Russian propaganda. Tech. rep., EU DisinfoLab (2022), https://perma.cc/73QF-WJEB, accessed: 2026-05-11

work page 2022
[2]

Bryjka, F.: Unravelling Russia’s Network of Influence Agents in Europe. Tech. rep., Polish Institute of International Affairs (PISM) (2024), https://perma.cc/8CPM- VUKZ, accessed: 2026-05-04

work page 2024
[3]

In: Advances in Knowledge Discovery and Data Mining

Campello, R.J.G.B., Moulavi, D., Sander, J.: Density-Based Clustering Based on Hierarchical Density Estimates. In: Advances in Knowledge Discovery and Data Mining. vol. 7819, pp. 160–172. Springer Berlin Heidelberg (2013). https://doi.org/10.1007/978-3-642-37456-2_14

work page doi:10.1007/978-3-642-37456-2_14 2013
[4]

Delmer, S.: Black Boomerang: An Autobiography, vol. 2. Secker & Warburg (1962)

work page 1962
[5]

European External Action Service: 1st EEAS Report on Foreign Information Ma- nipulation and Interference Threats. Tech. rep., European External Action Service (2023), https://perma.cc/AFN2-3V27, accessed: 2026-05-04

work page 2023
[6]

European External Action Service: Disinfo: Ukraine is a neo-Nazi Russophobic state (2024), https://perma.cc/8EL8-KNXQ, published by EUvsDisinfo, accessed: 2026-04-19

work page 2024
[7]

European External Action Service: 3rd EEAS Report on Foreign Information Ma- nipulation and Interference Threats. Tech. rep., European External Action Service (2025), https://perma.cc/8YDX-MQXU, accessed: 2026-05-04

work page 2025
[8]

European External Action Service: Disinfo: Ukrainian children are Zelenskyy’s main export commodity (2025), https://perma.cc/A8N8-XEBP, published by EU- vsDisinfo, accessed: 2026-04-18

work page 2025
[9]

BERTopic: Neural topic modeling with a class-based TF-IDF procedure

Grootendorst, M.: BERTopic: Neural topic modeling with a class- based TF-IDF procedure. arXiv preprint arXiv:2203.05794 (2022). https://doi.org/10.48550/arXiv.2203.05794

work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2203.05794 2022
[10]

In: Proceedings of the International AAAI Conference on Web and Social Media

Hanley, H.W.A., Kumar, D., Durumeric, Z.: Happenstance: Utilizing Semantic SearchtoTrackRussianStateMediaNarrativesabouttheRusso-UkrainianWaron Reddit. In: Proceedings of the International AAAI Conference on Web and Social Media. vol. 17, pp. 327–338 (2023). https://doi.org/10.1609/icwsm.v17i1.22149

work page doi:10.1609/icwsm.v17i1.22149 2023
[11]

In: Proceedings of the Interna- tional AAAI Conference on Web and Social Media

Haouari, F., Scarton, C., Faggiani, N., Nikolaidis, N., Kotseva, B., Abu Farha, I., Linge, J., Bontcheva, K.: UKElectionNarratives: A Dataset of Misleading Narra- tives Surrounding Recent UK General Elections. In: Proceedings of the Interna- tional AAAI Conference on Web and Social Media. vol. 19, pp. 2477–2495 (2025). https://doi.org/10.1609/icwsm.v19i1.35950

work page doi:10.1609/icwsm.v19i1.35950 2025
[12]

In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

Heppell, F., Bontcheva, K., Scarton, C.: Analysing State-Backed Propaganda Web- sites: A New Dataset and Linguistic Study. In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. pp. 5729–5741. Associa- tion for Computational Linguistics (2023), https://doi.org/10.18653/v1/2023.emnlp-main.349

work page doi:10.18653/v1/2023.emnlp-main.349 2023
[13]

InProceedings of the 29th Symposium on Operating Systems Principles(Koblenz, Germany)(SOSP ’23)

Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C.H., Gonzalez, J., Zhang, H., Stoica, I.: Efficient memory management for large language model serv- ing with PagedAttention. In: Proceedings of the 29th Symposium on Operating Systems Principles. pp. 611–626. Association for Computing Machinery (2023). https://doi.org/10.1145/3600006.3613165

work page doi:10.1145/3600006.3613165 2023
[14]

Linvill, D., Warren, P.: Infektion’s Evolution: Digital Technologies and Narrative Laundering. Tech. Rep. 3, Clemson University (2023), https://perma.cc/7BAW- B4EC, accessed: 2026-05-10

work page 2023
[15]

UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction

McInnes, L., Healy, J., Melville, J.: UMAP: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426 (2020). https://doi.org/10.48550/arXiv.1802.03426

work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.1802.03426 2020
[16]

Miskimmon, A., O’Loughlin, B., Roselle, L.: Strategic Narratives: Communication PowerandtheNewWorldOrder.No.3inRoutledgeStudiesinGlobalInformation, Politics and Society, Routledge (2013)

work page 2013
[17]

In: Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics

Muennighoff, N., Tazi, N., Magne, L., Reimers, N.: MTEB: Massive Text Embed- ding Benchmark. In: Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. pp. 2014–2037. Association for Computational Linguistics (2023), https://doi.org/10.18653/v1/2023.eacl-main.148

work page doi:10.18653/v1/2023.eacl-main.148 2014
[18]

BBC News (2024), https://perma.cc/J3BN- 9UR5, accessed: 2026-05-10

Myers,P.,Robinson,O.,Sardarizadeh,S.,Wendling,M.:ABugatti,afirstladyand the fake stories aimed at Americans. BBC News (2024), https://perma.cc/J3BN- 9UR5, accessed: 2026-05-10

work page 2024
[19]

Zeitschrift für Rechtsextremismusforschung2(1), 91–109 (2022)

Müller, P.: Extrem rechte influencer*innen auf telegram: Normalisierungsstrategien in der corona-pandemie. Zeitschrift für Rechtsextremismusforschung2(1), 91–109 (2022). https://doi.org/10.3224/zrex.v2i1.06

work page doi:10.3224/zrex.v2i1.06 2022
[20]

Nimmo, B., Torrey, M.: Taking down coordinated inauthentic behavior from Russia and China. Tech. rep., Meta (2022), https://perma.cc/6Z78-FZLT, accessed: 2026- 05-05

work page 2022
[21]

https://perma.cc/8P3E-J72X (2026), official Blog Post, accessed: 2026-05-10

Qwen Team: Qwen3.5: Towards Native Multimodal Agents. https://perma.cc/8P3E-J72X (2026), official Blog Post, accessed: 2026-05-10

work page 2026
[22]

Reimers, N., Gurevych, I.: Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natu- ralLanguageProcessing.pp.3982–3992.AssociationforComputationalLinguistics (2019). https://doi.org/10.18653/v1/D19-1410

work page doi:10.18653/v1/d19-1410 2019
[23]

arXiv preprint arXiv:2407.08417 (2024)

Schäfer, K., Choi, J.E., Vogel, I., Steinebach, M.: Unveiling the Potential of BERTopic for Multilingual Fake News Analysis – Use Case: Covid-19. arXiv preprint arXiv:2407.08417 (2024). https://doi.org/10.48550/arXiv.2407.08417

work page doi:10.48550/arxiv.2407.08417 2024
[24]

In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Pro- cessing

Sosnowski, W., Modzelewski, A., Skorupska, K., Wierzbicki, A.: DiNaM: Dis- information Narrative Mining with Large Language Models. In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Pro- cessing. pp. 30212–30239. Association for Computational Linguistics (2025). https://doi.org/10.18653/v1/2025.emnlp-main.1537

work page doi:10.18653/v1/2025.emnlp-main.1537 2025
[25]

BBC News (2022), https://perma.cc/H879-XEW6, accessed: 2026-05-10

Spring, M.: Marianna Vyshemirsky: ’My picture was used to spread lies about the war’. BBC News (2022), https://perma.cc/H879-XEW6, accessed: 2026-05-10

work page 2022
[26]

In: Proceedings of the International AAAI Conference on Web and Social Media

Steffen, E.: More than Memes: A Multimodal Topic Modeling Approach to Conspiracy Theories on Telegram. In: Proceedings of the International AAAI Conference on Web and Social Media. vol. 19, pp. 1831–1844 (2025). https://doi.org/10.1609/icwsm.v19i1.35904

work page doi:10.1609/icwsm.v19i1.35904 2025
[27]

TIME (2024), https://perma.cc/CZ7B-77W6, accessed: 2026-05-10

Syed, A.: How Online Misinformation Stoked Anti-Migrant Riots in Britain. TIME (2024), https://perma.cc/CZ7B-77W6, accessed: 2026-05-10

work page 2024
[28]

Department of State: How the People’s Republic of China Seeks to Re- shape the Global Information Environment

U.S. Department of State: How the People’s Republic of China Seeks to Re- shape the Global Information Environment. Tech. rep., Global Engagement Center (GEC) (2023), https://perma.cc/E4CF-7JZ5, accessed: 2026-05-05

work page 2023
[29]

VIGINUM: RNN: A complex and persistent information manipulation campaign. Tech. rep., SGDSN (2024), https://perma.cc/JNZ9-JN75, accessed: 2026-05-04

work page 2024
[30]

Wardle, C., Derakhshan, H.: Information disorder: Toward an interdisciplinary framework for research and policy making. Tech. rep., Council of Europe (2017), https://perma.cc/U3JD-BSAE, accessed: 2026-05-05

work page 2017
[31]

Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models

Zhang, Y., Li, M., Long, D., Zhang, X., Lin, H., Yang, B., Xie, P., Yang, A., Liu, D., Lin, J., Huang, F., Zhou, J.: Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models. arXiv preprint arXiv:2506.05176 (2025). https://doi.org/10.48550/arXiv.2506.05176

work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2506.05176 2025