REVIEW 3 major objections 4 minor 14 references
Echoes in the Sky: Computational Thematic Analysis of Online Public Discourse on Bluesky Across Trump's Reelection
T0 review · 3 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read The paper claims that 30.2 million English-language Bluesky posts about Trump can be organized into 14 interpretable discourse themes, and that the same hybrid coding of 258 executive orders yields 19 thematic categories, with issue…
desk verdict A genuinely useful large Bluesky dataset and mostly transparent pipeline, but the theme-level percentages rest on the clustered third of the corpus and the paper has not shown that third is representative. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the microtopic discovery and theme-generation pipeline: each post is embedded, dimension-reduced, density-clustered, and each resulting microtopic is titled and described by a large language model, then reviewed by human researchers. The microtopics are converted into initial codes by an LLM using a seed codebook, and researchers iteratively merge and refine these codes into the final 14 overarching themes. A separate but analogous hybrid coding, seeded by a policy-agendas codebook, maps the 258 executive orders into 19 categories. The structural break analysis of daily post-volume time series is the tool that turns the theme annotations into a temporal claim about event-sensitive attention.
What would settle it
Run the same embedding, clustering, and theme-assignment pipeline on a random sample of the roughly 19.7 million posts that were not assigned to microtopics; if those posts show a different sentiment split or different dominant topics than the 10.5 million clustered posts, the reported theme shares and sentiment trends would have to be revised.
Extended reading notes
Core claim
The central claim, stated on the paper's own terms, is that Trump-related discourse on Bluesky has a discoverable thematic anatomy: 4,770 fine-grained microtopic clusters produced by an LLM-assisted clustering pipeline, validated by human review, organize 10.5 million posts into 14 overarching themes, with executive power and partisan conflict the most prominent. The same hybrid coding of the 258 executive orders signed between January 20, 2025 and May 1, 2026 produces 19 categories, dominated by administrative task forces, border and crime enforcement, foreign policy, and national security. The paper also claims that monthly theme volumes reveal persistent core topics alongside episodic spikes, and that a structural break analysis of immigration discourse places breakpoints near executive orders such as the April 2025 criminal-alien order and the August 2025 D.C. crime-emergency order. Sentiment analysis with a lexicon-based classifier assigns 47 percent negative, 17 percent neutral, and 36 percent positive sentiment overall and documents a widening negative-positive gap over time.
Load-bearing premise
The analysis treats the 10.5 million posts that clustering assigned to identifiable microtopics as standing in for the full 30.2 million-post corpus, yet it never checks whether the roughly 19.7 million unclustered posts differ systematically.
Editorial extensions
If this is right
- The public 30.2-million-post corpus and 14-theme annotation let other researchers track issue attention on Bluesky without recollecting the data from scratch.
- Monthly theme volumes can be aligned with future executive actions to detect attention spikes, making the approach a template for real-time policy-discourse monitoring.
- The 19 executive-order categories allow direct comparisons between policy emphasis and public discussion emphasis, revealing which policy areas receive disproportionate online attention.
- The structural break result suggests immigration discourse is episodic, switching abruptly around discrete policy events rather than drifting gradually.
- The widening negative sentiment gap, from about 2 percent in August 2024 to 15 percent by January 2026, indicates that critical reactions became increasingly dominant in Trump-related Bluesky conversation.
Reading between the lines
- The same microtopic pipeline could be applied to non-English posts or to other platforms, since the method itself is not language-bound and the authors explicitly chose to restrict to English.
- The event-sensitivity finding suggests a predictive design: executive-order signing dates could be used to forecast abrupt increases in issue-specific attention on Bluesky, though the paper only describes the relationship retrospectively.
- The roughly 19.7 million unclustered posts could be analyzed with a cheaper classifier to test whether the 10.5 million clustered posts are representative; if they differ, every theme-share estimate would need revision.
- Applying structural break detection to individual microtopics, rather than only to grouped themes, might pinpoint which specific events trigger attention shifts at a finer granularity than the paper reports.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents a large-scale computational analysis of Trump-related Bluesky posts, reporting 38.5 million collected posts from January 2019 to February 2026, reduced to 30.2 million cleaned English-language posts. The authors use an LLM-assisted clustering pipeline with human validation to identify 4,770 microtopics and 14 overarching discourse themes, and separately code 258 executive orders into 19 thematic categories. They report sentiment proportions from VADER (47.0% negative, 17.3% neutral, 35.7% positive), describe theme-level sentiment composition, analyze monthly theme dynamics, and apply structural break detection to identify shifts in attention. The core claims are that Bluesky discourse is simultaneously persistent and event-sensitive, and that negative sentiment grows over time, particularly after January 2025.
Significance. If the results hold, this would be a valuable descriptive resource for studying political discourse on an increasingly important platform, and the public release of the dataset and pipeline is a concrete contribution. The paper's strengths include the scale of the corpus, the explicit human validation of the thematic codebook, the transparent pipeline description, and the acknowledgment of several limitations. However, the central theme-level findings rest on a clustered subset covering only about one-third of the cleaned corpus, and the paper does not establish that this subset is representative. This selection issue is load-bearing for the theme distributions, theme-level sentiment composition, and structural break timing claims, so the current version requires substantive additional analysis rather than copyediting.
major comments (3)
- [Section 4.2, Figures 3 and 4] The theme distribution and all theme-level percentages are computed from the 10,456,083 posts assigned to microtopics, which is 34.6% of the 30.2 million cleaned posts. The paper offers no comparison between clustered and unclustered posts on length, sentiment, author characteristics, or temporal coverage, so the implicit assumption that unclustered posts are missing at random is unsupported. HDBSCAN's noise label is systematic rather than random, and the unclustered majority could plausibly be shorter, more generic, more or less event-focused, or drawn from different user populations. This would bias every theme share, the per-theme sentiment composition in Figure 3, and the monthly theme volumes in Figure 4. The authors should report coverage by temporal partition, compare clustered and unclustered posts on observable features, and provide a sensitivity analysis (for example, reweighting or keyword-based assignment of unclustered posts) to demonstrate that the 14-theme characterization is robust to the missing 65.4%.
- [Section 4.4, Figure 5] The structural break analysis constructs daily post-volume time series for each Bluesky theme from the same clustered subset. If the fraction of posts assigned to microtopics varies across time—which is plausible given the four temporal partitions and the platform's rapid growth—the estimated break dates may reflect changes in clustering coverage rather than changes in public discourse. For example, Figure 5's breaks in immigration discourse could coincide with periods when more posts happened to form dense clusters. The authors should plot the proportion of posts assigned to microtopics over time and rerun break detection on coverage-adjusted series, or on full-corpus series such as keyword-based volumes or sentiment totals, before attributing the breaks to executive orders and media coverage.
- [Section 3.1, keyword lexicon] The keyword lexicon used for retrieval includes 'Republican' and 'maga' without any precision or recall validation. Because the abstract and research questions describe the corpus as 'Trump-related,' posts retrieved solely on the keyword 'Republican' may concern other political races, non-Trump candidates, or generic partisan topics, which would blur both the thematic and sentiment results. A small hand-coded precision sample, reported per keyword, would establish that the retrieval definition matches the intended construct. This is a load-bearing validity issue for the dataset's interpretation as Trump-specific discourse.
minor comments (4)
- [Section 5, Conclusion] The concession that VADER 'does not distinguish supportive from opposing sentiment' is only in the conclusion; it should also appear in Section 4.1 and be reflected in the abstract, since the claim of 'increasing proportion of negative sentiment' conflates criticism of Trump with negative affect more generally.
- [Section 3.1, Overarching Theme Generation] The 17 seed codes were derived from a topic model of the microtopic summaries, which are themselves products of the same clustering pipeline. This creates an anchoring risk in the subsequent coding step; the paper should acknowledge this explicitly and describe how the human reviewers guarded against simply reproducing the seed structure.
- [Figure 3] The left panel's percentages should state their denominator explicitly (theme-assigned posts rather than all cleaned posts), and the caption should clarify that sentiment composition is also computed only for clustered posts.
- [Table 1] The column labeled 'Part.' refers to temporal partitions A–D, but the caption does not define the date ranges of these partitions. Adding the ranges would make the table interpretable without consulting the supplementary material.
Circularity Check
No circular derivation found; the central measures (sentiment, clusters, theme shares, structural breaks) are computed from the corpus with external tools and human validation, and the main weakness is selective microtopic coverage, which is a validity threat rather than a circularity.
full rationale
I walked the derivation chain from data collection through sentiment analysis, microtopic clustering, theme induction, EO coding, and structural break detection. None of the central claims reduces to its own inputs by construction. Sentiment shares come from VADER applied to cleaned post text (Sections 3.1 and 4.1), an external lexicon-based tool, not from parameters fitted to the claims. Microtopic clusters are produced by Qwen3-Embedding-8B, UMAP, and HDBSCAN; the 4,770 microtopics and their theme labels are induced by LLM summaries plus two-researcher validation. The 17 seed codes were indeed derived from a topic model of the same microtopic summaries (Section 3.1, Overarching Theme Generation), which could anchor the later coding, but the paper explicitly allows GPT-5.4 mini to generate new codes and two researchers iteratively refine codes and definitions; the final 14 themes are not defined by the seed codebook, so this is at most a mild anchoring bias, not a self-definitional equivalence. EO themes use the external Policy Agendas Project codebook (Section 3.2). Structural breaks use the R strucchange package on daily post volumes (Section 3.3), with break dates determined by BIC rather than by the narrative of the EO events. The paper cites prior work by its own authors: [11] for the Bluesky microtopic framework, [12] for LLM-assisted thematic analysis provenance, and [13]/[14] for LLM method context. These are method-provenance citations, not uniqueness theorems or fitted inputs; the paper's descriptive findings (theme distributions, sentiment trends, break points) do not depend on accepting any contested result from those papers. The reader's weakest-point concern is real but not circular: Section 4.2 reports that 10,456,083 of roughly 30.2 million cleaned posts were assigned to identifiable microtopics, and Figure 3 (left) and Figure 4 compute theme shares only from that clustered subset, leaving about 19.7 million posts as unclustered observations. If unclustered posts differ systematically, theme-level percentages and temporal patterns are biased. This is a representativeness and missingness problem, and the limitation paragraph in Section 5 does not acknowledge it. However, it is not a circularity: the theme labels, sentiment scores, and break dates are not constructed to reproduce the unclustered subset, and no equation defines the reported results in terms of the clustered subset.
Assumptions & free parameters
free parameters (6)
- Keyword lexicon (10 terms)
- Minimum post length (n > 6 words)
- VADER compound sentiment thresholds =
default VADER thresholds
- UMAP/HDBSCAN hyperparameters =
reported in SM Table 2
- Number of overarching themes (14) =
14 (from 10-25 target range)
- BIC penalty for structural breaks
assumptions (5)
- domain assumption The Bluesky search API returns all available posts matching the keyword lexicon.
- domain assumption English-language posts are representative of all Trump-related Bluesky discourse.
- domain assumption VADER compound scores correspond to sentiment categories for political text.
- domain assumption UMAP and HDBSCAN on Qwen3 embeddings produce semantically meaningful microtopics.
- domain assumption Post volumes from different dates are comparable without adjusting for platform or user growth.
Cite this review
Pith. "Pith review of Echoes in the Sky: Computational Thematic Analysis of Online Public Discourse on Bluesky Across Trump's Reelection." pith.science (2026). https://pith.science/paper/TEVDEMYB
@misc{pith2026260804120,
author = {Pith},
title = {Pith review of: Echoes in the Sky: Computational Thematic Analysis of Online Public Discourse on Bluesky Across Trump's Reelection},
year = {2026},
howpublished = {\url{https://pith.science/paper/TEVDEMYB}},
note = {Machine review of arXiv:2608.04120}
}
read the original abstract
As political disruption intensifies online discourse, Bluesky has become an important platform for political discussion and public reaction. In this study, we examine large-scale discourse on Bluesky related to U.S. policy developments associated with the Trump administration. Using the historical retrieval API, we collected all available posts matching Trump and related keywords from 2019 to 2026, yielding 38.5 million posts. We leverage a large language model (LLM)-assisted clustering pipeline, combined with human validation, to identify 14 interpretable thematic domains in English-language posts and 19 thematic categories across 258 executive orders (EOs) signed between January 20, 2025, and May 1, 2026. Our findings identify several dominant themes in Bluesky discourse, including executive governance, political identity, and national security, as well as recurring themes in EOs, including executive task forces, border enforcement, and foreign policy. We also find substantial variation in the persistence and volatility of issue attention, accompanied by an increasing proportion of negative sentiment over time. The dataset and resources are publicly available at https://github.com/Sensify-Lab/Echoes-in-the-Sky
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
In: International Conference on 10 Wang et al
Baik, K., Kim, J.: Analyzing the discourse around russo-ukrainian war in ger- many: Understanding variances in public stances. In: International Conference on 10 Wang et al. Advances in Social Networks Analysis and Mining. pp. 315–331. Springer (2025). https://doi.org/10.31235/osf.io/qe4m3_v1
-
[2]
Barberá, P., Jost, J.T., Nagler, J., Tucker, J.A., Bonneau, R.: Tweeting from left to right: Is online political communication more than an echo chamber? Psychological science26(10), 1531–1542 (2015).https://doi.org/10.1177/0956797615594620
-
[3]
Scientific reports 8(1), 8673 (2018).https://doi.org/10.1038/s41598-018-26951-y
Bovet, A., Morone, F., Makse, H.A.: Validation of twitter opinion trends with national polling aggregates: Hillary clinton vs donald trump. Scientific reports 8(1), 8673 (2018).https://doi.org/10.1038/s41598-018-26951-y
-
[4]
Qualita- tive research in psychology3(2), 77–101 (2006).https://doi.org/10.1191/ 1478088706qp063oa
Braun, V., Clarke, V.: Using thematic analysis in psychology. Qualita- tive research in psychology3(2), 77–101 (2006).https://doi.org/10.1191/ 1478088706qp063oa
work page 2006
-
[5]
In: Proceedings of the international aaai con- ference on web and social media
Conover, M., Ratkiewicz, J., Francisco, M., Gonçalves, B., Menczer, F., Flammini, A.: Political polarization on twitter. In: Proceedings of the international aaai con- ference on web and social media. vol. 5, pp. 89–96 (2011).https://doi.org/10. 1609/icwsm.v5i1.14126
work page 2011
-
[6]
Gkikas, D.C., Tzafilkou, K., Theodoridis, P.K., Garmpis, A., Gkikas, M.C.: How do text characteristics impact user engagement in social media posts: Modeling contentreadability,length,andhashtagsnumberinfacebook.InternationalJournal of Information Management Data Insights2(1), 100067 (2022).https://doi.org/ 10.1016/j.jjimei.2022.100067
arXiv 2022
-
[7]
Political psychology39, 85–118 (2018)
Jost, J.T., Barberá, P., Bonneau, R., Langer, M., Metzger, M., Nagler, J., Ster- ling, J., Tucker, J.A.: How social media facilitates political protest: Informa- tion, motivation, and social networks. Political psychology39, 85–118 (2018). https://doi.org/10.1111/pops.12478
-
[8]
Online social networks and media24, 100149 (2021).https://doi.org/10.1016/j.osnem.2021.100149
Kejriwal, M., Wang, Q., Li, H., Wang, L.: An empirical study of emoji usage on twitter in linguistic and national contexts. Online social networks and media24, 100149 (2021).https://doi.org/10.1016/j.osnem.2021.100149
arXiv 2021
Show all 14 references
-
[9]
Social Network Analy- sisandMining15(1), 91(2025).https://doi.org/10.1007/s13278-025-01504-0
Paoletti, G., Ferreira, C.H., Vassio, L., Rocha, L., Almeida, J.M.: Tracing the 2024 us election debate on telegram with llms and graph analysis. Social Network Analy- sisandMining15(1), 91(2025).https://doi.org/10.1007/s13278-025-01504-0
2025 doi
-
[10]
PloS one20(2), e0318034 (2025).https://doi.org/10.1371/journal
Quelle, D., Bovet, A.: Bluesky: Network topology, polarization, and algorithmic curation. PloS one20(2), e0318034 (2025).https://doi.org/10.1371/journal. pone.0318034
2025 doi
-
[11]
Salloum, A., Quelle, D., Iannucci, L., Bovet, A., Kivelä, M.: Politics and polariza- tion on bluesky (2025),https://arxiv.org/abs/2506.03443
2025 arXiv
-
[12]
Proceedings of the ACM on Human-Computer Interaction9(2), 1–28 (2025)
Wang, Q., Erqsous, M., Barner, K.E., Mauriello, M.L.: Lata: A pilot study on llm-assisted thematic analysis of online social network data generation experi- ences. Proceedings of the ACM on Human-Computer Interaction9(2), 1–28 (2025). https://doi.org/10.1145/3711022
2025 doi
-
[13]
In: Proceedings of the International AAAI Conference on Web and Social Media
Wang, Q., Erqsous, M., Khatiwada, P., Karwankar, A., Alhassan, F.M., Chan- drasekaran, A., Abraham, B., Lovell, F., Ngo, A.A., Mauriello, M.L.: Leverag- ing large language models for review classification and rating estimation of men- tal health applications. In: Proceedings o...
2025
-
[14]
election-related harmful social media content (2026),https://arxiv.org/abs/ 2602.11962
Wang, Q., Khatiwada, P., Vieira, C.C., Bagozzi, B.E., Barner, K.E., Mauriello, M.L.: Wisdom of the llm crowd: A large scale benchmark of multi-label u.s. election-related harmful social media content (2026),https://arxiv.org/abs/ 2602.11962
2026
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.