How is science discussed on Bluesky?

Er-Te Zheng; Mike Thelwall; Xiaorui Jiang; Zhichao Fang

REVIEW 2 major objections 4 minor 7 references

How is science discussed on Bluesky?

T0 review · 2 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Bluesky science posts beat X on engagement and original commentary

desk verdict A solid descriptive baseline of scholarly posting on Bluesky, but the headline comparison with X rests on unmatched prior-study numbers and should be either recomputed or softened before publication. read the letter →

arxiv 2507.18840 v2 pith:UOFFMDAL submitted 2025-07-24 cs.DL cs.CY

Er-Te Zheng , Xiaorui Jiang , Zhichao Fang , Mike Thelwall This is my paper · ORCID

classification cs.DLcs.CY

keywords BlueskyaltmetricssciencecommunicationscholarlysocialmediametricsOpenAlextextualsimilarityuserengagement

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether Bluesky, the decentralized social platform that many academics migrated to after leaving X, can serve as a credible venue for science communication and as a new source of altmetrics. Analyzing more than 2.6 million Bluesky posts referencing over 530,000 scholarly articles, it finds that scholarly activity on Bluesky surged sharply between November 2024 and January 2025 as researchers left X. The paper's central claim is that Bluesky posts attract substantially more likes, reposts, replies, and quotes than previously reported for X, and that Bluesky posts are more textually original, less often copying article titles verbatim. If true, this means Bluesky is not merely a replacement for X but a place where scientific discourse is more participatory and more interpretive.

What carries the argument

The machinery is a three-stage data-collection and measurement pipeline: first, scholarly post URLs and DOIs were pulled from the Altmetric Explorer; second, post metadata and engagement counts were retrieved through the official Bluesky API; third, article metadata and disciplinary classifications were matched through the OpenAlex database. The two central measurement instruments are complementary cumulative distribution functions of likes, reposts, replies, and quotes, and cosine similarity scores between cleaned post text and the referenced article's title, computed from TF-IDF vectors. The cosine similarity score supplies the operational definition of textual originality, distinguishing posts that merely repeat a title from those that summarize, comment on, or reinterpret the research.

What would settle it

A matched-baseline study would settle the matter: collect Bluesky and X posts linking to the same set of scholarly articles over the same calendar months, compute the same engagement thresholds and cosine-similarity-to-title scores for both platforms, and compare. If the engagement and originality gaps shrink to near zero under matching, the paper's headline claim that Bluesky discourse is more interactive and more interpretive would be unsupported.

Watch

Extended reading notes

Core claim

The paper establishes that scholarly discourse on Bluesky has become both more interactive and more interpretive than comparable discourse previously reported for X. Specifically, 48.2% of scholarly Bluesky posts received at least ten likes and 34.4% at least ten reposts, versus prior X findings of 3.9-7.5% and 1.4-4.4% respectively; replies and quotes on Bluesky likewise occur at rates up to two orders of magnitude higher than earlier X reports. Only 6.3% of Bluesky posts nearly replicate an article title, compared with earlier X title-replication rates ranging from 11.8% to 92.4%. The paper also documents that 50.7% of scholarly Bluesky posts appear within one week of publication, that health, social, and environmental sciences dominate the discourse, and that 91% of posts are in English, leading the authors to position Bluesky as a promising altmetrics source and a stable post-X venue for science communication.

Load-bearing premise

The paper's central comparison assumes that engagement and title-repetition statistics from earlier studies of X are directly comparable to the new Bluesky numbers, even though the two sets of numbers come from different time periods, sampling methods, user populations, and platform mechanics.

Editorial extensions

If this is right

Bluesky can serve as a bona fide altmetrics data source, capturing early post-publication attention in a way that complements traditional citations.
Science communication metrics built on Bluesky may reflect more substantive engagement than metrics built on X, because the platform's posts skew toward commentary and interpretation rather than title repetition.
The sharp post-migration surge indicates that altmetric aggregators tracking Bluesky from late 2024 will record a genuinely growing stream of scholarly attention, not an artifact of indexing alone.
Disciplinary and linguistic biases observed on X, such as English dominance and the prominence of health, social, and environmental sciences, carry over to Bluesky, meaning new-platform altmetrics inherit the same coverage limitations.
Because most posts appear within a week of publication, Bluesky can be used for near-real-time monitoring of research visibility.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the engagement gap between Bluesky and X were re-tested with matched samples (same articles, same days, same types of users), the gap could shrink; the paper's comparison rests on numbers from different studies, time periods, and sampling designs.
The higher textual originality may partly reflect Bluesky's early-adopter user base of active academics rather than a permanent feature of the platform; as users grow more diverse, title-copying behavior could rise.
A testable extension would be to compare Bluesky engagement rates before and after November 2024 to see whether the migration itself, rather than platform culture, drives the interactive style the paper documents.
Because the dataset excludes posts that discuss articles without including a link, the originality and engagement numbers are estimates for link-bearing discourse only; full-text conversations on Bluesky remain unmeasured.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First, what this paper actually gives you: the first large-scale picture of how scholarly articles circulate on Bluesky — 2.6 million posts, 530,000 articles, with a transparent pipeline from Altmetric and OpenAlex, an independent API check in the appendix that corroborates the November 2024–January 2025 surge, and topic-level data on Figshare. For anyone tracking where altmetrics is heading, the descriptive core is genuinely useful and the temporal story is convincing. The finding that most posts appear within a week of publication, and the discipline/language distributions, are exactly the kind of baseline the field needs right now.

The soft spot is exactly where the reader's stress test lands: the comparative claims in Section 4.1 and the abstract. The “substantially higher engagement” line compares Bluesky's own numbers to X statistics from Fang et al. (2022), collected in a different period, from a different aggregator, under different platform mechanics, and the title-originality comparison is even messier — the 6.3% near-verbatim figure is TF-IDF cosine ≥0.9 after stripping usernames/hashtags/URLs, while the cited X rates come from studies using exact or near-duplicate matching with heterogeneous definitions. That is not an internal inconsistency, but it is an external-validity gap. The paper's own limitations section is honest about data omissions and normalization issues, but it never flags this mismatch. The fix is straightforward: either re-compute X baselines with the same collection and metric definitions in a contemporaneous window, or drop the direct comparison and frame the engagement/originality numbers as descriptive. As written, the headline claim is unproven.

Minor points: no code or full data released (only topic-level data), and the language analysis relies on Bluesky's own `langs` field, which users can override — a small uncertainty but not a distorting one for the main conclusions.

This paper is for altmetrics researchers, science-of-science people, and anyone thinking about replacing X as a source of social media metrics. It deserves a serious referee: the descriptive contribution is new, the data work is careful, and the comparative weakness is repairable. I would send it to peer review expecting major revision on the X comparison, not rejection.

Referee Report

2 major / 4 minor

Summary. This paper presents the first large-scale empirical study of how scholarly articles are discussed on Bluesky, analyzing over 2.6 million posts referencing more than 530,000 articles from January 2023 to July 2025. The authors collect post metadata via the Bluesky API (with post URLs from Altmetric) and article metadata from OpenAlex, then describe temporal trends, disciplinary coverage, language use, textual similarity between posts and article titles, and engagement metrics. They report a sharp increase in scholarly activity between November 2024 and January 2025, corroborated by an independent API-based check in the Appendix. The paper's headline comparative claims are that Bluesky posts show 'substantially higher levels of interaction' and 'greater textual originality' than previously reported for X, suggesting that Bluesky supports more interactive and interpretive scholarly communication.

Significance. If the descriptive conclusions hold, this paper is a valuable early map of scholarly communication on a rapidly growing platform. Its strengths include a detailed data-collection pipeline, a large openly described dataset, and an independent corroboration of the temporal surge using a separate retrieval strategy. The paper is likely to be widely cited as a reference point for Bluesky's role in post-Twitter science communication and altmetrics. However, the central comparative claims about engagement and originality rest on previously published X statistics that are not methodologically matched to the Bluesky measurements. The descriptive findings are sound, but the comparative conclusions require substantial additional evidence or a rescoped interpretation.

major comments (2)

[Section 4.1 (and Abstract)] The claim that Bluesky posts receive 'substantially higher' engagement than X is based on a comparison with previously published X statistics from Fang et al. (2022), but those statistics were collected under different conditions: a different time period, a different data source (Crossref Event Data versus Altmetric), a different sampling approach, and X's then-current platform mechanics. For example, the paper contrasts 48.2% of Bluesky posts receiving at least ten likes with 3.9-7.5% for X, yet without a matched contemporaneous X sample processed through the same pipeline, the gap could reflect differences in user base, post universe, or data-collection coverage rather than a genuine platform difference. The current comparison is not sufficient to support the paper's central 'substantially higher interaction' claim.
[Section 4.1 (title replication rates)] The originality comparison is similarly unmatched. The paper computes a TF-IDF cosine-similarity threshold of 0.9 on cleaned post text and article titles, finding 6.3% near-verbatim posts, and compares this to 'title replication rates' of 11.8-92.4% from prior studies (Didegah et al., 2018; Kumar et al., 2019; Na, 2015; Sergiadis, 2018; Thelwall, Tsou, et al., 2013). Those prior studies use heterogeneous definitions of replication, including exact title matching and near-duplicate detection with different cleaning rules. Because the metrics and definitions are not aligned, the conclusion that Bluesky posts have 'greater textual originality' than X is not established by the data presented in this paper.

minor comments (4)

[Section 2.1 and Section 3.1] There is an inconsistency in the reported start date of Altmetric's Bluesky tracking: Section 2.1 says Bluesky was added in December 2024 (Kidambi, 2024a), while Section 3.1 says 'Altmetric began collecting Bluesky posts on 24 October 2024' (Altmetric team, 2025). Please reconcile these dates.
[Figure 3 note] The treatment of 40,593 post-article mentions (1.5%) that precede the official publication date as occurring 'within one month after publication' may inflate the reported 50.7% within-first-week and 65.7% within-first-month figures; consider reporting these cases separately or as negative time lags.
[Section 4.2 (Limitations)] The limitations section does not mention the lack of a matched X baseline for the engagement and originality comparisons, even though this is the most important caveat for the paper's headline claims.
[Table 2] The sample posts in Table 2 are paraphrased and anonymized to protect privacy; this is reasonable, but the paper should clarify whether the paraphrasing was performed before or after computing the cosine similarity scores, since the displayed text may not correspond exactly to the analyzed text.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: all Bluesky measurements are original and self-contained; X comparisons come from external prior empirical studies, not from fitted inputs.

full rationale

This is a descriptive empirical study with no fitted parameters or derivation chain that could reduce to its own inputs. The Bluesky engagement, language, and textual-similarity statistics are computed directly from the collected 2.6 million posts and OpenAlex metadata, and no equation defines a Bluesky result in terms of an X baseline. The comparative claims about X rest on external prior publications, including Fang et al. (2022), which is a co-author's earlier peer-reviewed empirical measurement; citing it as a baseline is a data-comparability consideration rather than a self-referential derivation. The originality comparison similarly uses the paper's own cosine-similarity computation against previously reported title-replication rates, but the two metrics are not identical by construction and the limitation is one of external validity, not circularity. The temporal-surge result is corroborated by an independent Bluesky API retrieval described in the Appendix, which strengthens the self-contained character of the main empirical conclusion. No prediction is fitted to a subset of data and then renamed, and no uniqueness theorem or prior-authority citation is used to force a choice. Any concern about unmatched X baselines belongs to correctness or generalizability, not to circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

This empirical paper introduces no fitted constants and no invented entities. The central comparisons rest on a chain of proxy assumptions: DOI-link posts represent scholarly discussion, Bluesky's language tags are accurate, cosine similarity to the title measures originality, and prior X statistics are directly comparable. None of these are fitted parameters, but they are measurement assumptions that a re-analysis would need to challenge.

free parameters (2)

Engagement comparison threshold (k=10) = 10 interactions
The headline comparison with X in Section 4.1 uses the share of posts with at least 10 likes, reposts, replies, or quotes; the threshold was taken from Fang et al. (2022) and is not derived from the Bluesky data.
Title-replication similarity threshold = 0.9 cosine similarity
The claim that only 6.3% of Bluesky posts replicate titles nearly verbatim uses the 0.9 to 1.0 similarity bin, and the comparison with X relies on prior papers' thresholds; changing the cutoff changes the reported percentage.

assumptions (4)

domain assumption Posts containing direct DOI links are a representative proxy for scholarly discussion on Bluesky
The full dataset is built from link-containing posts only; posts that mention papers without links are excluded, as acknowledged in Section 4.2.
domain assumption Bluesky's automatically detected language field accurately identifies the language of each post
The 91% English result in Section 3.3 relies on the platform's 'langs' value, which can be user-set and was not validated by the authors.
domain assumption Low cosine similarity between a post and the article title indicates original or interpretive engagement
Used in Section 3.3 to conclude that posts are not passive reproductions; low similarity could also reflect off-topic text, images, or missing context.
ad hoc to paper Earlier published statistics on X engagement and title replication are directly comparable to the Bluesky measurements
The central comparative claim in Section 4.1 depends on treating Fang et al. (2022) and other X studies as a valid baseline despite different periods, sampling, and platform mechanics.

how reviews work

0 comments

Cite this review

Pith. "Pith review of How is science discussed on Bluesky?." pith.science (2026). https://pith.science/paper/UOFFMDAL

@misc{pith2026250718840,
  author       = {Pith},
  title        = {Pith review of: How is science discussed on Bluesky?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UOFFMDAL}},
  note         = {Machine review of arXiv:2507.18840}
}

read the original abstract

Amid the migration of academics from X, the social media platform Bluesky has emerged as a potential alternative. To assess its viability and relevance for science communication, this study presents the first large-scale analysis of scholarly article dissemination on Bluesky, exploring its potential as a new source of social media metrics. We collected and analysed over 2.6 million Bluesky posts referencing 532,302 scholarly articles from January 2023 to July 2025, integrating metadata from the OpenAlex database. Temporal trends, disciplinary coverage, language use, textual characteristics, and user engagement were examined. A sharp increase in scholarly activity on Bluesky was observed from November 2024 to January 2025, coinciding with broader academic shifts away from X. As on X, Bluesky posts primarily concern the health, social, and environmental sciences and are predominantly written in English. Nevertheless, Bluesky posts demonstrate substantially higher levels of interaction (likes, reposts, replies, and quotes) and greater textual originality than previously reported for X, suggesting both stronger interactive and more interpretive engagement. These findings highlight Bluesky's emerging role as a credible platform for science communication and a promising source for altmetrics. The platform may facilitate not only early visibility of research outputs but also more meaningful scholarly dialogue in the evolving social media landscape.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

7 extracted references · 5 canonical work pages

[1]

P., Portenoy, J., Demes, K., Larivière, V ., & Haustein, S

Alperin, J. P., Portenoy, J., Demes, K., Larivière, V ., & Haustein, S. (2024). An analysis of the suitability of OpenAlex for bibliometric analyses (No. arXiv:2404.17663). arXiv. https://doi.org/10.48550/arXiv.2404.17663 Altmetric team. (2025, January 21). Bluesky tracking enhancement: An increase to our volume of historical. https://updates.altmetric.co...

arXiv doi:10.48550/arxiv.2404.17663 2024
[2]

altmetrics

Costas, R. (2017). Towards the social media studies of science: Social media metrics, present and future. Bibliotecas. Anales de Investigación, 13(1), 1–5. Costas, R., Zahedi, Z., & Wouters, P. (2015). Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. Journal of the ...

work page doi:10.48550/arxiv.2406.15154 2017
[3]

new media

McKee, M., Pagel, C., & Buse, K. (2024). Disinformation enabled donald trump’s second term and is a crisis for democracies everywhere. BMJ, 387, q2485. https://doi.org/10.1136/bmj.q2485 Na, J.-C. (2015). User motivations for tweeting research articles: A content analysis approach. In R. B. Allen, J. Hunter, & M. L. Zeng (Eds.), Digital Libraries: Providin...

work page doi:10.48550/arxiv.2505.24801 2024
[4]

Sergiadis, A. (2018). Analysis of tweets mentioning works from an institutional repository. Journal of New Librarianship, 3(1), 130–150. https://doi.org/10.21173/newlibs/4/27 Sethi, J., Stauss, M., Mirioglu, S., Floyd, L., & Woywodt, A. (2025). Only the bluesky is the limit: Ten tips for a trending #skytorial. Clinical Kidney Journal, 18(2), sfae414. http...

work page doi:10.21173/newlibs/4/27 2018
[7]

The resulting dataset comprised 126,535 posts authored by 26,306 unique users and citing 102,740 distinct articles

Using the Bluesky API, we retrieved all posts containing links with the string “doi.org”, parsed the DOIs embedded in these links , and retained only those whose DOIs could be matched to records in the OpenAlex database . The resulting dataset comprised 126,535 posts authored by 26,306 unique users and citing 102,740 distinct articles. Although searching ...

work page 2024
[1497]

https://doi.org/10.1002/asi.24830 Zhang, Y ., & Fang, Z. (2025). The Tower of Babel in science communication on social media: An analysis of linguistic diversity in twitter mentions of scientific publications. Journal of the Association for Information Science and Technology. https://doi.org/10.1002/asi.70002 32 Appendix To assess whether the sharp increa...

work page doi:10.1002/asi.24830 2025
[5233]

G., Lal, V ., V ahidi, N., Ebadi, A., Carli, M., Sima, A., & Coelho, D

https://doi.org/10.1038/s41598-019-41695-z Vaghjiani, N. G., Lal, V ., V ahidi, N., Ebadi, A., Carli, M., Sima, A., & Coelho, D. H. (2024). Social media and academic impact: Do early tweets correlate with future citations? Ear, Nose & Throat Journal, 103(2), 75–80. https://doi.org/10.1177/01455613211042113 Vidal Valero, M. (2023). Thousands of scientists ...

doi:10.1038/s41598-019-41695-z 2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

P., Portenoy, J., Demes, K., Larivière, V ., & Haustein, S

Alperin, J. P., Portenoy, J., Demes, K., Larivière, V ., & Haustein, S. (2024). An analysis of the suitability of OpenAlex for bibliometric analyses (No. arXiv:2404.17663). arXiv. https://doi.org/10.48550/arXiv.2404.17663 Altmetric team. (2025, January 21). Bluesky tracking enhancement: An increase to our volume of historical. https://updates.altmetric.co...

arXiv doi:10.48550/arxiv.2404.17663 2024

[2] [2]

altmetrics

Costas, R. (2017). Towards the social media studies of science: Social media metrics, present and future. Bibliotecas. Anales de Investigación, 13(1), 1–5. Costas, R., Zahedi, Z., & Wouters, P. (2015). Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. Journal of the ...

work page doi:10.48550/arxiv.2406.15154 2017

[3] [3]

new media

McKee, M., Pagel, C., & Buse, K. (2024). Disinformation enabled donald trump’s second term and is a crisis for democracies everywhere. BMJ, 387, q2485. https://doi.org/10.1136/bmj.q2485 Na, J.-C. (2015). User motivations for tweeting research articles: A content analysis approach. In R. B. Allen, J. Hunter, & M. L. Zeng (Eds.), Digital Libraries: Providin...

work page doi:10.48550/arxiv.2505.24801 2024

[4] [4]

Sergiadis, A. (2018). Analysis of tweets mentioning works from an institutional repository. Journal of New Librarianship, 3(1), 130–150. https://doi.org/10.21173/newlibs/4/27 Sethi, J., Stauss, M., Mirioglu, S., Floyd, L., & Woywodt, A. (2025). Only the bluesky is the limit: Ten tips for a trending #skytorial. Clinical Kidney Journal, 18(2), sfae414. http...

work page doi:10.21173/newlibs/4/27 2018

[5] [7]

The resulting dataset comprised 126,535 posts authored by 26,306 unique users and citing 102,740 distinct articles

Using the Bluesky API, we retrieved all posts containing links with the string “doi.org”, parsed the DOIs embedded in these links , and retained only those whose DOIs could be matched to records in the OpenAlex database . The resulting dataset comprised 126,535 posts authored by 26,306 unique users and citing 102,740 distinct articles. Although searching ...

work page 2024

[6] [1497]

https://doi.org/10.1002/asi.24830 Zhang, Y ., & Fang, Z. (2025). The Tower of Babel in science communication on social media: An analysis of linguistic diversity in twitter mentions of scientific publications. Journal of the Association for Information Science and Technology. https://doi.org/10.1002/asi.70002 32 Appendix To assess whether the sharp increa...

work page doi:10.1002/asi.24830 2025

[7] [5233]

G., Lal, V ., V ahidi, N., Ebadi, A., Carli, M., Sima, A., & Coelho, D

https://doi.org/10.1038/s41598-019-41695-z Vaghjiani, N. G., Lal, V ., V ahidi, N., Ebadi, A., Carli, M., Sima, A., & Coelho, D. H. (2024). Social media and academic impact: Do early tweets correlate with future citations? Ear, Nose & Throat Journal, 103(2), 75–80. https://doi.org/10.1177/01455613211042113 Vidal Valero, M. (2023). Thousands of scientists ...

doi:10.1038/s41598-019-41695-z 2024