Pith. sign in

REVIEW 2 major objections 4 minor 13 references

How do Developers Promote Open Source Projects?

T0 review · 2 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that popular GitHub projects promote themselves mainly through Twitter, user meetings, and blogs, and that successful Hacker News posts coincide with a doubling of the median GitHub stars gained in the following three…

desk verdict Useful descriptive census of promotion channels for top GitHub projects, but Section 3.4's Hacker News effect is confounded by release-driven star gains and post-hoc success selection. read the letter →

arxiv 1908.04219 v1 pith:Z5HWE7IW submitted 2019-08-12 cs.SE

Hudson Borges , Marco Tulio Valente This is my paper · ORCID
classification cs.SE
keywords opensourcepromotionGitHubHackerNewsTwitterusermeetingsbloggingsoftwarepopularitysocialmedia
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper investigates how developers promote open source projects by manually inspecting the top 100 most-starred GitHub repositories. It claims that Twitter, user meetings, and blogs are the most common promotion channels, used by 56, 41, and 38 projects respectively, and that popular projects use these channels far more than a random sample. It also claims that successful Hacker News posts—the top 10 percent by upvotes—are associated with a median increase in stars gained over three days from 74 to 138. A sympathetic reading is that promotion is a real and measurable part of open source project management, not an afterthought. The paper concludes with practical recommendations for project leaders to invest in Twitter, user meetings, blogs, and news aggregator sites.

What carries the argument

The machinery is a manually curated taxonomy of promotion channels—blogs, events, user meetings, Twitter, Facebook, Google+, newsletters, and RSS feeds—built by inspecting READMEs and project websites, plus a before-and-after comparison around Hacker News posts. User meetings are operationalized through Meetup topic groups, and Hacker News data come from its public API. The load-bearing comparison is the star-gain measurement in the three days before versus after each successful post, where 'successful' means a post in the top 10 percent by upvotes. That comparison is what converts the descriptive channel survey into a claim about promotion impact.

What would settle it

Repeat the Hacker News measurement on a control set of GitHub projects with similar pre-post star trajectories but without any successful Hacker News post; if the control projects show the same median growth from 74 to 138 stars in the same calendar windows, the claimed promotional effect would be contradicted. Alternatively, remove from the analysis any post immediately followed by a project release or a concurrent major announcement, and check whether the 74-to-138 gap survives.

Watch

Extended reading notes

Core claim

The central discovery is a descriptive map of promotion practice among the most popular GitHub projects, together with quantitative evidence that successful promotion on Hacker News coincides with increased popularity. On the paper's own terms, the most common channels are Twitter (56 projects), user meetings (41), and blogs (38), with events and RSS feeds tied at 33; Facebook and Google+ are rare. Popular projects use more channels than a random sample, while the ranking of channels is strongly correlated across the two groups. For Hacker News, the paper finds 3,019 posts referencing 96 of the 100 projects, but attention is highly skewed: the top 10 percent of posts receive at least 132 upvotes. Comparing the three days before and after such successful posts, the median number of stars gained rises from 74 to 138, a difference the paper reports as statistically significant under a one-tailed Mann-Whitney U test with medium effect size.

Load-bearing premise

The before-and-after star comparison assumes that no other event—such as a new release or other media coverage—coincides with the successful Hacker News posts, so the observed star increase can be attributed to the posts themselves.

Editorial extensions

If this is right

  • If the paper is right, promotion is a distinguishing practice of successful open source projects, with two thirds of top projects using at least one channel and more than half using at least two.
  • Twitter is the channel most worth investing in: it is the most common, and 83.9 percent of projects with a Twitter account keep it active.
  • User meetings are a substantial promotion mechanism, led by torvalds/linux with 2,261 Meetup groups across 96 countries.
  • Successful Hacker News posts coincide with a doubling of median three-day star gains, so a single high-attention post can matter more than routine social media activity.
  • Hacker News outcomes are highly unequal: only 10 percent of posts attract substantial attention, so project leaders should treat front-page success as rare rather than typical.
  • The ranking of channels in popular projects strongly correlates with the ranking in a random sample, suggesting that popularity changes the intensity of promotion more than the choice of channels.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves causality open: its before-and-after design cannot rule out that a project's own momentum or a coinciding release caused both the Hacker News success and the star increase. A natural extension would compare projects with successful Hacker News posts against matched projects without such posts in the same calendar window.
  • The finding implies that open source managers should not expect average Hacker News posts to drive growth, since the distribution is extremely skewed; the practical question becomes how to craft the rare post that reaches the front page, which the paper does not address.
  • The channel taxonomy could be extended to measure not just presence but sustained activity, for example linking blog post frequency or Twitter cadence to star growth over months, which would tell managers whether consistency matters more than occasional spikes.
  • The paper's method could be applied to other aggregator communities such as Reddit to test whether the Hacker News effect is specific to that site or generalizes to any high-traffic technology forum.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. This paper reports a manual study of promotion channels used by 100 top-starred GitHub repositories, compares them with 100 repositories randomly selected from the top-5,000, and measures the association between successful Hacker News posts and GitHub star growth. The authors find that Twitter, user meetings, and blogs are the most common channels, that popular projects use these channels more often than the random sample, and that median stars gained in the three days after successful Hacker News posts (138) exceed those gained in the three days before (74). They conclude with practical recommendations for open source project maintainers.

Significance. If taken as a descriptive measurement study, the paper is a useful addition to the empirical software engineering literature: the channel taxonomy is concrete, the top-100 versus random-sample comparison is a sensible design, and the Hacker News effect-size estimate is transparently reported. However, the causal claim that Hacker News has a positive impact on project popularity rests on a before/after comparison without a control group, and the paper's own post-category data show that successful posts predominantly announce releases. The manuscript therefore needs a substantially strengthened causal analysis, or a reinterpretation as associational evidence, before the central RQ3 conclusion is supported.

major comments (2)
  1. [Section 3.4, Fig. 8] The before/after comparison in Figure 8 does not identify a causal effect of Hacker News because 'successful' posts are defined post hoc as the top 10% of posts by upvotes (at least 132 upvotes), and those posts overwhelmingly coincide with major project events: 44.9% announce new releases, 16.5% first releases, 10.6% new features, and 1.6% open-sourcing. Such events plausibly drive GitHub stars independently of whether they are discussed on Hacker News, so the median increase from 74 to 138 stars may reflect release-day momentum rather than promotion via Hacker News. The analysis should include a control group of comparable releases or announcements not posted on Hacker News, a difference-in-differences specification, or project-level fixed effects that absorb time-invariant project popularity; without such controls, the abstract's and Section 5's 'Hacker News promotes' statements should be downgraded to associational language.
  2. [Section 3.4, Mann-Whitney U test] The statistical test treats each successful post as an independent observation, but projects contribute multiple posts and the three-day windows around posts from the same project can overlap, violating the independence assumption and likely inflating statistical significance. The authors should account for clustering by project (e.g., a mixed-effects model, a per-project paired test, or a block bootstrap) and report the number of distinct projects contributing to Figure 8.
minor comments (4)
  1. [Section 2] Because the promotion-channel coding is manual and performed by a single author, the paper should report an inter-rater reliability check or at least state the coding rules for ambiguous cases, such as when a README links to a Twitter account that is neither official nor verified.
  2. [Section 3.4] The phrase 'first three days before and after the publication date' should be made precise; if star counts are timestamped per hour, specify whether the windows are exact 72-hour intervals or inclusive calendar days, since this affects the count and the comparison.
  3. [Section 3.4, Fig. 8] The boxplot appears truncated at 600 stars while the text reports a median of 138; clarify the y-axis treatment of outliers so that the visual scale does not mislead readers.
  4. [Throughout] Typos should be corrected: 'appple/swift' (Section 3.4), 'torvalds/linuxhas' (Section 3.2), and 'Maleej' (Section 4, should be 'Maalej').

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the study is an empirical measurement, and none of its claims reduces to its inputs or to load-bearing self-citations.

full rationale

The paper does not offer a derivation or a fitted model; it reports hand-coded observations of promotion channels and statistical comparisons of independent measures. The Hacker News analysis defines success by upvotes (top-10%, at least 132) and measures GitHub stars separately, so the finding that stars increase after successful posts is not true by construction—the before/after data could have shown no increase. The random-sample comparison uses a rank correlation but no parameters fitted to the target outcome. The only self-citation, [3], supports the background statement that stars indicate popularity and is not load-bearing for any central claim. The main methodological weakness is potential confounding in the Hacker News before/after comparison, such as concurrent releases or other media coverage, but confounding is a validity threat, not circularity: the conclusion is not equivalent to its inputs. Thus no circular step is present.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The study rests on common empirical-software-engineering assumptions about sample selection, manual measurement, and post hoc thresholds. No free parameters or invented entities are introduced.

assumptions (3)
  • domain assumption The top-100 most-starred GitHub projects are a valid sample of 'popular projects' for studying promotion behavior.
    Section 2 selects projects by star count; the paper does not justify that star popularity is the relevant dimension for promotion behavior.
  • domain assumption Manual inspection of READMEs and websites by the first author reliably captures each project's promotion channels.
    Section 2 reports the manual procedure; no inter-rater reliability check is described.
  • ad hoc to paper The threshold for 'successful' Hacker News posts (top 10%, at least 132 upvotes) is a meaningful split for measuring promotion impact.
    Section 3.4 defines successful posts from the data's own distribution rather than from an independent criterion.

how reviews work

0 comments
Cite this review

Pith. "Pith review of How do Developers Promote Open Source Projects?." pith.science (2026). https://pith.science/paper/Z5HWE7IW

@misc{pith2026190804219,
  author       = {Pith},
  title        = {Pith review of: How do Developers Promote Open Source Projects?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z5HWE7IW}},
  note         = {Machine review of arXiv:1908.04219}
}
read the original abstract

Open source projects have an increasing importance on modern software development. For this reason, these projects, as usual with commercial software projects, should make use of promotion channels to communicate and establish contact with users and contributors. In this article, we study the channels used to promote a set of 100 popular GitHub projects. First, we reveal that Twitter, user meetings, and blogs are the most common promotion channels used by the studied projects. Second, we report a major difference between the studied projects and a random sample of projects, regarding the use of the investigated promotion channels. Third, we show the importance of a popular news aggregation site (Hacker News) on the promotion of open source. We conclude by presenting a set of practical recommendation to open source project managers and leaders, regarding the promotion of their projects.

Figures

Figures reproduced from arXiv: 1908.04219 by the authors.

Figure 1
Figure 1. Number of GitHub stars of the analyzed projects [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. presents the most common promotion channels used by the top-100 projects on GitHub. The most common channel is Twitter, which is used by 56 projects. The second one is Users Meetings (41 projects), followed by Blogs (38 projects), Events (33 projects), and RSS feeds (33 projects). The least common channels are Facebook and Google+, which are used by 18 and 7 projects, respectively. Projects 0 10 20 30 40 50 60 Twitt… view at source ↗
Figure 3
Figure 3. Number of promotion channels per project [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Distribution of the number of posts on the last 12 months (outliers are omitted) [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Number of groups, cities, and countries of the user meetings [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Most common promotion channels used by random projects [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Number of posts, upvotes, and comments (outliers are omitted) [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Number of GitHub stars received by projects covered by successful Hacker News [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 11 canonical work pages

  1. [1]

    Comino, F

    S. Comino, F. M. Manenti, and M. L. Parisi, ``From planning to mature: On the success of open source projects,'' Research Policy, vol. 36, no. 10, pp. 1575--1586, 2007

    work page 2007

  2. [2]

    I. GitHub. (2018, Feb.) Code search. [Online]. Available: https://github.com/search

    work page 2018

  3. [3]

    Borges, A

    H. Borges, A. Hora, and M. T. Valente, ``Understanding the factors that impact the popularity of GitHub repositories,'' in 32nd IEEE International Conference on Software Maintenance and Evolution (ICSME), 2016, pp. 334--344

    work page 2016

  4. [4]

    (2018, Feb.) What is a meetup group? [Online]

    Meetup. (2018, Feb.) What is a meetup group? [Online]. Available: https://www.meetup.com/help/article/902256

    work page 2018

  5. [5]

    V. D. Bianco, L. Lavazza, V. Lenarduzzi, S. Morasca, D. Taibi, and D. Tosi, ``A study on OSS marketing and communication strategies,'' in 8th International Conference on Open Source Systems (OSS), 2012, pp. 338--343

    work page 2012

  6. [6]

    Singer, F

    L. Singer, F. F. Filho, and M.-A. Storey, ``Software engineering at the speed of light: how developers stay current using Twitter ,'' in 36th International Conference on Software Engineering (ICSE), 2014, pp. 211--221

    work page 2014

  7. [7]

    Pagano and W

    D. Pagano and W. Maalej, ``How do developers blog?: An exploratory study,'' in 8th Working Conference on Mining Software Repositories (MSR), 2011, pp. 123--132

    work page 2011

  8. [8]

    Bajic and K

    D. Bajic and K. Lyons, ``Leveraging social media to gather user feedback for software development,'' in 2nd International Workshop on Web 2.0 for Software Engineering, 2011, pp. 1--6

    work page 2011

Show all 13 references
  1. [9]

    Hansson, Y

    C. Hansson, Y. Dittrich, and D. Randall, ``How to include users in the development of off-the-shelf software: A case for complementing participatory design with agile development,'' in 39th Annual Hawaii International Conference on System Sciences (HICSS), 2006, pp. 175c--175c

    2006

  2. [10]

    Aniche, C

    M. Aniche, C. Treude, I. Steinmacher, I. Wiese, G. Pinto, M.-A. Storey, and M. Gerosa, ``How modern news aggregators help development communities shape and share knowledge,'' in 40th International Conference on Software Engineering (ICSE), 2018, pp. 1--12

    2018

  3. [11]

    write newline

    " write newline "" initialize.prev.this.status FUNCTION begin.bib " write newline preamble empty 'skip preamble write newline if " thebibliography " longest.label * " " * write newline " [1] #1 " write newline " url@samestyle " write newline " " write newline " [2] #2 " write ...

  4. [12]

    4" FUNCTION default.is.dash.repeated.names #1 FUNCTION default.name.format.string

    11em plus .33em minus .07em 4000 4000 100 4000 4000 500 `\.=1000 = #1 \@IEEEnotcompsoconly \@IEEEcompsoconly #1 * [1] 0pt [0pt][0pt] #1 * [1] 0pt [0pt][0pt] #1 * \| ** #1 \@IEEEauthorblockNstyle \@IEEEcompsocnotconfonly \@IEEEcompsocconfonly \@IEEEauthorblockAstyle \@IEEEcomps...

    2007

  5. [13]

    write newline

    " write newline "" initialize.prev.this.status FUNCTION begin.bib " write newline preamble empty 'skip preamble write newline if " thebibliography " longest.label * " " * write newline " [1] #1 " write newline " url@samestyle " write newline " " write newline " [2] #2 " write ...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.