{"id":"d6b19c9e-a546-49d6-b793-a06077c8b3e0","arxiv_id":"2507.19373","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Facebook's news-deprioritization algorithm changes reduced user reactions to news posts by roughly 78% between 2021 and 2024, with the largest drops at low-quality outlets, while non-news engagement rose.","lead":"This study tracked billions of Facebook reactions to posts from 40 US news outlets and 21 non-news pages between 2016 and 2025. It finds that Meta's algorithmic news deprioritization reduced reactions to news posts by about 78% from 2021 to 2024, while reactions to non-news pages rose, suggesting platform algorithms, not falling supply or user interest, drove the decline.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Non-news counterfactual and reaction-as-visibility proxy are the load-bearing assumptions; both need direct tests before the causal claim is accepted.","rationale":"The reader's weakest assumption correctly identifies the parallel-trends assumption between news and non-news pages as the key to the causal interpretation of Eq. (1). I agree that the pre-period test is underpowered and that the control group is small and heterogeneous. However, I would also flag a second, equally load-bearing assumption that the reader did not emphasize: the temporal stability of reactions as a proxy for visibility. The paper validates reactions against views only cross-sectionally (Appendix B.2), which does not rule out a changing reaction-per-view ratio during the intervention period. This matters because the abstract and Sec. 3 frame the result as a decline in 'news visibility' while the analysis actually measures reactions. If the mechanism by which the algorithm reduced news visibility also altered who sees news, the proxy could be biased. Both concerns are testable with existing data: the Content Library view counts for a subset of posts can be used to check whether the reaction/views ratio changes by epoch, and a synthetic control or placebo test can assess the counterfactual assumption. Since the paper's descriptive evidence (stable posting, divergent non-news trends, changepoints clustering after 2021, reversal in early 2025) is strong and the reader already assigned CONDITIONAL with MODERATE confidence, my review does not move the verdict; it confirms the need for these additional checks before the causal claim is treated as established.","tokens_in":59985,"tokens_out":5052,"duration_ms":57424,"concrete_test":"Construct a synthetic control for the news-post series using the 21 non-news pages as donor units, weighting them to match the news series on reaction levels and posting counts during epochs 0-4 (pre-intervention), following the synthetic control method of Abadie. Then compute the post-2021 gap between the observed news series and the synthetic control. If the gap is near zero, the original DiD estimate in Eq. (1) is an artifact of the specific, unbalanced control group; if the gap persists, the parallel-trends assumption is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central causal claim, that Facebook's news-deprioritization algorithms reduced news visibility by 78% between 2021 and 2024, rests on two assumptions. First, Eq. (1) treats non-news pages as the counterfactual for news pages, assuming parallel trends in the absence of algorithmic intervention. The pre-period test in Eq. (12) has only 5 epochs and 21 highly heterogeneous control pages (Netflix, Walmart, NBA), so it has little power to detect divergent trends driven by content-type shifts, such as the growth of video and entertainment, or by platform policies that disproportionately favored non-news pages. Second, the outcome is reactions per post, not views. Appendix B.2 validates reactions as a proxy for visibility using a cross-sectional correlation (r = 0.791), but this does not establish that the reaction-to-view ratio is stable over time. If the algorithm changes the composition of the audience that still sees news (for example, leaving only highly engaged loyal followers), reactions per view could rise, so a 78% decline in reactions could overstate or understate the true visibility decline. Both assumptions are necessary for the causal claim to land; the paper's robustness checks do not directly test either.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper analyzes 5.2 million Facebook posts from 40 U.S. news outlets and 396,468 posts from 21 non-news pages between January 2016 and February 2025, using reaction counts as a proxy for news visibility. It reports that reactions to news posts declined by about 78% between 2021 and 2024, while reactions to non-news posts increased by a similar amount, and that low-quality outlets were especially suppressed. The authors use Bayesian changepoint detection (BEAST) to define epochs, negative binomial mixed-effects models to estimate epoch-level reaction means, and a difference-in-difference comparison against non-news pages to support a causal interpretation that Meta's algorithmic 'War on News' drove the decline. They also present robustness checks against alternative explanations including user-base changes, general news-interest trends, and off-platform traffic.","tokens_in":60164,"tokens_out":4302,"duration_ms":45056,"significance":"If the causal claim holds, this is an important contribution: it provides large-scale, longitudinal, on-platform evidence that a major platform's algorithmic choices substantially reduced public exposure to news, with differential effects on low-quality outlets and a reversal when the policy ended. The paper is unusually strong in descriptive scope, with nearly a decade of posting and reaction data, a plausible control series, external quality ratings anchored outside the authors' own work, and a policy reversal that fits the intervention hypothesis. The changepoint detection is data-driven rather than constructed from announcement dates, which mitigates circularity. The central weakness is that two load-bearing assumptions—that non-news pages are a valid counterfactual and that reactions track visibility stably over time—are not directly tested at the temporal resolution required for the causal claim.","major_comments":[{"comment":"The pre-period parallel-trends test (chi-square(4)=3.88, p=0.423) is based on only five epochs and a control group of 21 highly heterogeneous pages (Netflix, NBA, Walmart, Chipotle, etc.). This test has low power, and it cannot rule out divergent trends driven by content-type shifts such as the growth of video and entertainment or by platform changes that disproportionately favored non-news content. Since the non-news series increased by roughly 78% over the same period in which news declined, the difference-in-difference estimate in Eq. (1) may partly reflect a relative shift toward entertainment content rather than targeted suppression of news. Please add a direct test using a control set restricted to non-entertainment pages, or a model with post-type-by-time interactions, and report the parallel-trends test at higher temporal resolution.","section":"Sec. 4.4, Eq. (12)"},{"comment":"The validation of reactions as a visibility proxy is cross-sectional: the correlation between views and reactions (r=0.791) is computed across posts at a single point in time. The paper's central claim is about changes in visibility over time, and the reaction-to-view ratio may itself have changed as the algorithm altered which users see news posts (e.g., leaving only highly engaged loyal followers). If the ratio is not stable over time, a 78% decline in reactions could overstate or understate the true visibility decline. Please estimate the reactions-per-view ratio by epoch for the Content Library subset and show that it is stable, or qualify the visibility claims accordingly.","section":"Appendix B.2"},{"comment":"The assignment of detected changepoints to causes is partly ambiguous, which is acknowledged in Sec. 4.4. For example, changepoint 5 at 2021-03-01 is matched to both the Biden inauguration and the February 10, 2021 algorithmic announcement, and changepoint 8 has no identified cause. Because the causal claim depends on the alignment of epochs with algorithmic interventions, please provide a formal sensitivity analysis—such as placebo changepoints or a comparison of changepoint density around algorithmic versus political events—to demonstrate that the matching is not post-hoc selection.","section":"Sec. 2.2, Fig. 2"}],"minor_comments":[{"comment":"In the Discussion, 'we identifying the feed algorithm changes' should read 'we identify the feed algorithm changes'.","section":"Sec. 3"},{"comment":"In the Discussion, 'making them more susceptibility to misinformation' should read 'making them more susceptible to misinformation'.","section":"Sec. 3"},{"comment":"The word 'algoirthmically' is a typo and should be 'algorithmically'.","section":"Appendix F.1"},{"comment":"The header 'Ad Fonts Bias' should be 'Ad Fontes Bias' for consistency with the cited rating system.","section":"Table A.1"},{"comment":"The main text says reaction counts were missing for approximately 10% of posts in the Content Library data, while Appendix E says the preliminary model was used for imputing about 1.1% of missing reaction counts; please clarify that these percentages refer to different denominators.","section":"Sec. 4.1 and Appendix E"},{"comment":"Changepoint 8 (2022-10-31) is listed without an event description; consider adding a brief note that no proximal algorithmic event was identified.","section":"Fig. 2b"}],"recommendation":"major_revision","confidential_remarks":"The descriptive results are strong and the paper is likely publishable after revision. The main burden is on the two causal assumptions: the non-news counterfactual and the temporal stability of reactions as a proxy for visibility. If the authors can provide the suggested direct tests or appropriately weaken the causal wording, I would support acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth taking seriously. It's the most comprehensive longitudinal look at Facebook's news deprioritization to date, and the descriptive pattern is strong: reactions to news dropped by roughly 78% between 2021 and 2024, while reactions to non-news pages rose, posting rates stayed stable, and the 2025 policy reversal brought a partial rebound. The data scale is genuinely new—5.2M posts from 40 outlets, 7.9B reactions, spanning 2016–2025 with quality tiers and a non-news comparison. The changepoint detection is data-driven, not fit to announcement dates, and the authors are transparent about the limitations of their causal design.\n\nThe modeling is careful: negative binomial mixed models, validation plots, robustness checks against user base, off-platform traffic, and news interest. The citation pattern is solid, and the external quality ratings anchor the analysis outside the authors' own work.\n\nTwo soft spots deserve attention. First, the non-news counterfactual rests on only 21 heterogeneous control pages (Netflix, Walmart, NBA), and the pre-period parallel-trends test has limited power: five epochs, chi-square(4)=3.88, p=0.423. That test cannot rule out content-type shifts or platform policies that favored entertainment over news for reasons unrelated to the \"War on News.\" Second, reactions are a proxy for visibility, validated only cross-sectionally (r=0.79 with views). If the algorithm changed who still sees news—leaving mostly loyal, high-engagement followers—reactions per view could rise, making the 78% reaction decline an over- or under-statement of the true reach decline. The authors acknowledge this, but they don't directly test the stability of the reaction-to-view ratio over time. Also, the headline number is from data-selected peak and trough epochs; the pattern is real, but the precise magnitude should be treated with caution.\n\nNone of this sinks the paper. The overall trajectory, the non-news divergence, and the 2025 reversal make a strong case that something news-specific happened on Facebook. The causal claim is plausible but quasi-experimental, and the authors say so. This is a contribution for computational social scientists and media policy researchers. It deserves serious peer review—send it out. I'd ask the authors for alternative control pages, sensitivity analyses around epoch selection, and any available time-varying view data to validate the proxy. But this is a desk-accept-to-review, not a desk reject.","headline":"A large, carefully analyzed observational dataset showing a striking decline in reactions to news on Facebook after 2021, with a plausible but not airtight causal claim.","tokens_in":60727,"tokens_out":2159,"would_cite":true,"duration_ms":23515,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Facebook's algorithm changes cut reactions to news posts by 78 percent between 2021 and 2024.","keywords":["Facebook","algorithmic news feed","news visibility","user reactions","difference-in-difference","changepoint detection","news quality","platform governance"],"falsifier":"Look for an internal Facebook metric of news feed impressions or reach: if impressions of news posts stayed flat or rose while reactions fell by 78 percent, the 'decreased visibility' claim would fail because the decline would be in engagement propensity rather than exposure. Conversely, if reach fell with reactions, the claim is supported. A second check would be to match each news outlet to non-news pages by post type and posting volume; if the divergence disappears or shrinks sharply, the counterfactual comparison was not valid.","tokens_in":59741,"feed_emoji":"📉","tokens_out":6018,"duration_ms":56484,"temperature":0.7,"pith_summary":"This paper tries to establish that Facebook's own algorithmic choices, not outside forces, caused a sharp drop in how often Americans engaged with news on the platform between 2021 and 2024. Drawing on 5.2 million posts from 40 U.S. news outlets and nearly 8 billion reactions, it reports that average reactions per news post fell by roughly 78 percent while reactions to posts from sports, retail, and entertainment pages rose by a comparable amount. It argues the decline cannot be explained by news outlets posting less, by users leaving Facebook, or by falling interest in news, and that the January 2025 reversal of the policy produced an immediate rebound. If right, the finding shows that a platform's feed design can quietly redraw the boundaries of public attention to news.","feed_headline":"Facebook's algorithm changes cut reactions to news by 78%","feed_subtitle":"Non-news pages rose 78% over the same period, pointing to deliberate news deprioritization.","key_machinery":"The load-bearing object is the difference-in-difference contrast $\\tau(t) = \\psi(t|\\text{news})/\\psi(t|\\text{non-news})$, a ratio of ratios that uses engagement with 21 non-news pages as the counterfactual for what news engagement would have been without algorithmic intervention. Around it sits a pipeline: user reactions validated as a proxy for visibility (correlation with views about 0.79), a Bayesian changepoint detector that splits the decade into 12 epochs, and negative binomial mixed-effects models that estimate per-post reaction counts while absorbing outlet-level and daily variation.","core_discovery":"On the paper's own terms, the discovery is that Facebook's 'War on News' worked as announced: after the February 2021 announcement that civic and political content would be deprioritized, user reactions to news posts entered a steady decline, reaching a minimum of about 526 reactions per post in mid-2023 to early 2024 versus peaks above 3,000 during election periods, a cumulative drop of about 78 percent. The decline was largest for low-quality outlets (about 86 percent), which had historically drawn the most reactions, while all quality tiers converged to similarly depressed levels. Crucially, 21 large non-news pages showed no such decline; their reactions rose by about 78 percent in the same interval, and a difference-in-difference contrast finds the two statistically significant epoch-to-epoch drops (2022 and 2023) were specific to news. The paper also reports a partial rebound after January 2025, concentrated among low- and medium-quality outlets, and rules out shrinking platform use, falling news supply, and reduced overall news interest as explanations.","pith_inferences":["Beyond the paper: if the parallel-trends comparison is trusted, the 78 percent rise in non-news reactions implies that attention freed from news was partly reallocated to entertainment and retail content, not just lost; that reallocation is the paper's implicit mechanism but is not directly measured.","Beyond the paper: a natural extension would test the same design on other content types the policy targeted, such as posts by politicians and parties, which the paper notes it did not include; finding a similar divergence there would widen the claim beyond news organizations.","Beyond the paper: the analysis cannot distinguish which of the roughly 399 announced changes did the work; a stricter test would exploit staggered rollout dates or platform-internal impression logs to isolate each intervention.","Beyond the paper: because reactions track views only partially (r ≈ 0.79), internal impression data from the platform could either confirm or revise the magnitude; the paper's proxy assumption is testable in principle."],"forward_implications":["If the central claim is right, Facebook's 2021-2024 feed changes reduced on-platform reactions to news by about 78 percent while leaving non-news engagement intact, implying the effect was content-specific rather than a general engagement slump.","Low-quality outlets, the original target of the policies, were hit hardest (about 86 percent), meaning the algorithm acted as a quality-filtering mechanism during that period.","The January 2025 reversal caused an immediate increase in reactions to news, with the rebound concentrated in low- and medium-quality outlets, suggesting the earlier decline was reversible rather than a structural loss of audience.","The absence of a parallel decline in off-platform traffic and in news interest surveys implies that on-platform engagement metrics, not actual news consumption, were the thing the algorithm moved.","Election spikes in 2024 were markedly weaker than in 2016 and 2020, so even peak-interest events could not overcome the algorithmic suppression."],"supporting_citations":[{"why":"Supplies the aggregated news-quality ratings used to split outlets into low, medium, and high quality tiers.","marker":"(24)"},{"why":"Provides the difference-in-difference estimator and parallel-trends framework used to estimate causal effects.","marker":"(25)"},{"why":"Documents the February 2021 announcement and rollout of news and civic content deprioritization that anchors the treatment period.","marker":"(9)"},{"why":"Records the January 2025 announcement ending news deprioritization, corresponding to the recovery epoch.","marker":"(7)"},{"why":"The changepoint-detection method used to segment the reaction time series into 12 epochs.","marker":"(46)"},{"why":"Documents the Canadian news ban used to identify one of the significant news-specific drops.","marker":"(34)"},{"why":"Prior analysis of the 2020 algorithm change that this study extends to on-platform data for nearly a decade.","marker":"(28)"},{"why":"Describes the 2018 feed change prioritizing friends and family over pages, which serves as a baseline comparison.","marker":"(8)"},{"why":"The mixed-model estimation software used to fit the negative binomial regressions with outlet and time random effects.","marker":"(62)"}],"fun_headline_variants":["Facebook algorithm shift cut news reactions by 78%","News reactions on Facebook dropped 78% after 2021 change","Meta's 'war on news' reduced reactions by 78%, study finds","Facebook news visibility fell 78%, non-news rose in same period"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole causal story rests on the assumption that, absent algorithmic changes, news and non-news pages would have kept parallel reaction trends; the pre-2021 test supports this but uses only 5 epochs and 21 non-news pages, so it cannot rule out that the two groups were headed for different trajectories anyway.","fun_headline_variants_meta":{"raw":{"variants":["Facebook algorithm shift cut news reactions by 78%","News reactions on Facebook dropped 78% after 2021 change","Meta's 'war on news' reduced reactions by 78%, study finds","Facebook news visibility fell 78%, non-news rose in same period"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000176,"raw_usage":{"total_tokens":1283,"prompt_tokens":933,"completion_tokens":350,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":549,"completion_tokens_details":{"reasoning_tokens":275}},"tokens_in":549,"tokens_out":350,"duration_ms":3985,"temperature":1.0,"reasoning_tokens":275,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:53:25.472861+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Look for an internal Facebook metric of news feed impressions or reach: if impressions of news posts stayed flat or rose while reactions fell by 78 percent, the 'decreased visibility' claim would fail because the decline would be in engagement propensity rather than exposure. Conversely, if reach fell with reactions, the claim is supported. A second check would be to match each news outlet to non-news pages by post type and posting volume; if the divergence disappears or shrinks sharply, the counterfactual comparison was not valid.","supporting_citations":[],"review_version":2}