{"id":"2154ca0e-0c10-4b79-ab89-7cab725ca3df","arxiv_id":"2501.14040","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"In 42,853 news articles from 27 countries, societal and legal risks dominate AI coverage, while environmental risks are almost absent, and outlet political bias shifts the emphasis.","lead":"This study maps which AI harms get the most attention in global English-language news, finding that societal and legal risks dominate while environmental risks are nearly invisible. It also shows that coverage varies with a news outlet's political leaning, a pattern relevant to AI governance and regulation.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 26.2% scrape-success sample is never shown to represent the eligible URL corpus, so the global prevalence ranking rests on an untested selection assumption.","rationale":"The reader's weakest assumption identifies exactly the load-bearing issue: the 26.2% scrape success rate with no representativeness check. I agree this is the most important threat to the paper's central descriptive claim. The paper is otherwise transparent: it reports scrape rates, validates the LLM filtering step (F1=0.82), reports inter-coder reliability (K-alpha=0.97), and provides category-level LLM classification metrics. The LLM annotation error (macro F1=0.60) is a real limitation, but a rough bias correction using Table 3 precision/recall values suggests the large gaps between Societal, Legal, and the remaining categories are unlikely to flip; the scrape-selection problem is more fundamental because it threatens external validity before any measurement adjustment. The proposed test is feasible without new data collection because GDELT retains metadata for all retrieved URLs, and it directly addresses whether the unweighted article counts can support the global-prioritization claim. Since the reader already conditioned the verdict on this issue and recommended robustness checks, no verdict change is needed; the appropriate disposition remains conditional on such checks being performed.","tokens_in":28449,"tokens_out":5723,"duration_ms":56611,"concrete_test":"Pull GDELT metadata (source domain, country, tone, and theme counts) for all 163,314 post-MBFC URLs and fit a logistic regression predicting scraped=1 from domain, country, tone, and theme indicators. Then compute inverse-propensity-weighted prevalence estimates for the six risk categories, using the 16,312-article LLM labels but weighting each article by the inverse of its predicted scrape probability. If the weighted ordering of Societal, Legal, Cognitive, and Content Safety risks changes, or if any category shifts by more than 5 percentage points relative to Figure 1B, the headline ranking is not robust to scraping selection.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3 reports that of the 163,314 eligible URLs remaining after domain and MBFC filtering, only 42,853 (26.2%) could be scraped, yet the paper provides no comparison between scraped and unscraped articles. Because scrapability depends on paywalls, bot-blocking, and site-specific page structure, and because the United States contributes 42.9% of the final sample, the unadjusted prevalence estimates in Figure 1B could shift if unscraped coverage differs in risk emphasis or regional composition. The subsequent LLM filtering (32,439 articles with impacts, then 16,312 with negative impacts) compounds this selection, and the denominator for all prevalence claims is the 16,312 subset rather than the original eligible corpus. The paper acknowledges English-only and paywall limitations in Section 6, but it never quantifies whether the scraped subset is representative of the eligible URL set on any observable dimension. The central ranking (Societal Risks 50.6%, Legal & Rights 32.9%, Cognitive 14.2%, Content Safety 8.3%, Existential 7.2%, Environmental 1.9%) is therefore an estimate over a convenience sample, and the 'global' interpretation depends on an unverified assumption that scraped coverage is unbiased with respect to risk category prevalence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper analyzes English-language news coverage of AI from January 2022 to October 2024 by collecting URLs through GDELT using 41 AI-related keywords, filtering to Media Cloud national news domains and Media Bias Fact Check-rated domains, and scraping 42,853 articles (26.2% of the eligible URL set). The authors use GPT-4o to identify articles reporting AI impacts and to summarize negative impacts, then manually annotate a sample of 1,060 impact summaries into the AIR taxonomy plus emergent categories with high inter-coder reliability (Krippendorff's alpha = 0.97), and scale the annotation with GPT-4o. They retain six risk categories with LLM recall at or above 0.70 and report global prevalence estimates (Societal Risks 50.6%, Legal & Rights-related Risks 32.9%, Cognitive Risks 14.2%, Content Safety Risks 8.3%, Existential Risks 7.2%, Environmental Risks 1.9%), regional comparisons, and variation by outlet political bias. The paper also claims to contribute 16 risk categories absent from the AIR taxonomy and proposes eight possible Level-1 extensions.","tokens_in":28761,"tokens_out":6182,"duration_ms":57481,"significance":"The topic is timely and important: layering news-media coverage of AI harms onto expert-driven risk taxonomies is a genuinely useful direction for AI governance and impact assessment. The paper is transparent about its pipeline, publishes its prompts in the appendix, uses a validation set for both filtering and annotation, and reports inter-coder reliability. The broad geographic ambition, even with the acknowledged gaps, goes beyond the usual US/UK focus of prior media-coverage studies. If the sampling and annotation concerns below are addressed—especially the representativeness of the scraped subset and the measurement error of the LLM labels—the resulting risk-prevalence ranking and the proposed taxonomy extensions could be a valuable contribution to the AI-risk literature.","major_comments":[{"comment":"The paper does not establish that the 42,853 successfully scraped articles (26.2% of the 163,314 eligible URLs) represent the eligible URL corpus. Scrapability depends on paywalls, bot-blocking, and site structure, and the United States alone contributes 42.9% of the final sample, so the global prevalence ranking in Figure 1B could shift if unscraped coverage differs in risk emphasis or regional composition. The limitation paragraph in Section 6 mentions paywalls and English-only coverage, but it does not quantify or test representativeness on any observable dimension (e.g., outlet, country, date, or topic distribution). At minimum, the paper should compare scraped and unscraped URLs on available metadata and either adjust the estimates or explicitly reframe the central claims as estimates for the scraped, English-language, accessible subset.","section":"Section 3, Table 1"},{"comment":"The headline prevalence estimates inherit substantial measurement error from the LLM annotation, which has a macro F1 of 0.60; the six included categories are selected by recall >= 0.70, yet several have precision in the low-to-mid sixties. Reporting percentages to one decimal place (e.g., 50.6%, 32.9%) implies a precision that the classifier does not support, and dropping categories with lower recall, such as System & Operational Risks (14.2% in Table 4), changes the apparent global ranking. The paper should report per-category confidence intervals or a confusion-matrix-adjusted estimate, and should make clear that the six-category ranking is conditional on a recall-based selection rule rather than an exhaustive ranking of all risks in the corpus.","section":"Section 4.2, Table 3 and Section 5.2.1"},{"comment":"The definition of 'existential_threats' in the LLM annotation prompt is a copy of the 'ethical_impact' definition (inequalities in accessing AI, fairness, accountability, transparency) and does not match the existential-risk definition used in the human annotation and in Section 5.1 (extinction, loss of control, misalignment). Because this prompt was used to scale the annotations, it is unclear what the LLM's Existential Risks labels actually measured, and the reported 7.2% prevalence for this category is therefore suspect. The authors should correct the prompt and re-run the annotation, or at minimum show that the incorrect definition did not materially affect the existential-risk estimates.","section":"Appendix A.6, category 11"},{"comment":"The denominator used for prevalence is 16,312 articles judged by GPT-4o to contain negative impacts, not the 42,853 scraped articles or the 163,314 eligible URLs, and articles can be counted in multiple risk categories because a single article can report more than one negative impact. The abstract and results text often say 'global news coverage ... prioritizes' certain risks, which overstates the inferential target; more precise wording would be 'among sampled articles classified as reporting negative AI impacts, the proportion mentioning category X was ...'. The paper should also report how many articles had multiple risk categories and whether the category percentages are based on unique articles or on article-category observations, since the ambiguity affects the interpretation of all reported percentages.","section":"Section 5.2.1, Figure 1B caption"},{"comment":"Several regional political-bias claims are based on very small cell sizes and are presented with undue confidence. Examples include the Asia right-biased cell with 25 articles, the Middle East left-biased cell with 1 article, and the Oceania right-biased existential-risk share of 23.1%, which likely corresponds to a handful of articles. The paper should report the article count n for each region-by-bias cell and avoid narrative generalizations from cells of this size; confidence intervals or a note that these are descriptive counts would make the regional politicization claims appropriately cautious.","section":"Section 5.2.2 and Figure 3"}],"minor_comments":[{"comment":"The CCS Concepts field still contains the placeholder text 'Do Not Use This Code' and 'Generate the Correct Terms for Your Paper' and should be replaced with actual CCS terms before submission.","section":"Title page, CCS Concepts"},{"comment":"The prompt says '32 categories' but also refers to 'the above 33 categories,' and the list actually contains 32 numbered entries except that the final 'other' entry makes the count ambiguous; the text and the list should be made consistent.","section":"Appendix A.6"},{"comment":"The definitions contain duplicates and internal inconsistencies: 'Hate/Toxicity Risks' appears twice in the Content Safety Risks block, and 'Criminal Activities Risks' and 'Defamation Risks' appear both inside the included Level-1 categories and in the excluded list. These entries should be deduplicated and reconciled.","section":"Appendix A.7"},{"comment":"The name 'Kripendorph's alpha' is a typo for 'Krippendorff's alpha.'","section":"Section 4.2"},{"comment":"The word 'hightest' appears twice and should be corrected to 'highest.'","section":"Section 5.2.2, Oceania paragraph"},{"comment":"The counts for the 12 listed categories sum to 47,347, not the stated total of 47,731 negative impacts; the paper should clarify which categories are omitted (e.g., 'other' or 'no_impact') and make the accounting complete.","section":"Table 4"},{"comment":"Reference [5] is listed as 'Anonymous 2024. Under review,' which may make identifying the work unnecessarily difficult; if it is the authors' own work, it should be cited transparently per journal policy.","section":"References"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper delivers a genuinely broader view of AI risk coverage than prior media studies: 27 countries, six regions, and political-bias stratification as a lens. That is the real contribution, and it is a useful one for anyone doing AI governance or risk-assessment design. The top-level pattern the authors find — societal risks dominate, legal/rights second, environmental risks negligible — is plausible and consistent with what I would expect. The paper is also transparent about its method and limitations. That said, the headline numbers are weaker than the prose suggests.\n\nThe main problem is the sample. Only 26.2% of eligible URLs were scraped, and the paper never checks whether that subset represents the full URL corpus on any observable dimension. With the US contributing 43% of the final sample, the regional and global percentages in Figure 1B could shift if paywalled or blocked articles emphasized different risks. The authors acknowledge paywalls and English-only coverage in the discussion, but acknowledgment is not the same as a representativeness analysis. This is fixable: weighting by domain or outlet circulation, comparing scraped vs. unscraped domains on metadata (country, outlet type, GDELT tone), or reporting a range of estimates.\n\nSecond, the LLM annotation has macro F1 of 0.60, and the prevalence ranking is built only from six categories with recall ≥0.7. That is a defensible choice, but it means the \"global prevalence\" claim covers only part of the taxonomy. Several of the 16 \"emerging\" categories have low LLM F1, so those extensions are weaker than the main ranking. The authors do say this, but it is easy for a reader to miss.\n\nOne concrete issue worth fixing: in Appendix A.6, the prompt's definition for Existential Threats looks like a copy-paste of the Ethical Impact definition. If that is what was sent to the model, it is a real bug; if it is just a typo in the paper, it needs correcting either way.\n\nThe paper is descriptive, so the selection problem does not sink a causal claim — there isn't one. But it does mean the central ranking should be presented as an estimate over a convenience sample with confidence intervals and robustness checks. I would send this to peer review. The global scope and political-bias angle are enough to warrant referee time, and the flaws are addressable in revision. A careful reviewer should push on sample representativeness and the annotation pipeline before the numbers are treated as established.\n\nWho should read it: AI governance folks, risk-assessment researchers, and media scholars. I would bring it to a reading group.","headline":"Useful global map of AI risk coverage in English-language news, but the headline prevalence ranking rests on an unverified 26.2% scrape sample and a partial taxonomy.","tokens_in":29247,"tokens_out":3091,"would_cite":true,"duration_ms":27587,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Global news coverage of AI skews toward societal risks, study of 42,853 articles finds.","keywords":["AI risks","news media coverage","risk prioritization","societal risks","political bias","AI governance","GPT-4o annotation","global media analysis"],"falsifier":"If a complete corpus that includes paywalled and non-English articles from the same 27 countries placed a different category at the top or lifted environmental or existential coverage above the 1.9% and 7.2% shares reported here, the paper's claim of a global societal-risk skew would fall.","tokens_in":28266,"feed_emoji":"🌍","tokens_out":6295,"duration_ms":55399,"temperature":0.7,"pith_summary":"The paper tries to establish that, when global English-language news coverage of AI is read as a record of public concerns, one skew shows up consistently: societal risks dominate, then legal and rights-related risks, with cognitive, content-safety, existential, and environmental risks trailing far behind. The authors analyze 42,853 articles from 27 countries across six regions and argue that this ordering reflects how media set the agenda for what publics and policymakers treat as important. They extend an existing AI risk taxonomy with 16 newly observed categories and show that the political leaning of an outlet changes which risks get reported. A reader should care because this offers a bottom-up, geographically inclusive source of evidence for AI risk assessment that is not limited to expert or corporate perspective.","feed_headline":"Half of global AI news coverage focuses on societal risks","feed_subtitle":"In 42,853 articles across 27 countries, existential and environmental harms get far less attention than economic and political ones.","key_machinery":"The central object is the emerging six-category risk scheme and the annotation pipeline that produces it. An LLM summarizes negative impacts from each article, yielding 47,731 impact statements; two human annotators label 1,060 of them against the AIR-taxonomy's Level-2 categories plus new categories as they emerge, reaching an inter-coder reliability of 0.97; the same LLM then classifies the full set, and the six categories with recall of at least 0.70 are kept for prevalence reporting. The AIR-taxonomy is a risk categorization distilled from government regulations and company policies, and the paper uses it as the baseline it extends with 16 novel categories proposed as eight additions at Level-1.","core_discovery":"The paper claims that global news coverage of AI, as measured in a sample of 42,853 English-language articles from 27 countries, prioritizes Societal Risks (50.6% of 16,312 articles with negative impacts), followed by Legal & Rights-related Risks (32.9%), Cognitive Risks (14.2%), Content Safety Risks (8.3%), Existential Risks (7.2%), and Environmental Risks (1.9%). It further claims that this skew holds across all six regions, with societal risks the most prevalent category everywhere, and that political bias shapes coverage: fringe media report more on risks than least-biased outlets, right-biased media lead on existential risks and underreport environmental ones, and left-biased media emphasize legal and rights-related issues. The paper also presents 16 risk categories absent from the AIR-taxonomy and proposes grouping them into eight additions at the top level, arguing that expert-driven taxonomies omit risks that are salient in journalistic coverage.","pith_inferences":["The published percentages describe only the 26.2% of URLs that could be scraped; a replication covering paywalled and non-English national outlets would test whether the global ranking survives.","The six chosen categories were selected partly on LLM recall, so categories excluded for annotation quality, such as governance and information risks, may still be central to public discourse and merit targeted study.","The word 'prioritization' here means coverage prevalence, not audience attitudes; linking article shares to survey measures of public concern would test the agenda-setting interpretation.","If the observed political-bias gaps hold in other languages, the design of AI risk communication could be adapted per outlet ideology rather than treated as a neutral factual transfer."],"forward_implications":["If the prevalence skew holds, stakeholders using media as a proxy for public concern will see societal and legal/rights harms as the issues publics most strongly associate with AI.","The very low share of environmental coverage (1.9%) suggests environmental costs are being underweighted in public discourse relative to their physical scale, a gap regulators and companies could address.","The finding that left- and right-fringe outlets report risks more than least-biased ones implies that AI risk debates are partly polarized, so governance efforts should anticipate political framing.","Regional differences in rank order, such as content-safety risks ranking third in the Middle East and legal/rights risks peaking in Oceania, imply that global risk assessments need regional calibration rather than a single priority list."],"supporting_citations":[{"why":"Supplies the AIR-taxonomy, the baseline coding scheme the paper extends with 16 new risk categories.","marker":"[110]"},{"why":"Supplies the event database used to retrieve 1,218,058 initial AI-related article URLs from 30 countries.","marker":"[61]"},{"why":"Supplies the list of national English-language news outlets used to filter URLs by country.","marker":"[69]"},{"why":"Supplies the domain-level political bias ratings used in the outlet-bias analysis.","marker":"[25]"},{"why":"Supplies the scraping library used to retrieve article text for the final sample.","marker":"[81]"},{"why":"Supplies the zero-shot prompting approach and evaluation framing for classifying whether articles describe AI impacts.","marker":"[84]"},{"why":"Computes the inter-coder reliability statistic (0.97) that supports the consistency of the human-annotated risk categories.","marker":"[65]"},{"why":"Supplies the 41 AI keywords and the earlier news-based impact-assessment approach this study builds on.","marker":"[2]"}],"fun_headline_variants":["Societal risks dominate AI news across 27 countries","AI risk news skewed by politics, region, study finds","News media spotlight societal AI harms, ignore environmental","16 AI risk types missing from expert taxonomy, news shows","Political bias shapes which AI risks make headlines"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The 42,853 scraped articles, which are 26.2% of the eligible links and are dominated by the United States, are assumed to represent the AI risk coverage of all English-language national news in the 27 countries.","fun_headline_variants_meta":{"raw":{"variants":["Societal risks dominate AI news across 27 countries","AI risk news skewed by politics, region, study finds","News media spotlight societal AI harms, ignore environmental","16 AI risk types missing from expert taxonomy, news shows","Political bias shapes which AI risks make headlines"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000757,"raw_usage":{"total_tokens":3409,"prompt_tokens":1036,"completion_tokens":2373,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":652,"completion_tokens_details":{"reasoning_tokens":2298}},"tokens_in":652,"tokens_out":2373,"duration_ms":16047,"temperature":1.0,"reasoning_tokens":2298,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T15:26:55.292163+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"If a complete corpus that includes paywalled and non-English articles from the same 27 countries placed a different category at the top or lifted environmental or existential coverage above the 1.9% and 7.2% shares reported here, the paper's claim of a global societal-risk skew would fall.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the list of national English-language news outlets used to filter URLs by country."}],"review_version":1}