{"id":"87b964b4-0698-4939-8a19-8e1f518c9f97","arxiv_id":"2502.03421","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Topic modeling and text summarization of 15 Nasdaq firms' Covid-19 press releases shows CSR messaging shifting from employee safety and operations toward community help, testing, and student learning by mid-2020.","lead":"This paper applies four established text-analysis methods, topic modeling and extractive summarization, to Covid-19 press releases from the 15 largest Nasdaq firms. It shows how corporate CSR messaging changed between early and later 2020 and offers a template for policymakers analyzing large text corpora.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central workflow claim rests on unverified outputs: Tables 4–5 (doc2vec topics and retrieval) are empty, and the Amazon summaries are shown without any reference-based or human evaluation, so 'understanding the gist' is asserted rather than demonstrated.","rationale":"The reader's weakest assumption focuses on the validity of press releases as a proxy for actual CSR engagement. That is a real limitation, and the paper itself concedes it in the Discussion when it calls for multiple sources to get a 'more holistic and unbiased view.' However, the more immediately load-bearing gap for the paper's central claim is that the outputs themselves are not validated: the doc2vec results are literally missing from Tables 4–5, and the summaries are shown without any metric, baseline, or comparison. Even if press releases were perfectly truthful, the paper would still need to show that the extracted topics and summaries are faithful to those press releases. The coherence scores in Tables 2–3 are reported without baselines or confidence intervals, and the alpha parameter for summary length is chosen by subjective inspection. These issues do not contradict the possibility that the methods work, but they mean the central demonstration is currently unverified. A reproduction with the archived Amazon corpus and a ROUGE comparison would settle the matter: if the summarizers do not beat a random-sentence baseline, the policy-relevant claim fails on its own terms regardless of source validity. This supports the reader's CONDITIONAL verdict rather than changing it, because the paper is plausible but incomplete as a validated demonstration.","tokens_in":15246,"tokens_out":4672,"duration_ms":46910,"concrete_test":"Using the Wayback Machine to reconstruct Amazon's March–August 2020 COVID-19 blog/press-release corpus, run the described TextRank and TF-IDF summarizers, and compare their top-5/10 sentence outputs with the paper's reported summaries via exact overlap; report ROUGE-L against a gold summary written by two independent annotators. If the reproduced summaries differ in focus or ROUGE-L is not above a random-sentence baseline, the claimed ability to 'sum up the gist' is not demonstrated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"To accept the central claim—that these four NLP methods give policy makers a reliable 'gist' of large document corpora—the outputs must be reproducible and faithful to the input text. The paper does not yet show this. First, the doc2vec portion, touted as the more powerful semantic model, is supported by Tables 4 and 5, which are empty in the supplied manuscript; the claimed 'localized topics' and keyword-based document retrieval cannot be inspected or replicated as specified. Second, the extractive summaries are presented as raw outputs with no evaluation: no ROUGE or similar metric, no comparison against human-written or reference summaries, no inter-annotator agreement on the manually assigned LDA topic labels, and no sensitivity analysis beyond picking alpha=1.5 and 2 as 'optimal interpretable summaries.' Without a baseline showing the selected sentences outperform random sentence selection, a reader cannot tell whether the method is summarizing the corpus or merely echoing frequent words. Third, the Discussion explicitly concedes that press releases alone may be a biased source and recommends additional sources for a 'more holistic and unbiased view'; that is a reasonable limitation, but it qualifies the policy implication that regulators can base standardization on these outputs. None of these points imply the methods are wrong; they mean the paper's demonstration is currently a set of examples, not a validated workflow.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a workflow for policymakers to extract the 'gist' of large unstructured text corpora using four NLP techniques: LDA and doc2vec for topic modeling, and TF-IDF sentence ranking and TextRank for extractive summarization. It applies these methods to Covid-19-related corporate press releases from the top 15 Nasdaq firms (March-August 2020), reporting LDA topics and coherence scores for two pandemic periods, claiming doc2vec yields localized semantic topics and keyword-based document retrieval, and presenting extractive summaries of Amazon's Covid-19 CSR communications. The paper argues the pipeline is replicable in other social decision-making contexts and can inform CSR standardization and regulation.","tokens_in":15564,"tokens_out":4097,"duration_ms":39092,"significance":"If the workflow were validated, the paper would offer a practical, low-cost template for stakeholders to monitor corporate CSR messaging during crises. Its strengths are the use of a real-world Covid-19 corpus, the coverage of four standard, reproducible methods, the explicit mathematical descriptions, and the honest acknowledgment that self-reported press releases may provide a biased view. However, the evidence in the manuscript is currently insufficient to establish the central claim: the doc2vec results are missing, the summaries are not evaluated against any baseline or reference, and the LDA topic labels are manually assigned without validation. The significance of the paper therefore remains prospective rather than demonstrated.","major_comments":[{"comment":"The doc2vec results are presented as key evidence for the semantic topic-modeling and document-retrieval capabilities, but Tables 4 and 5 are empty in the supplied manuscript. The text reports cosine similarity scores and document IDs (1622 and 1511) without any visible data, so the claims about 'localized topics' and keyword-based retrieval cannot be inspected or replicated. Please provide the actual tables, the full training details (vector dimension, window size, epoch count, HDBSCAN/UMAP parameters), and the exact retrieval procedure used to obtain the scores.","section":"§2.3, Tables 4–5"},{"comment":"The extractive summaries from the TF-IDF sentence-ranking method and TextRank are presented as raw outputs with no reference-based evaluation (e.g., ROUGE), no human assessment, and no baseline such as random sentence selection or lead-3. Without such a baseline, the assertion that these methods 'yield informative summaries' is unsupported; the reader cannot tell whether the summaries are faithful or merely echo frequent words. The choice of alpha = 1.5 and 2 is justified only as yielding 'optimal interpretable summaries,' which is circular unless the optimality criterion is defined. Please add quantitative or human evaluation, or explicitly reframe the results as illustrative examples rather than a validated workflow.","section":"§2.3, summarization results"},{"comment":"The LDA topic labels such as 'Prioritize Safety' and 'Help Community' are manually assigned from keyword lists, but there is no inter-annotator agreement, no validation of the labels, and no confidence interval or baseline for the reported coherence scores. The coherence values alone do not establish that the topics are meaningful, and the cross-period comparison in the Discussion rests on these unvalidated labels. Please provide validation (e.g., multiple annotators, comparison with random topic baselines, or a stated coherence threshold with justification) or temper the interpretive claims.","section":"§2.3, Tables 2–3"},{"comment":"The abstract claims the steps 'can be replicated,' but the manuscript does not provide the collected corpus, the number of press releases per firm, the scraping and preprocessing protocol (beyond generic NLTK/Gensim references), or any code or data availability statement. Combined with the missing doc2vec tables, this makes the workflow impossible to reproduce as described. Please include a detailed data protocol and make the data and code available, or explicitly state the paper is a conceptual demonstration rather than a fully replicable pipeline.","section":"Abstract and §2"}],"minor_comments":[{"comment":"The parameter alpha in the relevance formula is not the same as the LDA Dirichlet hyperparameter alpha discussed earlier; the text switches between them without distinction, which is confusing. Please rename one of them.","section":"§2.1.1, Eq. (2)"},{"comment":"The notation in Eq. (1) is nonstandard and not referenced later; please align it with standard LDA notation (Blei et al., 2003) or remove it.","section":"§2.1.1, Eq. (1)"},{"comment":"The text says the top 10 frequent words are 'shown in Table 4' and the top 10 raw sentence scores are 'shown in Table 5,' but Tables 4 and 5 contain doc2vec topics and retrieval, while the word frequencies and scores appear in Tables 6 and 7. This mismatch makes the summarization description hard to follow and should be corrected.","section":"§2.3, table numbering"},{"comment":"The reference list contains multiple errors: reference 44 is used twice for different works, the in-text citation '[32, 33)' has a misplaced parenthesis, 'Bercelona' should be 'Barcelona,' and Table 3 contains 'June-Aug 20220.' Please proofread the entire manuscript.","section":"References and typos"}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears to be an early draft with missing tables and inconsistent referencing. The application topic is timely and the proposed workflow is sensible, but the missing doc2vec output and the lack of any summarization evaluation are load-bearing gaps that prevent acceptance in a serious journal. I would encourage the editor to request a revision with the missing evidence rather than reject outright, as the methodological direction is sound and the gaps are fixable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a competent application of off-the-shelf NLP methods to a nicely chosen crisis corpus, but the paper currently cannot fully support its own workflow claim because key outputs (doc2vec topics and retrieval) are missing from the manuscript and the summaries are presented without any evaluation.\n\nWhat's new: the empirical outputs themselves—LDA topic sets for March–May and June–August 2020 from the top-15 Nasdaq press releases, and the Amazon summaries. The methods are textbook LDA, doc2vec+UMAP+HDBSCAN, TF-IDF ranking, and TextRank. That's fine; the paper doesn't claim new methods. The early-vs-later topic split is intuitively interesting: early press releases balance employee safety and relief efforts; later ones shift to community help, testing, student learning, and spreading information. I found those tables plausible.\n\nWhat it does well: it is clearly written, hyperparameters are stated, and the workflow could be followed by a policy analyst. The limitations section openly concedes that press releases alone are a self-reported source and that adding media and social media would give a 'more holistic and unbiased view.' That is the right thing to do, and it qualifies but does not destroy the policy implication.\n\nSoft spots, in order: (1) Tables 4 and 5 are empty in the supplied manuscript, so the doc2vec results—the part claimed to be more powerful and interpretable—cannot be inspected or replicated. That is not minor for that section. (2) The extractive summaries have no evaluation: no ROUGE or baseline, no random-sentence comparison, no inter-annotator agreement on the manually assigned LDA topic labels. The coherence scores in Tables 2–3 have no uncertainty or comparison baseline, so they are hard to read as evidence. (3) The alpha values 1.5 and 2 are chosen because they yield 'optimal interpretable summaries'; that is tuning on the outcome, and it should be flagged as illustrative rather than validated. (4) No code or corpus is released, which limits the 'replicable steps' claim for other policy contexts.\n\nNet: the central demonstration is plausible—these standard methods do produce readable topics and summaries from this corpus. The paper is not circular, and it is not overclaiming recklessly. But the missing tables and lack of any evaluation mean the workflow is currently a set of examples, not a validated pipeline. Who it's for: applied researchers and policy analysts who want a concrete recipe for topic modeling and extractive summarization of CSR text; they will get the recipe but should not treat the outputs as benchmarked.\n\nRecommendation: send to peer review, yes, provided the authors can restore Tables 4–5 and ideally add a basic evaluation or at least an explicit statement that the summaries are illustrative. It deserves referee time, but only with the expectation of a major revision.","headline":"An honest application of standard NLP methods to Covid-19 CSR press releases, but missing doc2vec tables and lack of summary evaluation keep it from being a validated workflow.","tokens_in":16053,"tokens_out":2318,"would_cite":false,"duration_ms":22131,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that standard NLP methods—LDA topic modeling, doc2vec, TF-IDF sentence ranking, and TextRank—can extract the policy-relevant gist from a large corpus of corporate Covid-19 press releases.","keywords":["topic modeling","Latent Dirichlet Allocation","doc2vec","text summarization","TextRank","corporate social responsibility","Covid-19 press releases","natural language processing"],"falsifier":"Compare the topic distributions extracted from firms' press releases with independently audited records of their actual crisis behavior—verified donations, employee-safety complaints, or inspection findings. If firms with weak verified performance produce press-release topics indistinguishable from firms with strong verified performance, the pipeline is recovering messaging rather than engagement, and the policy claim would not hold.","tokens_in":15051,"feed_emoji":"📄","tokens_out":11038,"duration_ms":91085,"temperature":0.7,"pith_summary":"This paper argues that policymakers facing an overload of self-published corporate text can use four widely available NLP tools—LDA topic modeling, doc2vec, TF-IDF sentence ranking, and TextRank—to extract the gist of corporate social responsibility (CSR) activity. The workflow is demonstrated on Covid-19 press releases from the top 15 Nasdaq firms for March through August 2020, and on Amazon's pandemic-response reports. If the claim holds, regulators, media, NGOs, and consumers can monitor what companies say they are doing for employees and communities during a crisis, compare firms and sectors, and set industry-level best-practice expectations. The contribution is a replicable recipe using existing methods rather than a new algorithm.","feed_headline":"Topic models and summaries condense corporate Covid-19 CSR claims","feed_subtitle":"Off-the-shelf NLP turns 15 firms' 2020 press releases into topics like healthcare, relief, and employee safety.","key_machinery":"The load-bearing machinery is a pipeline pairing two topic models with two summarizers. LDA—a generative model treating each document as a mixture of topics—produces top-keyword lists per period, with a relevance weighting that suppresses corpus-wide frequent words. Doc2vec—a neural model that learns a vector for each document—projects documents into a semantic space, which UMAP dimension reduction and HDBSCAN density-based clustering turn into localized topics, and which allows keyword-based retrieval of supporting sentences. For summarization, TF-IDF sentence ranking scores each sentence by the weight of its words, while TextRank—a graph-based method that ranks sentences by their cosine similarity to other sentences—produces extractive summaries. A central tuning mechanism is the alpha multiplier on the average sentence score in the word-frequency summarizer: lower alpha yields a fuller summary, higher alpha yields a shorter digest.","core_discovery":"The paper's central discovery is that unsupervised topic modeling and extractive summarization, applied to corporate press releases, can surface the main CSR themes firms emphasized during the Covid-19 pandemic. In the early period LDA produced topics labeled prioritize safety, flexible working, accelerated solutions, healthcare efforts, and relief efforts; in the later period the emphasis shifted to helping communities, testing, student learning, and spreading information. Doc2vec adds semantic clustering and keyword-based retrieval of the actual sentences behind a topic. Word-frequency scoring and TextRank then generate readable summaries of a single firm's output, with the summary length controlled by an alpha cutoff. The paper reads these results as evidence that policymakers can run this pipeline on large unstructured text collections to understand corporate societal engagement and guide the standardization of CSR practices.","pith_inferences":["Beyond the paper, the same workflow could be run on paired corpora—company press releases alongside media and social-media text—so topic models quantify the gap between self-report and independent coverage instead of just acknowledging it.","The 2020 topic lists could serve as a crisis-CSR lexicon; reapplying the pipeline during a later crisis would test whether firms' stated priorities shift with public scrutiny.","The alpha-tuning knob could be evaluated as a decision variable: future studies could measure whether summaries at different alpha values change stakeholder judgments or policy choices.","Because press releases are curated by firms, a natural extension is to apply the identical pipeline to the same firms' mandatory filings and check whether the CSR themes match."],"forward_implications":["A policymaker can apply the same four-step pipeline to any document corpus—CSR reports, news articles, NGO statements, or social media—and get comparable topic lists and summaries without custom model training.","Aggregating by industry makes it possible to see which CSR practices are common and which are neglected, and to standardize best-practice expectations per sector.","The gap between self-reported responses and actual conduct, illustrated in the paper by the Tesla example, becomes a governance question that regulators can close only by checking press-release topics against independent evidence.","Summary length becomes a policy choice: a smaller alpha gives a fuller picture for deliberation, while a larger alpha gives a concise digest for rapid decisions.","Companies themselves can use the methods to benchmark peers' crisis responses and adjust their own CSR strategies and communications."],"supporting_citations":[{"why":"Supplies the LDA generative model that produces the top-keyword topic lists for both pandemic periods.","marker":"[23]"},{"why":"Supplies the doc2vec distributed representation used for semantic topic clustering and keyword-based sentence retrieval.","marker":"[32]"},{"why":"Supplies the TextRank graph-based sentence-extraction algorithm used to summarize Amazon's Covid-19 response reports.","marker":"[61]"},{"why":"Provides the relevance and alpha-weighting guidance used to filter frequent but uninformative keywords in LDA.","marker":"[28]"},{"why":"Supplies HDBSCAN, which clusters the doc2vec document vectors into localized topics.","marker":"[37]"},{"why":"Supplies UMAP, which reduces the vector space before clustering and gives the doc2vec topics their spatial form.","marker":"[40]"}],"fun_headline_variants":["NLP pipeline distills Covid-era CSR claims from press releases","Topic modeling + TextRank summarize corporate Covid-19 CSR reports","LDA and Doc2vec reveal CSR themes firms pushed during Covid","Off-the-shelf NLP maps firms' Covid CSR priorities from releases","Policy tool: NLP condenses 15 firms' Covid CSR press releases"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The analysis stands on the assumption that a firm's press releases are an accurate enough record of what it actually did for employees and communities; if those releases routinely omit failures or exaggerate efforts, the topics and summaries will tell policymakers what companies want them to believe rather than what companies did.","fun_headline_variants_meta":{"raw":{"variants":["NLP pipeline distills Covid-era CSR claims from press releases","Topic modeling + TextRank summarize corporate Covid-19 CSR reports","LDA and Doc2vec reveal CSR themes firms pushed during Covid","Off-the-shelf NLP maps firms' Covid CSR priorities from releases","Policy tool: NLP condenses 15 firms' Covid CSR press releases"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000558,"raw_usage":{"total_tokens":2627,"prompt_tokens":891,"completion_tokens":1736,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":507,"completion_tokens_details":{"reasoning_tokens":1646}},"tokens_in":507,"tokens_out":1736,"duration_ms":10551,"temperature":1.0,"reasoning_tokens":1646,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T04:47:21.527003+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare the topic distributions extracted from firms' press releases with independently audited records of their actual crisis behavior—verified donations, employee-safety complaints, or inspection findings. If firms with weak verified performance produce press-release topics indistinguishable from firms with strong verified performance, the pipeline is recovering messaging rather than engagement, and the policy claim would not hold.","supporting_citations":[{"cited_title":"Blei, A.Y","cited_arxiv_id":null,"evidence_quote":"Supplies the LDA generative model that produces the top-keyword topic lists for both pandemic periods."},{"cited_title":"Mihalcea and P","cited_arxiv_id":null,"evidence_quote":"Supplies the TextRank graph-based sentence-extraction algorithm used to summarize Amazon's Covid-19 response reports."},{"cited_title":"Sievert and K","cited_arxiv_id":null,"evidence_quote":"Provides the relevance and alpha-weighting guidance used to filter frequent but uninformative keywords in LDA."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies HDBSCAN, which clusters the doc2vec document vectors into localized topics."}],"review_version":1}