{"id":"9e94a331-1ac7-4fed-ad1f-aaca045bf963","arxiv_id":"2501.00959","paper_version":3,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"IGGA is a new dataset of 160 publicly available industry documents on generative AI use, accompanied by descriptive word-frequency and similarity analyses.","lead":"This paper introduces IGGA, a dataset of 160 documents describing how companies and other organizations approach the use of generative AI in the workplace. The authors position the collection as a resource for natural language processing research on AI governance and requirements engineering.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central dataset claim is undercut by curation and metadata errors in Supplementary Table 1; the paper does not demonstrate that the 160 entries are unique, correctly attributed GAI policies.","rationale":"The reader identified the same load-bearing weakness: the dataset's value depends on the accuracy and uniqueness of its 160 entries. My review of Supplementary Table 1 confirms that the weakness is real and substantial. Duplicate organizations, incorrect continent assignments, third-party articles counted as organizational policies, and at least one citation pointing to the wrong document directly contradict the abstract's claim of a rigorously curated, representative dataset. The paper also contains internal inconsistencies about the number of continents and even calls the sources 'universities' in Figure 1. Because these are errors in the primary artifact, not just in the surrounding interpretation, they undermine both the descriptive contribution and any downstream NLP or requirements-engineering use. The lack of a public code repository further prevents independent verification, though the curation audit alone is sufficient to identify the problem. I therefore agree with the reader's verdict: the submission should not be accepted as-is. If the authors corrected the metadata, removed duplicates, reclassified non-policy entries, and made the audit reproducible, a revised version could be reconsidered, but the current dataset description is unreliable.","tokens_in":20999,"tokens_out":3478,"duration_ms":34666,"concrete_test":"Perform an independent audit of all 160 rows in Supplementary Table 1 and the Dataverse release: fetch each cited URL and verify (1) it is the named organization's own page, not third-party coverage; (2) it contains actual GAI/LLM use guidance or policy, not a product case study or news item; (3) no duplicate organization or overlapping document; and (4) country/continent fields match the organization's registered headquarters. Then recompute the reported word count, sector counts, and continent counts after removing invalid or duplicate entries. If more than 10% of entries fail any of these checks, or if the corrected counts change the claimed 160/14/seven-continent composition, the dataset as described does not support the paper's representativeness and NLP-utility claims.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that IGGA is a curated corpus of 160 industry GAI guidelines and policy statements from named organizations, suitable as an NLP resource. That claim fails if the entries are not what they are described to be. The evidence in Supplementary Table 1 shows this is not a marginal concern: (i) duplicates exist—CSL Limited appears as rows 9 and 128 with the same URL, Infosys appears as rows 26 and 86, and New York Times as rows 52 and 110; (ii) row 26's Infosys entry points to the ASMPT 2023 Interim Report URL rather than an Infosys document; (iii) continent labels are wrong in multiple rows—Spain-based amadeus, Grupo Planeta, and Grupo ACS are assigned to South America, and Sanofi is listed as Brazil; (iv) many entries are news articles or third-party commentary rather than the organization's own guideline or policy (e.g., China Daily, JPMorgan via Forbes, McKinsey via HRD, Sea via Bloomberg, Lendlease via AFR, WPP blog). The paper itself is internally inconsistent about coverage: the abstract says six continents, the body says seven including Antarctica via multinational representation, and Figure 1 text refers to '160 universities.' Because the dataset's value proposition is exactly that these are vetted, unique, correctly attributed organizational policies, these errors are load-bearing. A downstream NLP analysis built on IGGA would inherit a skewed sample and potentially miscount policy adoption. This is a correctness issue in the primary artifact, not a stylistic concern.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces IGGA, a curated dataset of 160 industry guidelines and policy statements for generative AI use, archived on Harvard Dataverse, with metadata and full text in Word, PDF, and Excel formats. The authors describe a collection methodology, a dual categorization by sector and geography, and a descriptive NLP-based validation workflow (frequency analysis and TF-IDF/cosine-similarity heatmaps). The stated purpose is to support requirements-engineering NLP tasks such as model synthesis, ambiguity detection, and document structure analysis.","tokens_in":21304,"tokens_out":5272,"duration_ms":48536,"significance":"If the dataset were accurate and comprehensive, it would fill a real gap: a structured corpus of corporate GAI policies with geographic and sector metadata could support reproducible studies of AI governance. The paper's openly archived dataset (DOI 10.7910/DVN/4LOXUW) and the detailed preprocessing documentation are practical strengths, and the multi-format release lowers access barriers. However, the dataset's value proposition rests entirely on curation quality—uniqueness, correct attribution, and genuine policy status—and the current version does not deliver that. Substantial re-curation is required before the resource can support reliable downstream NLP.","major_comments":[{"comment":"The same organization and document appear more than once: CSL Limited (rows 9 and 128, with the identical URL), Infosys (rows 26 and 86), and The New York Times (rows 52 and 110). Because the abstract and Data Records claim 160 guidelines, and the true unique count is lower, the headline number of the dataset is not supported by the submitted table.","section":"Supplementary Table 1, rows 9/128, 26/86, 52/110"},{"comment":"The Infosys entry titled \"Infosys Responsible AI\" points to the URL https://www.asmpt.com/site/assets/files/63620/e_00522ir-20230823.pdf, which is the ASMPT 2023 Interim Report, not an Infosys document. This is a direct attribution error that makes the per-organization policy mapping unusable for that row.","section":"Supplementary Table 1, row 26"},{"comment":"Several geographic metadata entries are wrong: amadeus it (Spain), Grupo Planeta (Spain), and Grupo ACS (Spain) are listed as South America; Sacyr is listed as Argentina even though the company is headquartered in Spain; Sanofi is listed as Brazil even though it is headquartered in France. These errors undermine the continental distribution claims in the abstract and the Technical Validation section.","section":"Supplementary Table 1, rows 37, 57, 77, 78, 127"},{"comment":"Many entries are third-party news articles or vendor commentary rather than the named organization's own guideline or policy statement—for example, JPMorgan via Forbes (row 41), Sea Group via Bloomberg (row 32), Lendlease via Australian Financial Review (row 79), and McKinsey via HRD (row 81). The Methods section states that collected materials are official guidelines and policy statements, so these entries violate the stated inclusion criteria and dilute the construct validity of the dataset.","section":"Methods and Supplementary Table 1, rows 32, 41, 79, 81"},{"comment":"The manuscript is internally inconsistent about what was collected: the abstract says the dataset represents six continents, while the Background & Summary and Technical Validation say seven continents (including Antarctica via multinational representation); Figure 1's text refers to \"160 universities\" and the Data Records section to \"160 academic guidelines,\" even though the paper claims to describe company guidelines. These inconsistencies obscure the sampling frame and must be resolved for the dataset description to be trustworthy.","section":"Abstract, Background & Summary, Figure 1, Data Records"},{"comment":"The validation analyses (keyword frequency, TF-IDF heatmap, cosine similarity) take the table's labels at face value and describe text patterns within the collected documents. They do not check for duplicate entries, misattributed URLs, incorrect continent labels, or whether a document is actually a policy. To support the central claim that this is a vetted set of unique, correctly attributed organizational policies, the validation section needs an explicit curation audit.","section":"Technical Validation"}],"minor_comments":[{"comment":"There are several typographical errors, including \"using usedin\" in the text-mining paragraph, \"Jupyer notebook\" in Computational Tools, and \"indisutry sectors\" in the keyword-frequency subsection.","section":"Text Mining and Computational Processing"},{"comment":"The figure captions are misleading: the schematic overview is labeled Figure 1 and the word-frequency chart is also labeled Figure 1, while the heatmap is called Figure 2 but appears before the second Figure 1; renumber the figures consistently.","section":"Figure numbering"},{"comment":"The Excel file is described as containing fields such as \"University,\" but the dataset covers companies; the field name should be \"Company\" or the description should be corrected.","section":"Data Records, Excel file description"},{"comment":"Several country assignments are questionable: State Bank of India is listed under Japan, Straker Translations (New Zealand) is listed under Australia, and WPP plc is listed under the USA although the company is UK-headquartered; these entries should be verified or corrected.","section":"Supplementary Table 1, rows 44, 69, 140"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a data-descriptor submission. I see no evidence of intended deception; the errors are consistent with careless or automated curation. The authors should re-verify every row of Supplementary Table 1, remove duplicates, correct attributions, and likely replace a substantial number of news-article entries with actual policy documents. That is a substantial revision, but it is feasible within the scope of a data-paper revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The idea here is sound and worth having: a curated set of corporate guidelines and policy statements for generative AI, organized by sector and region, with full text and metadata. That is a legitimate curation contribution, and hosting it on Dataverse with a DOI is the right move. If the dataset were clean, it would be a useful starting point for requirements engineering and AI-governance research.\n\nBut the paper as submitted does not support its own claim. Supplementary Table 1 is the core artifact, and it has load-bearing problems. There are duplicate entries (CSL Limited appears twice, Infosys twice, New York Times twice), which means the true unique count is below 160. Some rows point to the wrong document entirely: the Infosys entry for row 26 links to an ASMPT interim report. Continent labels are wrong in plain sight: Spain-based companies are assigned to South America, Sanofi is labeled as Brazil, and a UAE national strategy is filed under an Egyptian company. And many rows are not policies or guidelines at all: they are news articles, blog posts, or third-party commentary (China Daily, JPMorgan via Forbes, McKinsey via HRD, Sea via Bloomberg, and others). If the dataset is supposed to be vetted, unique, correctly attributed organizational policies, these are not minor blemishes; they break the value proposition.\n\nThe text analyses are descriptive and lightweight: TF-IDF, keyword counts, a cosine-similarity heatmap. They do not validate the dataset's quality; they just describe whatever is in it. The paper also contains internal inconsistencies: the abstract says six continents, the body says seven, a figure caption says \"160 universities,\" and the code is \"available upon request\" rather than shipped. None of these are fatal on their own, but together they reinforce the impression that the curation was not checked carefully.\n\nWho is this for? Someone who wants a seed list of corporate AI governance documents and is willing to verify every entry themselves. That person would get some value from the dataset, but not from the paper's claims.\n\nMy recommendation: this deserves peer review, not desk rejection, because the concept is useful and the errors are fixable. But it needs major revision: the authors must correct the metadata, remove or reclassify non-policy entries, supply the actual dataset for inspection, and fix the internal contradictions. As submitted, the central claim is not reliable.","headline":"Useful dataset idea, but the paper's central claim is undercut by verification failures in its own supplementary table.","tokens_in":21741,"tokens_out":1279,"would_cite":false,"duration_ms":13195,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"IGGA is a curated, publicly available corpus of 160 industry guidelines and policy statements for generative AI and large language models, spanning 14 sectors and six continents, intended as a resource for requirements-engineering NLP…","keywords":["generative AI governance","industry policy dataset","requirements engineering","NLP corpus","AI ethics guidelines","LLM workplace policies","text mining","cross-sector comparison"],"falsifier":"Spot-check all 160 entries by downloading each cited link and verifying that it is an official company or news-source document that actually states a policy or guideline for generative AI use, that the country, continent, and sector metadata match the organization's actual location, and that no two entries are duplicates.","tokens_in":106,"feed_emoji":"📄","tokens_out":4810,"duration_ms":103613,"temperature":0.7,"pith_summary":"This paper introduces IGGA, a curated corpus of 160 industry guidelines and policy statements governing the use of generative AI and large language models in workplace settings. The dataset spans 14 industry sectors and claims global coverage, drawing on official company websites and news sources. The authors' purpose is to give NLP researchers and requirements engineers a ready-made, broadly representative text collection for tasks like policy synthesis, ambiguity detection, and requirement categorization. The paper argues that the corpus is validated by qualitative analyses of geographic and sectoral diversity, and by text-mining metrics that reveal thematic alignment across industries. If the dataset is sound, it would lower the barrier to studying how companies actually govern generative AI.","feed_headline":"160 industry AI guidelines now form one open dataset","feed_subtitle":"Corpus spans 14 sectors and is built for requirements-engineering NLP tasks.","key_machinery":"The load-bearing object is IGGA itself, a cataloged collection of 160 documents together with metadata fields such as company, industry, country, and continent. The argument that the corpus is useful is carried by a standard text-mining pipeline: preprocessing with tokenization, stopword removal, stemming, and lemmatization, followed by TF-IDF vectorization, cosine-similarity heatmapping across 14 sectors, and keyword frequency analysis. The curated selection protocol and the metadata schema are what make the collection a dataset rather than a mere list of links.","core_discovery":"The central claim is that IGGA constitutes a systematically collected, structured dataset of 160 industrial guidelines and policy statements for generative AI and LLM use, containing 104,565 words and available in Word, PDF, and Excel formats, and that it can serve as a benchmark and training resource for NLP tasks in requirements engineering. The authors report a selection process that prioritized industry leadership, geographic diversity, sectoral representation, and the existence of official guidance, replacing companies that lacked such documents. They further claim that two validation analyses—one on inclusiveness across continents and sectors, and one on text structure and keyword content via TF-IDF, cosine similarity, and clustering—demonstrate the dataset's breadth and its capacity to expose shared and sector-specific AI governance themes.","pith_inferences":["If the dataset's quality holds, it could serve as a seed for tracking how corporate AI governance evolves over time, since new guidelines can be added to the same schema and compared against the existing corpus.","The paper's own examples suggest that several entries are news reports about company policy rather than the policy documents themselves; a stricter distinction between primary policy and secondary coverage would strengthen the dataset's claims.","Because the corpus includes both policy statements and promotional or news items, downstream NLP models trained on it would need to handle mixed genres; explicitly tagging document genre would make the benchmark more reliable.","The geographic metadata appears inconsistent in places, which suggests that continent-level comparisons drawn from the current metadata should be treated as provisional until corrected."],"forward_implications":["NLP researchers could use IGGA to train and evaluate models for extracting requirements from AI governance policies, since the corpus is already structured and publicly available.","The dataset enables cross-sector and cross-region comparisons of how companies phrase AI rules, because the metadata attaches industry and continent to each document.","Requirements engineers could use IGGA to benchmark ambiguity detection and requirements categorization, as the paper suggests further annotation could support those tasks.","The corpus could support policy synthesis and abstraction identification tasks, per the paper's stated NLP use cases.","The open license allows unrestricted reuse, so the dataset can grow with contributions and be integrated into larger governance-text collections."],"supporting_citations":[{"why":"Supplies the motivating gap: only 26% of organizations have a generative AI policy, with 23% developing one.","marker":"[1]"},{"why":"The dataset record itself, cited as the persistent identifier for the IGGA corpus that the paper describes and distributes.","marker":"[2]"},{"why":"Microsoft's AI usage policy, an example of a primary company guideline that the corpus is built from.","marker":"[13]"},{"why":"Alphabet's Google AI principles, a canonical source document representing the technology sector.","marker":"[15]"},{"why":"Fujitsu's generative AI use guidelines, illustrating the Asian corporate coverage in the dataset.","marker":"[24]"},{"why":"BBC AI Principles, a media-sector policy statement that anchors the journalism and news media category.","marker":"[116]"},{"why":"Pfizer's AI policy position, a healthcare-sector example that grounds the pharmaceutical research category.","marker":"[122]"}],"fun_headline_variants":["160 industrial AI guidelines become open dataset","New dataset compiles 160 corporate AI policy statements","IGGA: 160 guidelines for generative AI in industry","Dataset captures 160 AI guidelines from 14 sectors","Open corpus of 160 company AI governance policies"],"cache_read_input_tokens":23936,"weakest_assumption_plain":"The dataset's value depends on each of the 160 listed documents being a genuine, unique guideline or policy statement from the named organization, with accurate metadata about country, continent, and sector.","fun_headline_variants_meta":{"raw":{"variants":["160 industrial AI guidelines become open dataset","New dataset compiles 160 corporate AI policy statements","IGGA: 160 guidelines for generative AI in industry","Dataset captures 160 AI guidelines from 14 sectors","Open corpus of 160 company AI governance policies"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000527,"raw_usage":{"total_tokens":2495,"prompt_tokens":846,"completion_tokens":1649,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":462,"completion_tokens_details":{"reasoning_tokens":1576}},"tokens_in":462,"tokens_out":1649,"duration_ms":10452,"temperature":1.0,"reasoning_tokens":1576,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:37:47.745697+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Spot-check all 160 entries by downloading each cited link and verifying that it is an official company or news-source document that actually states a policy or guideline for generative AI use, that the country, continent, and sector metadata match the organization's actual location, and that no two entries are duplicates.","supporting_citations":[{"cited_title":"BBC AI Principles","cited_arxiv_id":null,"evidence_quote":"BBC AI Principles, a media-sector policy statement that anchors the journalism and news media category."},{"cited_title":"Artificial Intelligence in Wargaming | Centre for Emerging Technology and Security","cited_arxiv_id":null,"evidence_quote":"Pfizer's AI policy position, a healthcare-sector example that grounds the pharmaceutical research category."}],"review_version":1}