{"id":"60a7ca5c-02c9-4cb5-8b42-5db625734ae9","arxiv_id":"2412.18442","paper_version":4,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"This SoK systematizes offensive AI from literature, briefings, a 549-person survey, and 12 expert statements, yielding a checklist and ten open problems.","lead":"This paper maps how artificial intelligence is already being used offensively, across academic papers, industry security briefings, a survey of 549 people, and 12 experts. It offers a shared checklist and a research agenda so future work on malicious AI can be compared and tracked.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Survey representativeness is the load-bearing weak point: the 'laypeople' pillar (N=549) is 75% IT-affiliated and 80% tertiary-educated, so the >80% concern finding may not generalize; post-stratification would settle it.","rationale":"I read the paper as a genuine attempt at a multi-source SoK. The checklist, the 133-work archive, and the transparent dual-review methodology are independent contributions that do not hinge on the survey. However, the abstract's 'holistic picture' claim explicitly rests on including 'laypeople' as a knowledge source. The survey is the only operationalization of that source. Its composition (75% IT, 80% tertiary, 91% Europe/North America) is a textbook convenience sample, and the paper's own OECD comparison shows large deviations. The null IT-correlation finding is informative but underpowered for the key claim because the non-IT group is small and itself skewed. I do not see this as a reason to reject: the paper discloses the limitation and positions the survey as exploratory. But it is the most load-bearing weakness because it directly affects the novelty claim of a 'holistic' cross-source view. The expert-priming concern (§VI-C) is real but explicitly designed into the method and transparently reported; the uniqueness claim is supported by the Shujun Li SoK bibliography check and the paper's differentiation from Mirsky et al. [20]. Hence the reader's CONDITIONAL verdict is appropriate; my test would determine whether the condition should be strengthened (e.g., requiring a representative follow-up survey) or can be relaxed.","tokens_in":52704,"tokens_out":8708,"duration_ms":83790,"concrete_test":"Post-stratify the N=549 survey against OECD population margins (gender, age, education) using the demographic data already reported in Table V/Appendix B. Compute weight_i = target_prop(cell)/sample_prop(cell) and re-estimate the weighted proportion of respondents concerned about OAI and the proportion expecting personal harm. If the weighted concern proportion drops by more than 5 percentage points or falls below 80%, the headline laypeople result is not robust to sampling bias, and the 'holistic' claim must be qualified. Also report the concern rate for the non-IT subgroup (n≈137) separately; a significant difference from the IT subgroup would confirm that the overall figure is driven by the IT-heavy sample.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central 'holistic picture' claim depends on four knowledge sources; the laypeople survey (§V) is the only source representing the general public. The convenience sample (N=549) comprises 70% European and 21% North American respondents, 80% with at least a Bachelor's degree, 75% employed in IT, and 77% self-rating at least intermediate cybersecurity knowledge (Appendix B, Table V). These attributes make the sample a tech-literate, highly educated, Western-centric group, not the 'laypeople' the abstract invokes. The headline that 'over 80% of respondents are concerned' (442/525 of those who had considered OAI; §V-B1) and the lessons in §V-B2/§VII-A are therefore descriptive of this skewed group. The paper's disclaimer in §V-A/§VII-B that it cannot claim representativeness and its null correlation between IT employment and concern (Appendix B) mitigate but do not resolve the issue: the null correlation is computed within a sample that is 75% IT, and the non-IT subgroup (n≈137) is neither sized nor selected to represent the public. Table V quantifies the skew: 88% tertiary education vs 40% in the OECD, 36% female vs 50.8%, and 2% over 65 vs ~23%. Because the abstract explicitly offers the survey as a distinct 'laypeople' perspective, a systematically unrepresentative sample makes the 'holistic' account overstate the public-perception pillar.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a systematization of offensive AI (OAI) that combines four knowledge sources: 95 academic papers, 38 InfoSec briefings from BlackHat/DefCon, a 549-person survey, and statements from 12 experts. The authors introduce an OAI Assessment Checklist, an online tool for applying it, and derive ten open problems from the expert statements. The central claim is that this is the first work to draw a holistic picture of OAI by aligning academic, industrial, lay, and expert perspectives under a common set of criteria.","tokens_in":52965,"tokens_out":5266,"duration_ms":51337,"significance":"If the validity concerns below are resolved, this would be a valuable contribution: the methodology is transparent (published search queries, dual reviewing with adjudication, grounded theory coding), the curated archive of 133 works is reusable, the online checklist tool lowers the barrier for future systematization, and the multi-source design is genuinely novel among SoK papers. The checklist and its operationalization are falsifiable and can be applied by other researchers. The paper also explicitly documents limitations and provides code for the NLP analysis. These strengths are real and should be credited. However, the two pillars that most distinguish this SoK from prior work—the laypeople survey and the expert opinions—currently have significant validity concerns that affect the central 'holistic picture' claim.","major_comments":[{"comment":"The 12 experts whose opinions anchor Section VI and Contribution C3 are, according to the author list, co-authors of this paper (the 12 names after the four corresponding authors). Nowhere in §II-D or elsewhere is this disclosed; the text says 'we reached out to 12 experts' and describes them as external sources. This compromises the independence of the expert pillar, and the pre/post comparison in §VI-C is especially affected because the 'post' statements were written by people who had already seen and contributed to the draft and its framework. The authors must either explicitly disclose that the experts are co-authors and re-characterize the findings as an internal author elicitation, or recruit independent external experts. Without this, the claim of consolidating 'expert opinions' as a separate knowledge source is not substantiated.","section":"§II-D and author list (page 1)"},{"comment":"The survey sample is described in the abstract and Section V as representing 'laypeople' with 'diverse backgrounds and expertise,' but the demographic data show 70% European, 21% North American, 80% with at least a Bachelor's degree, 75% in IT-related work, and 77% self-rating at least intermediate cybersecurity knowledge. The paper acknowledges in §VII-B that it 'cannot claim representativeness,' yet the headline '84% of respondents are concerned' and the lessons learned in §V-B2 and §VII-A are presented without this caveat. The non-IT subgroup (n≈137) is too small and not randomly selected to support generalizable claims about the general public. The abstract and the survey sections should be reframed to describe this as a convenience sample of a tech-literate, educated, Western-centric population, and ideally supplemented with post-stratified estimates or a sensitivity analysis.","section":"§V-A and Appendix B, Table V"},{"comment":"The comparison of expert opinions before and after reading the draft is interpreted as evidence that the paper raised awareness of topics such as privacy and cost. Because the experts are co-authors who had already collaborated on the systematization, this comparison does not measure the effect of the paper on independent experts; it measures a change in the authors' own framing after being exposed to the paper's checklist and findings. This is circular with respect to Contribution C3, which presents the ten open problems as if they were distilled from external expert input. The authors should remove the causal interpretation or reframe this subsection as a design artifact of the elicitation process.","section":"§VI-C"}],"minor_comments":[{"comment":"The 'social perspective' metric counts occurrences of the strings 'society,' 'social,' 'societal,' and 'socio' and describes this as an objective measure of a paper's social focus. A raw word count is a crude proxy (for instance, 'social' appears in routine technical phrases such as 'social engineering'), and the paper would benefit from acknowledging this limitation or supplementing the count with a manual check.","section":"§II-E.2"},{"comment":"The screening funnel shows 3311 papers, then '128 papers' after preliminary screening, then 95 final papers, but the figure and text do not clearly explain how many papers were excluded during each of the two screening rounds. Please clarify the intermediate counts in the figure or in the captions.","section":"§II-A and Fig. 2"},{"comment":"The custom icon set (♂shield-alt, ⋆, †, /user, ♂server, /coins, /calcula◎or, /commen◎, etc.) is dense and may be difficult to parse; adding a plain-text legend with short definitions in each caption or in a single table of symbols would improve readability.","section":"Tables I–III"},{"comment":"Several references are informal or volatile (e.g., [47] is a LinkedIn post, [48] is a bare URL, [212] is an archived news page). For a SoK aimed at long-term utility, please replace these with archival or stable citations where possible, or clearly mark them as online resources.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The undisclosed co-authorship of the 12 'expert' participants is the most serious issue in this manuscript. If the authors cannot obtain genuinely independent expert statements, Contribution C3 should be substantially weakened, and the paper should be reframed as an author-generated research agenda rather than a synthesis of external expert opinion. The survey representativeness issue, while acknowledged, also needs to be addressed in the abstract and conclusions, not only in the limitations section. I would not reject the paper: the corpus analysis, checklist tool, and comparative lessons are useful and could survive these fixes."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague —\n\nThe headline: this SoK delivers more than the usual survey. The genuinely new pieces are the 38 InfoSec briefings systematized with the same checklist as the academic papers, and the 12 expert statements reported verbatim in the appendix. Those two sources reveal real blind spots in prior surveys — attacks on society show up far more in briefings than in academic literature, and experts visibly shift their priorities after reading the paper. The checklist is a practical tool for future classification, and the methodology (dual review, adjudication, grounded-theory coding) is transparent enough to reproduce.\n\nThe soft spot is the survey pillar. The paper calls the 549 respondents 'laypeople,' but the sample is 75% IT-affiliated, 80% tertiary-educated, mostly European and North American, only 2% over 65, and 63% male. The 'over 80% are concerned' result and the lessons in §VII-A are descriptive of a tech-literate convenience sample, not the general public. The paper does acknowledge this in §V-A and §VII-B, and the correlation analysis showing IT employment does not predict concern is a fair attempt at mitigation — but that null is computed inside a sample that is 75% IT. The survey is not fatal: the core systematization of 133 works stands on its own, and the survey is one of four knowledge sources. But the 'holistic picture' claim is weakened if the human-perception pillar is this skewed. Post-stratification or a relabeling ('tech-literate public' rather than 'laypeople') would largely fix it.\n\nThe expert elicitation has a mild circularity — open problems were written after reading the draft — but the paper is upfront about it and explicitly checks for influence (privacy, cost). That is acceptable for a research agenda. Search limitations are admitted. These are minor issues.\n\nBottom line: this deserves serious refereeing and a cite. The survey needs caveats, not a rebuild. I would bring it to a reading group to discuss the multi-source method — and the sample problem is a good cautionary tale for anyone planning a similar design.","headline":"A genuinely broader SoK on offensive AI that mostly earns its 'holistic' claim, except the 'laypeople' survey turns out to be a convenience sample of tech-literate Westerners.","tokens_in":53608,"tokens_out":4098,"would_cite":true,"duration_ms":36559,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Offensive AI spans systems, humans, and society; this paper gives the first cross-source systematization—95 papers, 38 briefings, a 549-person survey, and 12 expert statements.","keywords":["offensive AI","AI-enabled cyberattacks","systematization of knowledge","security and privacy threats","user survey","expert elicitation","cost-benefit analysis","human-centric security"],"falsifier":"Run the same four-question survey on a probability sample of the general population; if the share concerned about offensive AI falls well below the paper's 84% or the open-ended answers no longer centre on misinformation and deepfakes, the survey pillar's general claims about laypeople would be contradicted.","tokens_in":52468,"feed_emoji":"⚔️","tokens_out":8750,"duration_ms":73797,"temperature":0.7,"pith_summary":"This paper tries to establish that no existing work has drawn a holistic picture of the offensive use of artificial intelligence, and that such a picture can be built by pooling four sources of knowledge: academic papers, industrial security briefings, the opinions of laypeople, and the views of experts. The authors analyse 95 research papers and 38 briefings with a common checklist, survey 549 non-experts, and collect statements from 12 experts, which they condense into ten open problems. The central claim is that offensive AI targets systems, humans, and society in ways that a single attack-tactic taxonomy cannot capture, so a reusable assessment checklist is needed. If the paper is right, future work on offensive AI gains a shared vocabulary and a way to compare attacks, costs, and countermeasures across otherwise incommensurable sources.","feed_headline":"First holistic map of offensive AI draws on 133 works and 549 voices","feed_subtitle":"A reusable checklist classifies AI attacks on systems, humans, and society across all knowledge sources.","key_machinery":"The OAI Assessment Checklist, a reusable set of criteria built around three questions: What is the use case? What is the target? What is the cost/benefit? It maps each work to a standardized attack taxonomy as its first step, adds an original target dimension distinguishing humans, systems, and society (and real versus toy systems), and records whether benefits, costs, and non-AI baselines are quantified. The checklist is the alignment device that lets the authors compare a peer-reviewed paper, an industrial briefing, a survey answer, and an expert statement on the same footing.","core_discovery":"On its own terms, the paper's discovery is that the offensive potential of AI is heterogeneous across three target classes—systems, humans, and society—and that each knowledge source reveals a different slice of that heterogeneity. Academic technical papers mostly propose novel attacks against toy systems and neglect attacker costs, while industrial security briefings demonstrate attacks against real systems and humans but rarely discuss cost. Non-technical academic work emphasises warfare and society, laypeople are broadly concerned yet hold some concerns orthogonal to actual offensive use, and experts, after reading the draft, shifted their stated priorities toward privacy and cost. From these observations the paper derives a three-question OAI Assessment Checklist—use case, target, cost/benefit—and uses it to show that existing mappings miss privacy attacks, attacks on society, and autonomous agents. The paper further claims to be the first systematization of offensive AI to combine all four knowledge sources and to report expert opinions verbatim.","pith_inferences":["If the checklist is adopted as a community standard, the same three questions could be asked of future AI security incidents, turning the curated archive into an early-warning signal for new offensive uses.","The survey's demographic skew means the 84% concern figure best describes a highly educated, IT-adjacent population; a representative sample could plausibly yield different levels and themes of concern.","The observed shift in expert priorities toward privacy and cost after reading the draft suggests that systematic reviews can change expert agendas; a controlled pre/post study with a placebo document could test this directly.","The human-targeting attacks the paper finds overlooked—attribute inference and profile matching—may become more central as generative models make personal data easier to exploit; monitoring that gap is a concrete follow-up."],"forward_implications":["Any future work on offensive AI can be classified with the checklist, making the snapshot extendable rather than frozen.","Researchers reporting novel AI attacks should expect to evaluate countermeasures; only about half of the surveyed attack papers do, and few offensive-security tools warn about malicious abuse.","Attacker cost/benefit and non-AI baselines should become standard reporting items, because the surveyed literature mostly omits them and real-world risk is therefore hard to judge.","Attacks targeting humans are under-represented in academic literature yet dominate laypeople's concerns, marking a concrete research gap.","Taxonomies built only on attack tactics miss important offensive AI, so society-level and privacy use cases need separate tracking."],"supporting_citations":[{"why":"Prior systematization of the offensive-AI threat to organizations; the main baseline this paper extends to humans and society.","marker":"[20]"},{"why":"Methodological guidelines that structure the systematic literature review and screening process.","marker":"[31]"},{"why":"Grounded-theory method used to code open-ended responses from laypeople and experts.","marker":"[57]"},{"why":"Source of the cost/benefit criteria and the requirement to compare with non-AI baselines.","marker":"[62]"},{"why":"The attack-tactic taxonomy used as the first mapping step in the checklist.","marker":"[63]"},{"why":"Prior socio-technical review of offensive AI, compared to demonstrate this paper's wider knowledge-source scope.","marker":"[219]"},{"why":"Bibliography of existing systematization papers used to support the novelty of the expert-opinion pillar.","marker":"[220]"},{"why":"Reference on convenience sampling that frames the acknowledged limits of the layperson survey.","marker":"[55]"}],"fun_headline_variants":["First SoK map of offensive AI: 4 sources, 3 targets","AI offensive power: systems, humans, society all at risk","New SoK: offensive AI threats often overlooked in prior maps","Offensive AI: four knowledge sources reveal different attack targets","Checklist from 133 works and 549 voices maps offensive AI"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the 549 convenience-sampled respondents, mostly from Europe and North America, largely degree-holding and IT-employed, stand in for 'laypeople' closely enough that the survey's concern levels and themes can support the paper's general lessons about public perception.","fun_headline_variants_meta":{"raw":{"variants":["First SoK map of offensive AI: 4 sources, 3 targets","AI offensive power: systems, humans, society all at risk","New SoK: offensive AI threats often overlooked in prior maps","Offensive AI: four knowledge sources reveal different attack targets","Checklist from 133 works and 549 voices maps offensive AI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000606,"raw_usage":{"total_tokens":2840,"prompt_tokens":974,"completion_tokens":1866,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":590,"completion_tokens_details":{"reasoning_tokens":1791}},"tokens_in":590,"tokens_out":1866,"duration_ms":11326,"temperature":1.0,"reasoning_tokens":1791,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:42:00.063133+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same four-question survey on a probability sample of the general population; if the share concerned about offensive AI falls well below the paper's 84% or the open-ended answers no longer centre on misinformation and deepfakes, the survey pillar's general claims about laypeople would be contradicted.","supporting_citations":[{"cited_title":"Artificial intelligence (AI) cybersecurity dimensions: a comprehensive framework for understanding adversarial and offensive AI,","cited_arxiv_id":null,"evidence_quote":"Prior socio-technical review of offensive AI, compared to demonstrate this paper's wider knowledge-source scope."},{"cited_title":"Shujun Li’s Bibliography of SoK Papers,","cited_arxiv_id":null,"evidence_quote":"Bibliography of existing systematization papers used to support the novelty of the expert-opinion pillar."}],"review_version":1}