{"id":"787c4793-3c03-4051-a48c-fdb356e10cd3","arxiv_id":"2507.01903","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A survey that organizes AI-for-research work into five tasks, comprehension, survey, discovery, writing, and peer review, and compiles associated tools and benchmarks.","lead":"This paper surveys how AI, especially large language models, is used across the scientific research process, from reading papers and finding related work to running experiments, writing papers, and reviewing them. It proposes a five-part taxonomy and a large resource list so researchers can quickly find relevant tools, benchmarks, and datasets.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The formal composition in Eq. 1 does not type-check: each module's input/output signatures (Eqs. 4–13) prevent composition, so the claimed 'unified perspective' is not actually a function composition.","rationale":"The most load-bearing concern I can verify directly from the manuscript is the formal composition in Eqs. 1–2. The paper lists this as a contribution and relies on it to frame the five taxonomy modules as a unified pipeline. If the type signatures do not match, then the 'unified perspective' is not a mathematically grounded framework, and Eq. 2 is not a valid consequence. This is an internal inconsistency, not an external disagreement, so it is more decisive than the unverifiable comprehensiveness concern. The reader's concern about missing search and inclusion criteria is legitimate and affects the 'comprehensive' claim, but it would require a large external audit to settle. The formalization issue can be resolved by a short analytic check and should be addressed before the paper can be considered to provide a formal unified framework. The survey's resource value is not destroyed by this flaw, so the existing CONDITIONAL verdict remains appropriate.","tokens_in":47510,"tokens_out":5950,"duration_ms":74843,"concrete_test":"Independently write out the input and output types of ASC, AAS, ASD, AAW, and APR from Eqs. 4, 6, 8, 10, and 12. Then check whether each stage's output type matches the next stage's input type in Eq. 1. If any mismatch is found, attempt to derive Eq. 2 by substituting q into ASC; the derivation will fail. This analytic check requires no external data and will settle whether the formal composition is valid as stated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's formal contribution rests on Eq. 1: A = APR ∘ AAW ∘ ASD ∘ AAS ∘ ASC. For this to be a valid composition, the codomain of each stage must equal the domain of the next stage. The definitions in §2.1 do not satisfy this. ASC maps a document set to knowledge (Eq. 4), AAS maps survey requirements to a survey (Eq. 6), ASD maps ideas/innovations to innovations (Eq. 8), AAW maps a manuscript to a manuscript (Eq. 10), and APR maps a manuscript/review to a review (Eq. 12). Consequently, AAS's domain is R_AS, not the knowledge output of ASC; AAW's domain is M, not ASD's innovation output; and the query q in Eq. 2 is not obviously in the domain of ASC, which expects documents. Eq. 2 therefore does not follow from Eq. 1. The equations describe a schematic pipeline, not a composable function, so the formal 'unified perspective' is not currently well-defined.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents a survey of AI for research (AI4Research), organizing the field into five task areas: AI for Scientific Comprehension, AI for Academic Survey, AI for Scientific Discovery, AI for Academic Writing, and AI for Academic Peer Reviewing. It introduces a formal composition of these task modules in Eqs. (1)-(2), defines per-module objectives in Eqs. (4)-(13), contrasts AI4Research with AI4Science, surveys methods and applications across natural, applied, and social sciences, compiles tools and datasets in Section 9, and proposes future directions in Section 10. The paper also reproduces several comparison tables from prior benchmark papers.","tokens_in":47723,"tokens_out":4807,"duration_ms":53277,"significance":"If the taxonomy and resource map are accurate, this survey is useful as an organizing frame and entry point for a rapidly growing literature. Its strengths include the breadth of the reference list, a generally sensible decomposition of the research lifecycle into five tasks, transparent attribution of benchmark tables to the original papers, and a substantial collection of tools, datasets, and applications. The formal composition in Eqs. (1)-(2), however, is not mathematically well-defined, and the optimization objectives in Eqs. (3)-(13) are presented without definitions or a probabilistic model; these formal elements should not be presented as a unified mathematical foundation without substantial repair or reframing.","major_comments":[{"comment":"The composition A = APR ∘ AAW ∘ ASD ∘ AAS ∘ ASC is not type-correct under the definitions given in Eqs. (4)-(13). ASC in Eq. (4) is described as taking documents DSC and producing knowledge K; AAS in Eq. (6) takes survey requirements RAS and produces a survey S; ASD in Eq. (8) produces innovations I; AAW in Eq. (10) produces a manuscript M; and APR in Eq. (12) produces a review R. The codomain of each stage is not the domain of the next stage, and the query q in Eq. (2) is not obviously in the domain of ASC, which expects documents. The equations therefore describe a schematic pipeline, not a function composition. Please either provide explicit domain/codomain types for each module and show that they chain, or remove the claim that Eqs. (1)-(2) provide a formal unified perspective.","section":"§2, Eqs. (1)-(2) and (4)-(13)"},{"comment":"The stated objectives are not well-defined. In Eq. (3), η(·), α(·), and τ(·) are not defined. In Eqs. (5)-(13), the quantities Coherence, Coverage, Relevance, Clarity, Novelty, Validity, Significance, Consistency, Readability, Compliance, Correctness, Helpfulness, and the expectations over K ∼ ASC, S ∼ AAS, I ∼ ASD, M ∼ AAW, and R ∼ APR have no specified definitions or probability models. As written, these expressions cannot be evaluated or optimized, so they do not provide a formal foundation for the proposed unified perspective. Either define these terms operationally or present them as informal schematics rather than as formal objectives.","section":"§2, Eqs. (3), (5), (7), (9), (11), (13)"},{"comment":"The paper claims to fill the absence of a comprehensive survey on AI4Research, but it does not report any literature search methodology. There is no statement of databases queried, time span covered, inclusion or exclusion criteria, keyword strategy, or screening process. Without such information, the comprehensiveness claim is not verifiable and the selection of cited works may be biased. Please add a methodology paragraph or, if the survey is intentionally selective, temper the comprehensiveness claim accordingly.","section":"Abstract and §1 (Introduction)"}],"minor_comments":[{"comment":"The text says 'we identify six core capabilities' but then lists only five: Scientific Comprehension, Academic Survey, Scientific Discovery, Academic Writing, and Academic Peer Reviewing. Change 'six' to 'five' or add the missing item.","section":"§2, first paragraph"},{"comment":"The symbol K appears both as the input and the output of ASC in Eq. (4), and the actual input documents DSC are not represented in the left-hand side. Clarify the input/output notation; the same ambiguity affects Eqs. (6), (8), (10), and (12).","section":"§2.1.1, Eq. (4)"},{"comment":"There is a typo: 'Furhter' should be 'Further'.","section":"§2, after Eq. (2)"},{"comment":"'give a manually created question' should be 'given a manually created question'.","section":"§3.1.1"},{"comment":"In the Semi-Automatic Academic Writing list, TikZero [55] appears twice; remove the duplicate entry.","section":"Figure 2"},{"comment":"The axis label 'Level of Automation ext' appears garbled, and the percentage '70%' is unexplained. Please fix the figure text.","section":"Figure 3"},{"comment":"The statement that papers generated through Zochi 'have even been accepted by ACL 2025' is a strong, specific claim without a citation or venue details. Please add evidence or soften the claim.","section":"§5.5"}],"recommendation":"major_revision","confidential_remarks":"The survey is broad and likely useful to the community, but the formal apparatus in Section 2 is currently decorative rather than mathematically meaningful; this should be either corrected or explicitly downgraded to a schematic. The lack of a search/inclusion methodology is a concern for a paper whose central claim is comprehensiveness. The Zochi 'accepted by ACL 2025' claim should be verified by an editor. The manuscript fits the journal's scope but would benefit from a more disciplined presentation of its formal contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing you should know: the survey's value is organizational, not formal. The five-part taxonomy and the resource list are a genuinely useful map of a crowded area. The equations are decorative, and the stress-test is right: Eq. 1 does not type-check. ASC outputs knowledge, AAS expects a survey draft, ASD expects ideas, AAW expects a manuscript, APR expects a review. None of these chain as function composition, so the 'unified perspective' is a figure of speech, not a formal result. That's fixable if the authors treat the equations as illustrative rather than load-bearing.\n\nWhat is new: little, but that's acceptable for a survey. Prior surveys cover hypothesis generation, AI4Science, and automated review; this one compiles them under a lifecycle framing (comprehension, survey, discovery, writing, reviewing) and adds a broad applications and resources section. The tables in Sections 9-10 are the best part; I'd point a new student there. The paper is transparent that Tables 2-5 are sourced from earlier papers, which is honest.\n\nWhere it's soft: (1) The formalization, as above. The definitions in Eqs. 4-13 neither match the prose nor compose. If the authors want a 'formal definition,' they need to actually type the modules. (2) No search or inclusion criteria. A survey covering roughly nine hundred papers should say how the corpus was assembled; otherwise the taxonomy's coverage is unverifiable. (3) There's a concrete unsupported claim: 'Papers generated through Zochi have even been accepted by ACL 2025' appears with no citation. That needs a source or removal. (4) Section 2 says 'six core capabilities' then lists five. Minor, but sloppy.\n\nProportion: the taxonomy itself holds up as a plausible high-level map. The descriptions I spot-checked (STORM, AutoSurvey, AI Scientist, AlphaFold) are accurate enough. I did not verify all nine hundred, and I wouldn't expect a referee to either. The missing selection methodology is the real issue for a survey; the equations are more embarrassing than substantive.\n\nWho benefits: practitioners and newcomers wanting a foothold, and anyone writing related-work sections. It deserves a serious referee because the compilation is useful and the flaws are correctable. My recommendation: send to peer review, but condition acceptance on fixing the formalization, adding the selection methodology, and supporting or cutting the ACL claim.","headline":"Useful organizational survey; the formal composition is decorative and the taxonomy is the real contribution.","tokens_in":48263,"tokens_out":4328,"would_cite":true,"duration_ms":45507,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A new survey maps AI for research onto five stage-level tasks and composes them into one pipeline.","keywords":["AI4Research","large language models","scientific discovery","academic survey","academic writing","peer review","scientific comprehension","research taxonomy"],"falsifier":"Run an independent, criteria-driven sweep of recent AI-for-research systems (for example, fixed search strings across a defined set of venues and years) and ask two questions: does every system fall into exactly one of the five taxonomy branches, and do the survey's comparison tables (Tables 2-5, sourced from other papers) reproduce under re-annotation? The central claim weakens if a substantial share of systems straddle branches or fit none, or if the reproduced numbers do not match the cited sources.","tokens_in":47332,"feed_emoji":"🔬","tokens_out":7631,"duration_ms":65586,"temperature":0.7,"pith_summary":"This survey tries to establish that AI for Research (AI4Research) is a field with a single spine: five task families — scientific comprehension, academic survey, scientific discovery, academic writing, and peer reviewing — that can be composed into one pipeline running from a research query to a reviewed publication. The authors argue this unified frame is what has been missing: earlier surveys focused on discovery and writing, so researchers lacked a map of the full lifecycle and the resources at each stage. If the frame holds, a newcomer can locate any AI research tool or benchmark in one of five slots, and progress at any stage can be measured against the pipeline as a whole. The paper couples the taxonomy with a formal composition (Eq. 1-2) and a per-stage resource compilation spanning the natural, applied, and social sciences.","feed_headline":"Five AI roles now map the whole research pipeline","feed_subtitle":"One map covers every stage of AI-assisted research, from reading papers to generating reviews.","key_machinery":"The load-bearing object is the five-part taxonomy itself, made formal by the composition equation $A = A_{PR} \\circ A_{AW} \\circ A_{SD} \\circ A_{AS} \\circ A_{SC}$ (Eq. 1), where $\\circ$ is function composition and each $A_i$ is the AI model tailored to one research task: comprehension ($A_{SC}$), survey ($A_{AS}$), discovery ($A_{SD}$), writing ($A_{AW}$), and peer review ($A_{PR}$). Applied to a research query $q$ (Eq. 2), the composed system produces a publication $A(q)$, and Eq. 3 states the objective as maximizing efficiency, performance, and innovation. The taxonomy does the work of the paper: it is the organizing frame that lets every surveyed system be placed in the lifecycle, gives each stage a definition in terms of input-output functions (Eqs. 4-13), and turns 'AI4Research' from a slogan into a pipeline with replaceable stages.","core_discovery":"The paper's central claim is that the field of AI for research can be organized — and should be understood — as five task families that together form a single pipeline: AI for Scientific Comprehension (extracting knowledge from a paper's text, tables, and charts), AI for Academic Survey (retrieving and synthesizing many papers into overviews and related-work sections), AI for Scientific Discovery (idea mining, novelty assessment, theory analysis, and experiment conduction), AI for Academic Writing (assisting or fully automating manuscript production), and AI for Academic Peer Reviewing (pre-review, in-review, and post-review stages). It formalizes this as the functional composition $A = A_{PR} \\circ A_{AW} \\circ A_{SD} \\circ A_{AS} \\circ A_{SC}$, so that a research query $q$ flows through comprehension, survey, discovery, writing, and review to yield a reviewed publication, and it states the system goal as maximizing efficiency, performance, and innovation (Eq. 3). On this basis the paper positions itself as the missing survey that spans the whole lifecycle: earlier surveys, it argues, covered only discovery and writing under the banner of AI4Science, whereas AI4Research includes scientific comprehension, academic survey, and peer review as first-class stages. It then identifies priority gaps — the rigor and scalability of automated experiments and societal impact — and compiles per-stage tools, datasets, and benchmarks across natural, applied, and social sciences.","pith_inferences":["A natural next step the paper does not take is to use its composition equation as a benchmark design, fixing every stage except the one under test, so that gains in, say, idea mining can be isolated from losses in writing or review.","The five-stage pipeline reads as an engineering blueprint: if stages are connected by standardized interfaces (a query, a survey, an idea, a manuscript, a review), teams could upgrade modules independently instead of rebuilding whole 'AI scientist' systems.","Because the survey sets no systematic inclusion criteria, a reader should treat its coverage as a curated map rather than a census; the same taxonomy could later be tested by a meta-analysis with explicit search strings and inter-annotator agreement.","The AI4Science versus AI4Research split implies a division of labour: domain-specific discovery systems plug into a broader research-workflow shell, which suggests integration (API-style composition) rather than competition between the two lines of work."],"forward_implications":["A researcher entering AI4Research can locate any existing system — from PaperQA2 to the AI Scientist to DeepReview — in one of five task families and see which stage of the lifecycle it automates.","Stage-level progress becomes measurable: because the framework composes the five stages, improving one module (for example, retrieval-backed surveys) can be evaluated by its effect on downstream discovery and writing quality.","The paper's identified gaps — rigor and scalability of automated experiments, plus societal impact — become concrete research targets rather than vague cautions.","The resource lists (benchmarks, datasets, tools per stage) lower the cost of entry for building or evaluating AI4Research systems.","The equations give a formal target: an AI4Research system is good insofar as it maximizes the efficiency, performance, and innovation of the publication it produces."],"supporting_citations":[{"why":"Defines the three-stage fully automatic research pipeline (idea mining, experiments, writing) that the five-part taxonomy generalizes and extends.","marker":"[509]"},{"why":"Zochi, an end-to-end discovery system whose full-lifecycle workflow recurs throughout the paper as the archetype of AI4Research.","marker":"[12]"},{"why":"Carl, an autonomous research workflow system the survey repeatedly classifies within scientific discovery and full-automatic writing.","marker":"[332]"},{"why":"AgentLaboratory, the role-based multi-agent lab the survey cites for collaborative discovery, writing, and evaluation workflows.","marker":"[669]"},{"why":"AgentArxiv, the multi-agent research system that adds survey and peer-review stages, supporting the taxonomy's lifecycle span.","marker":"[668]"},{"why":"STORM, the multi-agent survey generator that anchors the academic-survey branch and document-level survey generation.","marker":"[676]"},{"why":"ScienceAgentBench, source of the table comparing full-automatic discovery capability across models and prompting strategies.","marker":"[127]"},{"why":"SurveyForge, source of the document-level survey generation comparison table and its assessment metrics.","marker":"[855]"},{"why":"DeepSeek-R1, cited as a motivating demonstration of reasoning abilities that opened the door to AI-driven research.","marker":"[265]"}],"fun_headline_variants":["Five AI tasks form one pipeline for all of research","AI survey maps full research pipeline: read to review","One taxonomy unifies AI roles across research lifecycle","From comprehension to peer review: AI's pipeline defined","AI4Research: five stages cover the entire scientific process"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole frame rests on the assumption that the five-way taxonomy and its descriptions of individual systems are faithful to the underlying literature; because the paper sets no systematic search or inclusion criteria, a biased or inaccurate selection would leave the unified perspective unsupported even if every individual summary were right.","fun_headline_variants_meta":{"raw":{"variants":["Five AI tasks form one pipeline for all of research","AI survey maps full research pipeline: read to review","One taxonomy unifies AI roles across research lifecycle","From comprehension to peer review: AI's pipeline defined","AI4Research: five stages cover the entire scientific process"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000746,"raw_usage":{"total_tokens":3389,"prompt_tokens":1074,"completion_tokens":2315,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":690,"completion_tokens_details":{"reasoning_tokens":2239}},"tokens_in":690,"tokens_out":2315,"duration_ms":16028,"temperature":1.0,"reasoning_tokens":2239,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T20:40:22.440805+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run an independent, criteria-driven sweep of recent AI-for-research systems (for example, fixed search strings across a defined set of venues and years) and ask two questions: does every system fall into exactly one of the five taxonomy branches, and do the survey's comparison tables (Tables 2-5, sourced from other papers) reproduce under re-annotation? The central claim weakens if a substantial share of systems straddle branches or fit none, or if the reproduced numbers do not match the cited sources.","supporting_citations":[],"review_version":1}