Recognition: 3 theorem links
· Lean TheoremTowards an AI co-scientist
Pith reviewed 2026-05-11 12:54 UTC · model grok-4.3
The pith
A multi-agent AI system on Gemini 2.0 generates biomedical hypotheses that receive experimental validation in leukemia, fibrosis, and bacterial evolution.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The AI co-scientist employs a multi-agent architecture with asynchronous execution and a tournament evolution process to produce self-improving hypotheses that lead to concrete experimental findings: drug candidates for acute myeloid leukemia showing tumor inhibition, new epigenetic targets for liver fibrosis validated in organoids, and a parallel in silico discovery of a bacterial gene transfer mechanism that aligns with separate unpublished results.
What carries the argument
The generate-debate-evolve cycle in a multi-agent system with asynchronous task execution and tournament-based hypothesis refinement, scaled by test-time compute.
If this is right
- Increasing test-time compute produces measurable gains in automated hypothesis quality evaluations.
- Drug repurposing proposals for acute myeloid leukemia include compounds that inhibit tumor growth in vitro at clinically applicable levels.
- Epigenetic targets for liver fibrosis demonstrate anti-fibrotic activity and support liver cell regeneration in human hepatic organoids.
- The system identifies a gene transfer mechanism in bacterial evolution that recapitulates results from separate unpublished experiments.
Where Pith is reading between the lines
- If the process reliably avoids simple recombination of known information, the same multi-agent structure could be applied to hypothesis generation in non-biomedical domains by changing only the input objectives.
- Treating AI proposals and their lab validations as separate reports creates a workflow that could reduce confirmation bias in future uses.
- The emphasis on scaling test-time compute suggests that future versions might trade additional inference steps for higher rates of usable ideas without retraining the base model.
Load-bearing premise
The generated hypotheses are genuinely novel rather than recombinations of training data, and the experimental validations come from independent studies not influenced by the AI outputs.
What would settle it
Independent lab tests on the proposed acute myeloid leukemia candidates that fail to show tumor inhibition at the stated concentrations would falsify the claim of promising validation findings.
read the original abstract
Scientific discovery relies on scientists generating novel hypotheses that undergo rigorous experimental validation. To augment this process, we introduce an AI co-scientist, a multi-agent system built on Gemini 2.0. The AI co-scientist is intended to help uncover new, original knowledge and to formulate demonstrably novel research hypotheses and proposals, building upon prior evidence and aligned to scientist-provided research objectives and guidance. The system's design incorporates a generate, debate, and evolve approach to hypothesis generation, inspired by the scientific method and accelerated by scaling test-time compute. Key contributions include: (1) a multi-agent architecture with an asynchronous task execution framework for flexible compute scaling; (2) a tournament evolution process for self-improving hypotheses generation. Automated evaluations show continued benefits of test-time compute, improving hypothesis quality. While general purpose, we focus development and validation in three biomedical areas: drug repurposing, novel target discovery, and explaining mechanisms of bacterial evolution and anti-microbial resistance. For drug repurposing, the system proposes candidates with promising validation findings, including candidates for acute myeloid leukemia that show tumor inhibition in vitro at clinically applicable concentrations. For novel target discovery, the AI co-scientist proposed new epigenetic targets for liver fibrosis, validated by anti-fibrotic activity and liver cell regeneration in human hepatic organoids. Finally, the AI co-scientist recapitulated unpublished experimental results via a parallel in silico discovery of a novel gene transfer mechanism in bacterial evolution. These results, detailed in separate, co-timed reports, demonstrate the potential to augment biomedical and scientific discovery and usher an era of AI empowered scientists.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces an AI co-scientist, a multi-agent system built on Gemini 2.0 that uses a generate-debate-evolve process (inspired by the scientific method and scaled via test-time compute) to generate hypotheses aligned with scientist-provided objectives. It focuses on three biomedical areas—drug repurposing, novel target discovery, and bacterial evolution mechanisms—reporting that the system proposed AML drug candidates showing in vitro tumor inhibition, epigenetic targets for liver fibrosis with anti-fibrotic effects in organoids, and a novel bacterial gene-transfer mechanism that recapitulates unpublished results, with all experimental validations deferred to separate co-timed reports.
Significance. If the reported validations hold and demonstrate genuine novelty independent of training-data recombination or post-hoc alignment, the work would represent a meaningful step toward AI-augmented hypothesis generation with practical biomedical impact. The asynchronous multi-agent architecture and tournament evolution for self-improving hypotheses are concrete engineering contributions that could be adopted more broadly; the emphasis on test-time compute scaling is a timely strength.
major comments (2)
- [Abstract] Abstract: The central claim that the AI co-scientist 'uncovers new, original knowledge' and produces 'demonstrably novel research hypotheses' rests entirely on three headline validation outcomes (AML repurposing, liver-fibrosis targets, bacterial gene transfer) whose supporting data, methods, error bars, exclusion criteria, and independence from the AI outputs are deferred to 'separate, co-timed reports' with no quantitative summaries or blinding statements supplied in this manuscript. This prevents evaluation of whether the generate-debate-evolve loop contributed original insights or merely retrieved/recombined existing knowledge.
- [Abstract] Abstract and system-description sections: No evidence is presented that the final hypotheses differ from content already latent in the Gemini 2.0 training distribution or from the scientist-provided research objectives; the manuscript supplies neither pre-2024 literature comparisons, model-cutoff checks, nor logs of the tournament evolution that would allow assessment of whether the process avoids circular recombination.
minor comments (1)
- The description of the asynchronous task-execution framework would benefit from a diagram or pseudocode showing how agents interact across the tournament rounds.
Simulated Author's Rebuttal
We thank the referee for their constructive comments on our manuscript describing the AI co-scientist. We address the major concerns regarding the abstract claims and evidence for novelty below. We are prepared to make revisions to clarify the scope and add supporting details where feasible.
read point-by-point responses
-
Referee: [Abstract] Abstract: The central claim that the AI co-scientist 'uncovers new, original knowledge' and produces 'demonstrably novel research hypotheses' rests entirely on three headline validation outcomes (AML repurposing, liver-fibrosis targets, bacterial gene transfer) whose supporting data, methods, error bars, exclusion criteria, and independence from the AI outputs are deferred to 'separate, co-timed reports' with no quantitative summaries or blinding statements supplied in this manuscript. This prevents evaluation of whether the generate-debate-evolve loop contributed original insights or merely retrieved/recombined existing knowledge.
Authors: The primary focus of this manuscript is the presentation of the AI co-scientist architecture, including the asynchronous multi-agent framework and tournament evolution mechanism. The three biomedical applications are included to demonstrate the system's practical utility in generating hypotheses that align with scientist objectives and lead to experimental outcomes. As noted in the manuscript, the detailed experimental methods, quantitative results, error bars, exclusion criteria, and blinding information are provided in the separate co-timed reports to allow for comprehensive presentation in those venues. This manuscript supplies high-level descriptions of the outcomes. We will revise the abstract to more precisely indicate that the validations are detailed in accompanying reports and to qualify the claims of novelty accordingly, while retaining the description of the system's design. We maintain that the multi-agent debate and evolution process facilitates the generation of hypotheses that extend beyond direct retrieval, as evidenced by the system's ability to propose candidates leading to the reported validations. revision: partial
-
Referee: [Abstract] Abstract and system-description sections: No evidence is presented that the final hypotheses differ from content already latent in the Gemini 2.0 training distribution or from the scientist-provided research objectives; the manuscript supplies neither pre-2024 literature comparisons, model-cutoff checks, nor logs of the tournament evolution that would allow assessment of whether the process avoids circular recombination.
Authors: We recognize the value of providing explicit evidence for the originality of the generated hypotheses. The manuscript emphasizes the generate-debate-evolve process, where hypotheses are iteratively refined through agent-based critique and tournament selection, which is intended to promote novelty beyond the initial objectives or training data recombination. For the bacterial evolution case, the AI co-scientist independently identified a gene transfer mechanism that matched unpublished experimental findings, providing a strong indicator of originality. To address this comment, we will expand the system-description sections to include illustrative logs or step-by-step examples from the tournament evolution process for at least one case, showing how hypotheses were debated, critiqued, and evolved. This addition will allow readers to evaluate the contribution of the process. Pre-2024 literature comparisons and model-cutoff checks are not included due to the proprietary nature of the model and space constraints, but the use of scientist-provided objectives combined with the evolution mechanism helps ensure alignment and extension rather than circularity. revision: yes
Circularity Check
No circularity: descriptive system architecture with external validations
full rationale
The paper describes a multi-agent generate-debate-evolve system on Gemini 2.0 without any equations, fitted parameters, or first-principles derivations. All reported biomedical outcomes are explicitly deferred to separate co-timed experimental reports rather than derived internally. No self-definitional loops, load-bearing self-citations, or renaming of known results appear in the architecture or claims; the system is presented as aligned to external scientist objectives and prior evidence.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Large language models possess sufficient world knowledge and reasoning to generate useful novel scientific hypotheses when organized in a multi-agent debate-and-evolve loop.
invented entities (1)
-
AI co-scientist
no independent evidence
Forward citations
Cited by 51 Pith papers
-
AutoResearchBench: Benchmarking AI Agents on Complex Scientific Literature Discovery
AutoResearchBench is a new benchmark showing top AI agents achieve under 10% success on complex scientific literature discovery tasks that demand deep comprehension and open-ended search.
-
FermiLink: A Unified Agent Framework for Multidomain Autonomous Scientific Simulations
FermiLink is a unified AI agent framework that automates multidomain scientific simulations via separated package knowledge bases and a four-layer progressive disclosure mechanism, reproducing 56% of target figures in...
-
Why Do Multi-Agent LLM Systems Fail?
The authors create the first large-scale dataset and taxonomy of failure modes in multi-agent LLM systems to explain their limited performance gains.
-
An Experimental Method to Study Opinion Diffusion in Human-AI Hybrid Societies
Hybrid human-AI networks in 5x5 grids reached lower final polarization than human-only networks after eight rounds of opinion revision on polarizing topics.
-
AI co-mathematician: Accelerating mathematicians with agentic AI
An interactive AI workbench for mathematicians achieves 48% on FrontierMath Tier 4 and helped solve open problems in early tests.
-
AI CFD Scientist: Toward Open-Ended Computational Fluid Dynamics Discovery with Physics-Aware AI Agents
AI CFD Scientist autonomously discovers a Spalart-Allmaras runtime correction reducing lower-wall Cf RMSE by 7.89% on the periodic hill at Reh=5600 while using a vision-language gate to detect 14 of 16 silent failures...
-
AI CFD Scientist: Toward Open-Ended Computational Fluid Dynamics Discovery with Physics-Aware AI Agents
AI CFD Scientist autonomously finds a Spalart-Allmaras turbulence correction that lowers wall-friction error by 7.89% versus DNS on the periodic hill case using vision-language physics verification.
-
End-to-end autonomous scientific discovery on a real optical platform
An LLM agent autonomously identifies and experimentally validates a previously unreported optical bilinear interaction on a physical platform.
-
Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond
Proposes a levels x laws taxonomy for world models in AI agents, defining L1-L3 capabilities across physical, digital, social, and scientific regimes while reviewing over 400 works to outline a roadmap for advanced ag...
-
RosettaSearch: Multi-Objective Inference-Time Search for Protein Sequence Design
RosettaSearch applies LLM-driven multi-objective search at inference time to improve backbone-conditioned protein sequences, recovering designs with 18-68% better structural fidelity and 2.5x higher success rates than...
-
VERITAS: Verifiable Epistemic Reasoning for Image-Derived Hypothesis Testing via Agentic Systems
VERITAS is a multi-agent system for verifiable hypothesis testing on multimodal clinical MRI datasets that achieves 81.4% verdict accuracy with frontier models and introduces an epistemic evidence labeling framework.
-
AutoSOTA: An End-to-End Automated Research System for State-of-the-Art AI Model Discovery
AutoSOTA uses eight specialized agents to replicate and optimize models from recent AI papers, producing 105 new SOTA results in about five hours per paper on average.
-
AlphaEvolve: A coding agent for scientific and algorithmic discovery
AlphaEvolve is an LLM-orchestrated evolutionary coding agent that discovered a 4x4 complex matrix multiplication algorithm using 48 scalar multiplications, the first improvement over Strassen's algorithm in 56 years, ...
-
Unlocking LLM Creativity in Science through Analogical Reasoning
Analogical reasoning increases LLM solution diversity by 90-173% and novelty rate to over 50%, delivering up to 13-fold gains on biomedical tasks including perturbation prediction and cell communication.
-
GRAFT-ATHENA: Self-Improving Agentic Teams for Autonomous Discovery and Evolutionary Numerical Algorithms
GRAFT-ATHENA projects combinatorial method choices into factored trees that embed as fingerprints in a metric space, enabling an agentic system to accumulate experience across domains and autonomously discover new num...
-
Intervention-Based Time Series Causal Discovery via Simulator-Generated Interventional Distributions
SVAR-FM uses simulator clamping to produce interventional distributions and flow matching to identify time series causal structures, with an error bound that predicts sign reversal of causal effects below a simulator ...
-
MLS-Bench: A Holistic and Rigorous Assessment of AI Systems on Building Better AI
MLS-Bench shows that current AI agents fall short of reliably inventing generalizable ML methods, with engineering tuning easier than genuine invention.
-
Common-agency Games for Multi-Objective Test-Time Alignment
CAGE uses common-agency games and an EPEC algorithm to compute equilibrium policies that balance multiple conflicting objectives for test-time LLM alignment.
-
FAME: Forecasting Academic Impact via Continuous-Time Manifold Evolution
FAME models scientific topic trajectories in continuous time to forecast paper impact more accurately than LLMs by aligning manuscripts with field momentum in a dynamic latent space.
-
AI co-mathematician: Accelerating mathematicians with agentic AI
An interactive AI workbench called the AI co-mathematician supports open-ended mathematical research and achieves a new high score of 48% on FrontierMath Tier 4.
-
AI CFD Scientist: Toward Open-Ended Computational Fluid Dynamics Discovery with Physics-Aware AI Agents
An integrated AI agent framework for CFD uses vision-based physics gates to autonomously discover a Spalart-Allmaras runtime correction that cuts lower-wall skin-friction error by 7.89% versus DNS on the periodic hill...
-
Intentmaking and Sensemaking: Human Interaction with AI-Guided Mathematical Discovery
Expert mathematicians using an AI coding agent for discovery engage in repeated cycles of intentmaking to define goals and sensemaking to interpret outputs.
-
Hypothesis generation and updating in large language models
LLMs exhibit Bayesian-like hypothesis updating with strong-sampling bias and an evaluation-generation gap but generalize poorly outside observed data.
-
SciResearcher: Scaling Deep Research Agents for Frontier Scientific Reasoning
SciResearcher automates creation of diverse scientific reasoning tasks from academic evidence to train an 8B model that sets new SOTA at 19.46% on HLE-Bio/Chem-Gold and gains 13-15% on SuperGPQA-Hard-Biology and TRQA-...
-
AgentEconomist: An End-to-end Agentic System Translating Economic Intuitions into Executable Computational Experiments
AgentEconomist is an end-to-end agentic system with idea development, experimental design, and execution stages that uses a large economics paper database to produce research ideas with better literature grounding, no...
-
TSAssistant: A Human-in-the-Loop Agentic Framework for Automated Target Safety Assessment
TSAssistant is a human-in-the-loop multi-agent system that generates citable, evidence-grounded sections for target safety assessment reports by coordinating specialized subagents with interactive user refinement.
-
Human Cognition in Machines: A Unified Perspective of World Models
The paper introduces a unified framework for world models that fully incorporates all cognitive functions from Cognitive Architecture Theory, highlights under-researched areas in motivation and meta-cognition, and pro...
-
PRL-Bench: A Comprehensive Benchmark Evaluating LLMs' Capabilities in Frontier Physics Research
PRL-Bench evaluates frontier LLMs on 100 real physics research tasks and finds the best models score below 50, exposing a gap to autonomous discovery.
-
TREX: Automating LLM Fine-tuning via Agent-Driven Tree-based Exploration
TREX automates the LLM training lifecycle via collaborative agents and tree-based exploration, delivering consistent performance gains across 10 real-world fine-tuning tasks in FT-Bench.
-
Pioneer Agent: Continual Improvement of Small Language Models in Production
Pioneer Agent automates the full lifecycle of adapting and continually improving small language models via diagnosis-driven data synthesis and regression-constrained retraining, delivering gains of 1.6-83.8 points on ...
-
AIRA_2: Overcoming Bottlenecks in AI Research Agents
AIRA₂ improves AI research agents via asynchronous multi-GPU workers, hidden consistent evaluation, and interactive ReAct agents, reaching 81.5-83.1% percentile rank on MLE-bench-30 and exceeding human SOTA on 6 of 20...
-
Bridging the Experimental Last Mile: Digitizing Laboratory Know-How for Safe AI-Assisted Support
A video-plus-RAG human-in-the-loop system digitizes site-specific laboratory know-how and supplies safe, grounded guidance for experiments such as powder X-ray diffraction.
-
"When to Hand Off, When to Work Together": Expanding Human-Agent Co-Creative Collaboration through Concurrent Interaction
Concurrent human-agent interactions occur in 31.8% of turns and follow five action patterns explained by six triggers and four enabling factors, enabled by a context-aware design probe called CLEO.
-
CVEvolve: Autonomous Algorithm Discovery for Unstructured Scientific Data Processing
CVEvolve uses LLM agents with lineage-aware search to autonomously discover algorithms that outperform baselines on scientific image tasks including registration, peak detection, and segmentation.
-
GEAR: Genetic AutoResearch for Agentic Code Evolution
GEAR applies genetic algorithms to maintain and evolve multiple research states in autonomous code agents, outperforming single-path baselines by continuing to discover improvements over extended runs.
-
From Experimental Limits to Physical Insight: A Retrieval-Augmented Multi-Agent Framework for Interpreting Searches Beyond the Standard Model
HEP-CoPilot is a new multi-agent retrieval framework that retrieves, reconstructs, and compares experimental limits from HEP literature and HEPData to support interpretation of beyond-Standard-Model searches.
-
TSAssistant: A Human-in-the-Loop Agentic Framework for Automated Target Safety Assessment
TSAssistant is a modular, human-in-the-loop multi-agent system that generates citable, section-specific drafts for target safety assessment reports by coordinating specialized sub-agents with biomedical data sources a...
-
pAI/MSc: ML Theory Research with Humans on the Loop
pAI/MSc is a customizable multi-agent system that reduces human steering by orders of magnitude when turning a hypothesis into a literature-grounded, mathematically established, experimentally supported manuscript dra...
-
Towards Self-Improving Error Diagnosis in Multi-Agent Systems
ErrorProbe introduces a self-improving pipeline for attributing semantic failures in LLM multi-agent systems to specific agents and steps via anomaly detection, backward tracing, and tool-grounded validation with veri...
-
EvoMaster: A Foundational Evolving Agent Framework for Agentic Science at Scale
EvoMaster is a self-evolving agent framework that achieves state-of-the-art results on scientific benchmarks by enabling iterative hypothesis refinement and knowledge accumulation across domains.
-
Dive into Claude Code: The Design Space of Today's and Future AI Agent Systems
Claude Code centers on a model-tool while-loop surrounded by permission systems, context compaction, extensibility hooks, subagent delegation, and session storage; the same design questions yield different answers in ...
-
Inspectable AI for Science: A Research Object Approach to Generative AI Governance
Generative AI use in science can be governed through structured documentation and provenance capture by framing AI interactions as inspectable Research Objects rather than debating authorship.
-
Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models
The paper unifies perspectives on Long CoT in reasoning LLMs by introducing a taxonomy, detailing characteristics of deep reasoning and reflection, and discussing emergence phenomena and future directions.
-
Agentic AI Scientists Are Not Built For Autonomous Scientific Discovery
Agentic AI scientists face four structural challenges—McNamara fallacy in problem selection, missing tacit lab knowledge in LLMs, preference optimization reducing diversity, and single-turn benchmarks without physical...
-
ARIS: Autonomous Research via Adversarial Multi-Agent Collaboration
ARIS is a three-layer open-source system that uses cross-model adversarial collaboration plus claim-auditing pipelines to make LLM-driven research workflows more reliable.
-
LARA: Validation-Driven Agentic Supercomputer Workflows for Atomistic Modeling
LARA-HPC introduces a validation-first agentic system with dry-run verification and multi-phase refinement that improves robustness of AI-generated DFT workflows on HPC systems.
-
Agentic Risk-Aware Set-Based Engineering Design
Multi-agent LLM system applies set-based design and Conditional Value-at-Risk to explore and risk-filter airfoil designs with human manager coordination.
-
MIND: AI Co-Scientist for Material Research
MIND is an LLM-driven multi-agent system for automated hypothesis validation in materials science using scalable in-silico experiments with ML interatomic potentials.
-
Omakase: proactive assistance with actionable suggestions for evolving scientific research projects
Omakase monitors project documents to infer timely queries and distills research reports into actionable suggestions that users rated significantly more useful than raw reports.
-
From LLM Reasoning to Autonomous AI Agents: A Comprehensive Review
A survey consolidating benchmarks, agent frameworks, real-world applications, and protocols for LLM-based autonomous agents into a proposed taxonomy with recommendations for future research.
-
The Agentification of Scientific Research: A Physicist's Perspective
AI will evolve from a research tool into a collaborator, fundamentally reshaping scientific collaboration, discovery, publishing, and evaluation while requiring continuous learning and idea diversity for original cont...
Reference graph
Works this paper leans on
-
[1]
A foundation model for clinician-centered drug repurposing , author=. medRxiv , year=
-
[2]
Breaking Eroom’s law , author=. Nat Rev Drug Discov , volume=
-
[3]
Nature reviews Drug discovery , volume=
Drug repurposing: progress, challenges and recommendations , author=. Nature reviews Drug discovery , volume=. 2019 , publisher=
work page 2019
-
[4]
International Journal of Forecasting , volume=
Extension of the Elo rating system to margin of victory , author=. International Journal of Forecasting , volume=. 2020 , publisher=
work page 2020
-
[5]
arXiv preprint arXiv:2412.14427 , year=
Elo Ratings in the Presence of Intransitivity , author=. arXiv preprint arXiv:2412.14427 , year=
-
[6]
Computing ``elo ratings'' of move patterns in the game of go , author=. ICGA journal , volume=. 2007 , publisher=
work page 2007
-
[7]
ACM Transactions on Interactive Intelligent Systems (TiiS) , volume=
Bridging the gap between ethics and practice: guidelines for reliable, safe, and trustworthy human-centered AI systems , author=. ACM Transactions on Interactive Intelligent Systems (TiiS) , volume=. 2020 , publisher=
work page 2020
- [8]
-
[9]
Foot In The Door: Understanding Large Language Model Jailbreaking via Cognitive Psychology , author=. 2024 , eprint=
work page 2024
-
[10]
arXiv preprint arXiv:2311.09096 , year=
Defending large language models against jailbreaking attacks through goal prioritization , author=. arXiv preprint arXiv:2311.09096 , year=
-
[11]
JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models
Jailbreakbench: An open robustness benchmark for jailbreaking large language models , author=. arXiv preprint arXiv:2404.01318 , year=
work page internal anchor Pith review arXiv
-
[12]
Open sesame! universal black box jailbreaking of large language models
Open sesame! universal black box jailbreaking of large language models , author=. arXiv preprint arXiv:2309.01446 , year=
-
[13]
Jailbreaking Black Box Large Language Models in Twenty Queries
Jailbreaking black box large language models in twenty queries , author=. arXiv preprint arXiv:2310.08419 , year=
work page internal anchor Pith review arXiv
- [14]
-
[15]
European Conference on Computer Vision , pages=
Adversarial prompt tuning for vision-language models , author=. European Conference on Computer Vision , pages=. 2024 , organization=
work page 2024
-
[16]
Visual adversarial examples jailbreak aligned largelanguagemodels
Visual adversarial examples jailbreak large language models , author=. arXiv preprint arXiv:2306.13213 , year=
-
[17]
Adversarial demon- stration attacks on large language models
Adversarial demonstration attacks on large language models , author=. arXiv preprint arXiv:2305.14950 , year=
-
[18]
Universal and Transferable Adversarial Attacks on Aligned Language Models
Universal and transferable adversarial attacks on aligned language models , author=. arXiv preprint arXiv:2307.15043 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[19]
Misusing Tools in Large Language Models With Visual Adversarial Examples
Misusing tools in large language models with visual adversarial examples , author=. arXiv preprint arXiv:2310.03185 , year=
-
[20]
Promptbench: Towards evaluating the robustness of large language models on adversarial prompts , author=. arXiv e-prints , pages=
-
[21]
Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security , pages=
Large language models for code: Security hardening and adversarial testing , author=. Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security , pages=
work page 2023
-
[22]
Advances in Neural Information Processing Systems , volume=
On evaluating adversarial robustness of large vision-language models , author=. Advances in Neural Information Processing Systems , volume=
-
[23]
Survey of vulnerabilities in large language models revealed by adversarial attacks
Survey of vulnerabilities in large language models revealed by adversarial attacks , author=. arXiv preprint arXiv:2310.10844 , year=
-
[24]
Limitations of the LLM-as-a-Judge Approach for Evaluating LLM Outputs in Expert Knowledge Tasks , author=. 2024 , eprint=
work page 2024
-
[25]
Wenda Xu, Guanglei Zhu, Xuandong Zhao, Liangming Pan, Lei Li, and William Yang Wang
Systematic evaluation of llm-as-a-judge in llm alignment tasks: Explainable metrics and diverse prompt templates , author=. arXiv preprint arXiv:2408.13006 , year=
-
[26]
arXiv preprint arXiv:2402.04788 , year=
Mllm-as-a-judge: Assessing multimodal llm-as-a-judge with vision-language benchmark , author=. arXiv preprint arXiv:2402.04788 , year=
-
[27]
A Survey on LLM-as-a-Judge , author=. arXiv preprint arXiv:2411.15594 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[28]
Advances in Neural Information Processing Systems , volume=
Judging llm-as-a-judge with mt-bench and chatbot arena , author=. Advances in Neural Information Processing Systems , volume=
-
[29]
arXiv preprint arXiv:2407.10817 , year=
Foundational autoraters: Taming large language models for better automatic evaluation , author=. arXiv preprint arXiv:2407.10817 , year=
-
[30]
arXiv preprint arXiv:2311.18702 , year=
Critiquellm: Scaling llm-as-critic for effective and explainable evaluation of large language model generation , author=. arXiv preprint arXiv:2311.18702 , year=
-
[31]
arXiv preprint arXiv:2402.13764 , year=
Criticbench: Evaluating large language models as critic , author=. arXiv preprint arXiv:2402.13764 , year=
-
[32]
CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
Critic: Large language models can self-correct with tool-interactive critiquing , author=. arXiv preprint arXiv:2305.11738 , year=
work page internal anchor Pith review arXiv
-
[33]
Regulation of Emerging Risks , author=. Vand. L. Rev. , volume=. 2016 , publisher=
work page 2016
-
[34]
Human Decision-making is Susceptible to AI-driven Manipulation , author=. 2025 , eprint=
work page 2025
-
[35]
arXiv preprint arXiv:2305.15324 , year=
Model evaluation for extreme risks , author=. arXiv preprint arXiv:2305.15324 , year=
-
[36]
arXiv preprint arXiv:2412.15433 , year=
Quantifying detection rates for dangerous capabilities: a theoretical model of dangerous capability evaluations , author=. arXiv preprint arXiv:2412.15433 , year=
-
[37]
Nature machine intelligence , volume=
The global landscape of AI ethics guidelines , author=. Nature machine intelligence , volume=. 2019 , publisher=
work page 2019
-
[38]
Evaluating frontier models for dangerous capabilities , author=. arXiv preprint arXiv:2403.13793 , year=
-
[39]
Prioritizing Safeguarding Over Autonomy: Risks of LLM Agents for Science , author=. 2024 , eprint=
work page 2024
-
[40]
A., Rieser, V ., Iqbal, H., Tomašev, N., Ktena, I., Kenton, Z., Rodriguez, M., et al
The ethics of advanced ai assistants , author=. arXiv preprint arXiv:2404.16244 , year=
-
[41]
Journal of Law and Society , volume=
Regulating scientific research: A constitutional moment? , author=. Journal of Law and Society , volume=. 2018 , publisher=
work page 2018
-
[42]
Environment international , volume=
The risks of risk-based regulation: Insights from the environmental policy domain , author=. Environment international , volume=. 2006 , publisher=
work page 2006
-
[43]
Journal of the Royal Society of Medicine , volume=
Research governance: regulating risk and reducing harm? , author=. Journal of the Royal Society of Medicine , volume=. 2006 , publisher=
work page 2006
-
[44]
Regulating emerging and future technologies in the present , author=. NanoEthics , volume=. 2015 , publisher=
work page 2015
- [45]
-
[46]
Science and engineering ethics , volume=
Philosophical aspects of dual use technologies , author=. Science and engineering ethics , volume=. 2010 , publisher=
work page 2010
-
[47]
New England Journal of Medicine , volume=
Acute myeloid leukemia , author=. New England Journal of Medicine , volume=. 2015 , publisher=
work page 2015
-
[48]
Pharmaceutical statistics , volume=
Guidelines for accurate EC50/IC50 estimation , author=. Pharmaceutical statistics , volume=. 2011 , publisher=
work page 2011
-
[49]
The ethics of disseminating dual-use knowledge , author=. Research Ethics , volume=. 2013 , publisher=
work page 2013
-
[50]
Science and engineering ethics , volume=
Ethical and philosophical consideration of the dual-use dilemma in the biological sciences , author=. Science and engineering ethics , volume=. 2007 , publisher=
work page 2007
-
[51]
Bulletin of the World Health Organization , volume=
Governance of dual-use research: an ethical dilemma , author=. Bulletin of the World Health Organization , volume=. 2009 , publisher=
work page 2009
-
[52]
The Uppsala Code of Ethics for Scientists , urldate =
Bengt Gustafsson and Lars Ryd. The Uppsala Code of Ethics for Scientists , urldate =. Journal of Peace Research , number =
-
[53]
Federal Register. Vol. 44, no. 76. pp. 23191–7. Archived from the original (PDF) on October 17, 2011. , url=. 1979 , title=
work page 2011
-
[54]
Ethical principles for medical research involving human subjects , author=
World Medical Association Declaration of Helsinki. Ethical principles for medical research involving human subjects , author=. Bulletin of the world health organization , volume=
-
[55]
The Nuremberg code 70 years later , author=. Jama , volume=. 2017 , publisher=
work page 2017
-
[56]
Bulletin of the History of Medicine , volume=
The origins of informed consent: The international scientific commission on medical war crimes, and the Nuremberg Code , author=. Bulletin of the History of Medicine , volume=. 2001 , publisher=
work page 2001
-
[57]
The American journal of the medical sciences , volume=
The continuing legacy of the Tuskegee Syphilis Study: considerations for clinical investigation , author=. The American journal of the medical sciences , volume=. 1999 , publisher=
work page 1999
-
[58]
Science and Engineering Ethics , volume=
Scientific ethics: A new approach , author=. Science and Engineering Ethics , volume=. 2019 , publisher=
work page 2019
- [59]
-
[60]
The SAGE handbook of social research methods , pages=
Research ethics in social science , author=. The SAGE handbook of social research methods , pages=. 2008 , publisher=
work page 2008
- [61]
- [62]
-
[63]
The ethics of science: An introduction , author=. 2005 , publisher=
work page 2005
-
[64]
Health and human rights , volume=
Addressing inequity: neglected tropical diseases and human rights , author=. Health and human rights , volume=. 2018 , publisher=
work page 2018
-
[65]
Global Health Research and Policy , volume=
Addressing neglected tropical diseases in Africa: a health equity perspective , author=. Global Health Research and Policy , volume=. 2023 , publisher=
work page 2023
-
[66]
How AI Ideas Affect the Creativity, Diversity, and Evolution of Human Ideas: Evidence From a Large, Dynamic Experiment , author=. 2024 , eprint=
work page 2024
-
[67]
Divergent Creativity in Humans and Large Language Models , author=. 2024 , eprint=
work page 2024
-
[68]
We're Different, We're the Same: Creative Homogeneity Across LLMs , author=. 2025 , eprint=
work page 2025
-
[69]
Homogenization effects of large language models on human creative ideation
Anderson, Barrett R and Shah, Jash Hemant and Kreminski, Max , title =. Proceedings of the 16th Conference on Creativity & Cognition , pages =. 2024 , isbn =. doi:10.1145/3635636.3656204 , abstract =
-
[70]
The rating of chessplayers: Past and present , author=. (No Title) , year=
-
[71]
BMC health services research , volume=
Drug repurposing: a systematic review on root causes, barriers and facilitators , author=. BMC health services research , volume=. 2022 , publisher=
work page 2022
-
[72]
Proceedings of the National Academy of Sciences , volume=
Network medicine framework for identifying drug-repurposing opportunities for COVID-19 , author=. Proceedings of the National Academy of Sciences , volume=. 2021 , publisher=
work page 2021
-
[73]
Modeling polypharmacy side effects with graph convolutional networks , author=. Bioinformatics , volume=. 2018 , publisher=
work page 2018
-
[74]
Genetic Discovery Enabled by A Large Language Model , author=. bioRxiv , year=
-
[75]
Sara Mahdavi, Christopher Semturs, David Fleet, Vivek Natarajan, and Shekoofeh Azizi
Tx-LLM: A Large Language Model for Therapeutics , author=. arXiv preprint arXiv:2406.06316 , year=
-
[76]
ProtEx: A Retrieval-Augmented Approach for Protein Function Prediction , author=. bioRxiv , pages=. 2024 , publisher=
work page 2024
-
[77]
DepMap Q2 2024 Data Release , year =
work page 2024
-
[78]
Nucleic Acids Research , volume =
Tang, Zefang and Kang, Boxi and Li, Chenwei and Chen, Tianxiang and Zhang, Zemin , title = ". Nucleic Acids Research , volume =. 2019 , month =. doi:10.1093/nar/gkz430 , url =
-
[79]
Signal Transduction and Targeted Therapy , volume=
Drug repurposing for cancer therapy , author=. Signal Transduction and Targeted Therapy , volume=. 2024 , month=. doi:10.1038/s41392-024-01808-1 , url=
-
[80]
MIMIC-III, a freely accessible critical care database , author=. Scientific data , volume=. 2016 , publisher=
work page 2016
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.