REVIEW 41 cited by
Data Interpreter: An LLM Agent For Data Science
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Large Language Model (LLM)-based agents have shown effectiveness across many applications. However, their use in data science scenarios requiring solving long-term interconnected tasks, dynamic data adjustments and domain expertise remains challenging. Previous approaches primarily focus on individual tasks, making it difficult to assess the complete data science workflow. Moreover, they struggle to handle real-time changes in intermediate data and fail to adapt dynamically to evolving task dependencies inherent to data science problems. In this paper, we present Data Interpreter, an LLM-based agent designed to automatically solve various data science problems end-to-end. Our Data Interpreter incorporates two key modules: 1) Hierarchical Graph Modeling, which breaks down complex problems into manageable subproblems, enabling dynamic node generation and graph optimization; and 2) Programmable Node Generation, a technique that refines and verifies each subproblem to iteratively improve code generation results and robustness. Extensive experiments consistently demonstrate the superiority of Data Interpreter. On InfiAgent-DABench, it achieves a 25% performance boost, raising accuracy from 75.9% to 94.9%. For machine learning and open-ended tasks, it improves performance from 88% to 95%, and from 60% to 97%, respectively. Moreover, on the MATH dataset, Data Interpreter achieves remarkable performance with a 26% improvement compared to state-of-the-art baselines. The code is available at https://github.com/geekan/MetaGPT.
Forward citations
Cited by 41 Pith papers
-
InteractComp: Evaluating Search Agents With Ambiguous Queries
A new benchmark, InteractComp, shows that search agents rarely ask clarifying questions on ambiguous queries, causing large accuracy drops that forced interaction can partially recover.
-
Optimal-Agent-Selection: State-Aware Routing Framework for Efficient Multi-Agent Collaboration
A state-aware contrastive router that selects the most relevant agent at each step improves multi-agent LLM accuracy by up to 23.8% while using a fraction of the tokens of fixed-pipeline baselines.
-
Uncovering Vulnerabilities of LLM-Assisted Cyber Threat Intelligence
LLMs assisting cyber threat intelligence fail mainly due to spurious correlations, contradictory knowledge, and constrained generalization that stem from the threat landscape itself.
-
Tabularis Formatus: Predictive Formatting for Tables
Tafo, a neuro-symbolic system, predicts spreadsheet conditional-formatting rules including colors with no user input, and its authors report it matches user-applied formatting better than all tested baselines.
-
Non-programmers Assessing AI-Generated Code: A Case Study of Business Users Analyzing Data
Non-programmer business users often fail to spot critical mistakes in AI-generated data analyses, even when explicitly warned and incentivized.
-
MetaAgent: Automatically Constructing Multi-Agent Systems Based on Finite State Machines
An LLM-based framework that generates a tool-enabled, traceback-capable multi-agent system as a finite state machine from a task description, outperforming other automatic design methods on four benchmarks.
-
G-Memory: Tracing Hierarchical Memory for Multi-Agent Systems
G-Memory stores past multi-agent teamwork in a three-tier graph and retrieves it to boost performance on five benchmarks.
-
Large Language Models for Predictive Analysis: How Far Are They?
Existing LLMs perform poorly on predictive analysis, with the best model scoring 24.11/28 and most models failing to generate executable code.
-
MM-Agent: LLM as Agents for Real-world Mathematical Modeling Problem
MM-Agent, a multi-stage LLM pipeline with a hierarchical modeling method library, is claimed to outperform prior agents and award-winning human solutions on a new 111-problem MCM/ICM-based mathematical modeling benchmark.
-
Knowledge-Enhanced Program Repair for Data Science Code
DSrepair combines a knowledge graph of data science APIs with AST-level bug localization to repair LLM-generated code, fixing more DS-1000 tasks than five baseline repair methods.
-
Memory Offloading for Large Language Model Inference with Latency SLO Guarantees
Select-N meets LLM latency SLOs by choosing and continuously adjusting an offloading interval that controls how much model state lives in CPU memory, increasing host memory usage and throughput versus prior offloading...
-
CoddLLM: Empowering Large Language Models for Data Analytics
CoddLLM, a Mistral-NeMo-12B model post-trained on synthetic analytics instruction data, outperforms GPT-4o on the paper's table-selection average and improves text-to-SQL over its base model, with the largest gains on...
-
SOP-Agent: Empower General Purpose AI Agent with Domain-Specific SOPs
A decision-graph SOP navigator guides LLM agents through branching and looping workflows, with reported gains on household tasks, code generation, data cleaning, and a new customer-service benchmark.
-
DataLab: A Unified Platform for LLM-Powered Business Intelligence
DataLab is a unified notebook-based platform for LLM-powered BI tasks that shows strong efficiency gains and competitive accuracy, but its state-of-the-art claim is not supported on several benchmarks.
-
A corrective agentic hybrid RAG and an operations-grounded evaluation for a scientific facility
A deployed hybrid RAG for APS operations improves vital-nugget recall over BM25 mainly via cross-encoder reranking; graph and corrective loops help only marginally on a 50-question facility benchmark.
-
SCTA: An Agentic Framework for Stable and Interpretable Target Gene Discovery from Single-Cell RNA Sequencing
Decision-centric multi-agent orchestration with structured biological evidence improves repeated-run stability of scRNA-seq therapeutic target gene shortlists versus general agents and ablations.
-
StarCodex: Dynamic Coding Harness for Starlink Measurement Analysis and Experiment Automation
StarCodex uses Codex with multi-layer validation to convert evolving Starlink measurements into reusable analysis artifacts, matching supervised baselines while automating the workflow.
-
AlphaCrafter: Harnessing Multi-Agent Workflows for Cross-Sectional Quantitative Trading
A three-agent LLM pipeline for daily cross-sectional equity trading reports robust returns on CSI 300 and S&P 500, but post-hoc trial filtering, best-backbone selection, and missing code/data undermine the headline.
-
Large Language Model Agent for Structural Drawing Generation Using ReAct Prompt Engineering and Retrieval Augmented Generation
A six-stage LLM agent chain converts natural-language descriptions of three beam types into Python code for AutoCAD drawings, with per-step success rates between 77% and 100% over 100 runs.
-
Running in CIRCLE? A Simple Benchmark for LLM Code Interpreter Security
CIRCLE is a 1,260-prompt benchmark that measures how often commercial LLM code interpreters refuse, execute, or time out on resource-exhaustion tasks, revealing large and inconsistent safety gaps.
-
Agentomics-ML: Autonomous Machine Learning Experimentation Agent for Genomic and Transcriptomic Data
Agentomics-ML, an LLM-based agent with reflection, produced working classification code for genomic benchmarks in 93% of runs and beat all compared AI methods on six datasets.
-
SimuGen: Multi-modal Agentic Framework for Constructing Block Diagram-Based Simulation Models
SimuGen uses specialized LLM agents and a Simulink block database to convert diagram images into executable MATLAB/Simulink code, reporting 94.5% average accuracy on nine examples.
-
Adaptive Plan-Execute Framework for Smart Contract Security Auditing
SmartAuditFlow claims 100% detection on a standard smart contract benchmark and all 13 tested CVEs via a plan-execute LLM workflow, though the supporting evaluation has major reproducibility and validation gaps.
-
DSMentor: Enhancing Data Science Agents with Curriculum Learning and Online Knowledge Accumulation
Ordering data science problems easy-to-hard and accumulating their solutions in a memory buffer improves LLM agent pass rates on DSEval and QRData by up to 5.2%.
-
MLZero: A Multi-Agent System for End-to-end Machine Learning Automation
MLZero, an LLM-based multi-agent system with perception and dual memory, reports 92 percent success on a new 25-task multimodal AutoML benchmark and the best average rank on MLE-Bench Lite.
-
Prompt Stability Matters: Evaluating and Optimizing Auto-Generated Prompt in General-Purpose Systems
The paper shows that evaluating prompts by the semantic similarity of repeated LLM outputs, and refining prompts toward higher similarity, improves task success in general-purpose multi-agent systems.
-
MLE-Dojo: Interactive Environments for Empowering LLM Agents in Machine Learning Engineering
An open Gym-style environment running 200+ Kaggle competitions lets LLM agents iterate on ML solutions and provides a benchmark for training and evaluating them.
-
Optimization Problem Solving Can Transition to Evolutionary Agentic Workflows
An evolutionary loop of foundation-model agents could automate the full optimization pipeline, but the paper's evidence only covers two isolated components.
-
ScoreFlow: Mastering LLM Agent Workflows via Score-based Preference Optimization
ScoreFlow uses a score-weighted variant of direct preference optimization to automatically generate and refine per-task LLM agent workflows, reporting an average 8.2% improvement over baselines on six benchmarks.
-
Tree-of-Code: A Tree-Structured Exploring Framework for End-to-End Code Generation and Execution in Complex Task Handling
Tree-of-Code improves code-based LLM agents by generating complete end-to-end programs as tree nodes, using execution success to self-filter and majority voting to pick the final answer.
-
Exploring Agentic Workflows for Generating High Quality Math Visual Aids
An exploratory agentic self-improvement loop for TikZ math diagrams yields modest human-rated gains but fails to fix many spatial and coverage errors.
-
Data Agent: A Holistic Architecture for Orchestrating Data+AI Ecosystems
A position paper introducing a holistic 'Data Agent' architecture for autonomous Data+AI orchestration, with proposed iDataScience, data analytics, and DBA agents, but no empirical evaluation.
-
LLM-Agent-Controller: A Universal Multi-Agent Large Language Model System as a Control Engineer
A nine-agent LLM system with RAG and 140+ Python Control Library tools solves textbook control-theory tasks with a reported 83% completion rate.
-
GPT-HTree: A Decision Tree Framework Integrating Hierarchical Clustering and Large Language Models for Explainable Classification
GPT-HTree combines hierarchical clustering, per-cluster decision trees, and LLM-generated persona descriptions, and claims to identify VC founder clusters with up to 9x success rates, but the claim is not validated ou...
-
Flow: Modularized Agentic Workflow Automation
Flow represents a task as a dependency graph of subtasks and lets LLM agents redraw that graph during execution, reporting better success rates than three baselines on three coding tasks.
-
SRSA: A Cost-Efficient Strategy-Router Search Agent for Real-world Human-Machine Interactions
SRSA, a router that picks between direct, parallel, and planning searches for each query, improves informativeness and completeness on a new contextual-query benchmark while using fewer LLM inference steps than a ReAct agent.
-
Agentic Web: Weaving the Next Web with AI Agents
A position paper defines the Agentic Web as the next web era and proposes a three-dimensional conceptual framework for understanding and building it.
-
Simulation Agent: A Framework for Integrating Simulation and Large Language Models for Enhanced Decision-Making
The paper proposes an LLM-based agent as a natural-language bridge to simulation models, but it reports no implementation or evaluation.
-
Knowledge Augmented Complex Problem Solving with Large Language Models: A Survey
A survey that organizes LLM-based complex problem solving into three components, multi-step reasoning, domain knowledge, and result verification, and reviews methods and challenges across four application domains.
-
Applied Statistics in the Era of Artificial Intelligence: A Review and Vision
A review and vision paper: applied statistics and AI are complementary, and statisticians should focus on uniquely human skills as AI automates routine analysis.
-
A Survey on Human-Centric LLMs
A review that sorts existing evidence on how well large language models imitate individual human skills and collective social dynamics into one taxonomy.
Discussion (0). Continue with ORCID to comment.