Pith. sign in

REVIEW 5 major objections 6 minor 2 cited by

ROMAS: A Role-Based Multi-Agent System for Database monitoring and Planning

T0 review · 5 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A role-based multi-agent system with a global monitor reports higher accuracy than prior LLM agent systems on financial and multi-hop reasoning benchmarks.

desk verdict Plausible agent framework with a thin 100-sample evaluation; the monitor mechanism is interesting but the claimed superiority is not established. read the letter →

arxiv 2412.13520 v1 pith:7M7GLYO3 submitted 2024-12-18 cs.AI cs.DBcs.MA

Yi Huang , Fangyin Cheng , Fan Zhou , Jiahui Li , Jian Gong , Hongjun Yang , Zhidong Fan , Caigao Jiang
show 2 more authors
Siqiao Xue Faqiang Chen
This is my paper · ORCID
classification cs.AIcs.DBcs.MA
keywords multi-agentsystemsLLMagentsrole-basedcooperationself-monitoringself-planningdatabaseanalyticserrortreesearchlow-codedevelopment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that LLM-powered multi-agent systems for data analysis fail mainly because they cannot correct an executed plan, not because they cannot make one. It introduces ROMAS, a role-based system in which a planner builds an agent team and task list, workers execute, and a monitor observes the global state, classifies errors, and either repairs them directly or sends the planner back to re-plan. On 100-case subsets of a financial multimodal QA benchmark and a multi-hop reasoning benchmark, ROMAS reports success rates of 81.68% and 85.24%, above the strongest role-based comparison system. If the result holds, the practical lesson is that an explicit monitoring and re-planning role is what makes a multi-agent system robust on real data tasks.

What carries the argument

The load-bearing mechanism is the monitor's error tree search and the re-planning loop. The monitor uses a predefined error tree, built from empirical data, with two top-level branches: pipeline errors and logical (agent-team generation) errors; it searches from the root to a leaf node by similarity to classify what went wrong. Pipeline errors receive direct correction instructions, while agent-team errors send global information and recommendations back to the planner, which regenerates a strategy under the gap-narrow rule, a constraint that changes the previous plan as little as possible while fixing the observed failures.

What would settle it

Take a random sample of failure logs from the two benchmarks, have independent raters label each error's branch, and compare their labels with the monitor's classifications; if agreement is low, the error tree is not doing the claimed work. A sharper test is to inject a failure type outside the paper's two branches, such as a tool-permission or prompt-security error, and observe whether the monitor can still produce a repair.

Watch

Extended reading notes

Core claim

The central claim is that the bottleneck in LLM multi-agent data analytics is the absence of an execution-time correction channel. ROMAS instantiates a planner, a monitor, and a set of workers, then adds a re-planning phase: workers attempt self-reflection first, broadcast their global state to the monitor on persistent failure, and the monitor classifies each error as a pipeline error, which it fixes directly, or an agent-team generation error, which triggers the planner to rebuild the strategy. The paper reports that removing the monitor causes the largest success-rate drop on the financial benchmark, 22.66 percentage points, and that the full system reaches 81.68% on FAMMA and 85.24% on HotpotQA.

Load-bearing premise

The monitor's error classification depends on a predefined error tree built from empirical data that the paper does not describe or validate; if a real error falls outside the tree or is misclassified, the monitor cannot fix it and the claimed advantage over systems without a monitor disappears.

Editorial extensions

If this is right

  • Plan-only multi-agent architectures are insufficient: the ablation attributes the largest single drop on the finance benchmark, 22.66 percentage points, to removing the monitor.
  • Error correction is tiered by cost: workers self-correct, the monitor fixes pipeline errors directly, and only agent-team generation errors trigger a full re-plan.
  • The gap-narrow rule keeps re-planning cheap by reusing the old strategy, which is why the re-planning phase can run within a bounded number of retries.
  • The component that matters most depends on the task: the finance benchmark loses most from removing the monitor, while the multi-hop reasoning benchmark loses most from removing memory and self-reflection.
  • A database-focused low-code framework can implement the same system with less development code and lower average response time than general-purpose agent frameworks in the paper's comparison.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the monitor's error tree is the load-bearing component, a natural extension is to learn the tree from failure logs rather than fixing it by hand and to measure how often the monitor's classification agrees with human raters.
  • The gap-narrow rule states a general principle for LLM agents: constrain revisions to the smallest edit that fixes the observed failure, a principle that should transfer to code repair and other self-correcting settings.
  • The reported numbers come from 100-case subsets, so scaling to the full benchmarks would test whether the margins survive broader sampling.
  • A production side effect of an explicit monitor role is auditability: each correction can be attributed to a monitor decision rather than hidden inside a planner prompt.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. This paper proposes ROMAS, a role-based multi-agent system deployed inside the DB-GPT framework. It organizes agents into a planner, a monitor, and workers; the planner generates an agent team and task lists, workers execute and self-reflect, and the monitor classifies errors by searching a predefined error tree and either repairs pipeline errors or triggers re-planning through a 'gap narrow' rule. The empirical section compares ROMAS with LLM, single-agent, and multi-agent baselines on 100-sample subsets of FAMMA and HotpotQA, reports ablations, and compares DB-GPT with LangChain and AgentScope. The paper claims ROMAS achieves 81.68% success on FAMMA and 85.24% on HotpotQA, outperforming all baselines, and that DB-GPT enables significantly less code.

Significance. If the reported results were robust, ROMAS would be a useful instance of a role-based, self-monitoring multi-agent architecture for database-backed QA, and the deployment in a real open-source framework (DB-GPT) is a concrete strength. The separation of planner/monitor/workers and the gap-narrow replanning mechanism are clearly described, and the ablation design at least targets the main components. The central weakness is that every quantitative claim rests on 100-case self-selected subsets without uncertainty quantification, and the baselines and components are not fully specified; the empirical contribution is therefore not yet reproducible. The paper would be strengthened substantially by code/data release and full-benchmark or properly sampled evaluation with statistical tests.

major comments (5)
  1. [§4 Datasets / Table 3] The main claim that ROMAS 'outperforms the others' is supported only by point estimates on 100 self-selected samples per dataset. The margin over AutoAgents is 8.23 points on FAMMA and 6.25 points on HotpotQA, i.e. roughly 8 and 6 questions out of 100, and no confidence interval, bootstrap, McNemar test, or repeated-seed result is reported. The FAMMA subset is further filtered to cases containing both text and table images and then converted from images to tabular format, which removes the multimodal component and may change task difficulty. To support the central claim, report results on the full benchmarks or on a documented random sample with uncertainty quantification, and report baselines under identical conditions.
  2. [§3.2 / Appendix D] The monitor's error classification relies on a predefined error tree said to be built from 'empirical data', but the paper does not describe the tree's nodes, construction, coverage, or validation; Fig. 7 is only a coarse schematic. Because Table 4 attributes a 22.66-point FAMMA drop to removing the monitor, an underspecified error tree is a load-bearing component. Please provide the full tree (list of nodes/leaves), examples of error instances, and an analysis of classification accuracy/coverage.
  3. [§4 Setup / Analysis I] The baseline configurations are not reproducible. The paper does not state prompt templates, retry limits, temperature, tool access, or database connection details for CoT, ToT, ReAct, Generative Agents, and AutoAgents. Since ROMAS operates inside DB-GPT with its own tools, differences in tool availability and retry budget could explain the reported margins. Provide exact code/configurations and, ideally, run baselines within the same DB-GPT environment.
  4. [§4 Analysis II / Table 4] The ablation study is performed on the same 100-sample self-selected subsets and reports single runs without variance. In particular, the largest ablation drop (22.66 points on FAMMA) is not accompanied by repeated seeds or error bars, so it is not possible to separate the effect of the component from subset-selection effects. Please add repeated runs, confidence intervals, and ideally ablations on the full benchmark.
  5. [§4 Analysis III / Table 5] The DB-GPT effectiveness comparison is conducted by the authors of DB-GPT and ROMAS, using 'code volume (number of rows)' as a metric. The table does not report which features are implemented, the exact counting convention, or the code repositories, and no significance testing is provided. This weakens the low-code-development claim; please make the implementations public and specify the feature set per framework, or use an independent implementer.
minor comments (6)
  1. [Throughout] There are typos such as 'versatil', 'specailized', 'Re-plannig', and 'Cleanning'; these should be corrected.
  2. [§4 Analysis III] The sentence refers to 'table 4' when the DB-GPT comparison is in Table 5.
  3. [Appendix D] The caption and figure for the error tree do not show the actual nodes or leaf definitions; please include a legible tree or a tabular listing.
  4. [Appendix C] The description of the LLM and human evaluation does not state which model served as judge, how the ten dimensions are aggregated, or how expert disagreement was handled; an inter-annotator agreement statistic would help.
  5. [Algorithm 1] The loop over differences and rules is ambiguous, especially after a difference is regenerated; please clarify with explicit continue/break conditions.
  6. [Tables 1-2] The checkmarks in Tables 1 and 2 would benefit from a precise legend defining each capability and the evidence used for each mark.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ROMAS's headline results are measured against external baselines, and the DB-GPT/FAMMA self-citations are contextual rather than load-bearing.

full rationale

ROMAS's central claims are empirical benchmark comparisons and are not defined in terms of its own outputs. Table 3 compares ROMAS against external baselines (GPT-4, CoT, ToT, ReAct, Generative Agents, AutoAgents) on standard-option success rate, so the reported margins are externally falsifiable measurements rather than consequences of a fitted parameter. No parameter is fitted to a subset and then relabeled as a prediction: the 100-case FAMMA/HotpotQA selections (Sec. 4) are evaluation choices, and the ablation table (Table 4) measures actual removals of components. The self-citations to DB-GPT [Xue et al., 2023a, 2024b] and FAMMA [Xue et al., 2024a] provide the implementation substrate and dataset, respectively; they do not by themselves entail ROMAS's ranking. The paper does disclose a hand-built design input, the error tree "based on empirical data" (Sec. 3.2, Appendix D), whose completeness is unvalidated, and it uses GPT-4 both as the engine and as an LLM evaluator; these are correctness and validity risks, not circular reductions, because the headline success rates are not derived from the error tree or from the LLM evaluator. No uniqueness theorem, ansatz-by-citation, or rename-a-known-result pattern is present.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim depends on GPT-4 behaving as a reliable planner, worker, and monitor under the designed prompts; this is an unverified domain assumption about a proprietary model. The error tree, retry limits, temperature, and the 100-case subsets are choices made by hand with no sensitivity analysis, effectively free parameters of the evaluation. The paper postulates no new physical or conceptual entities; the role-based structure is standard in the multi-agent literature. The reliance on DB-GPT, the authors' own framework, introduces a mild self-citation concern but does not by itself invalidate the external benchmark comparisons.

free parameters (5)
  • Per-agent retry limits = 2 (self-reflection/planning), 3 (re-planning)
    Manually chosen limits affect the amount of correction possible and therefore the reported success rates; no sensitivity analysis is given.
  • Temperature = 0
    Chosen for consistency; affects determinism of LLM outputs and hence the reproducibility of results.
  • FAMMA subset size and selection = 100 cases
    The paper filters to 100 cases with specific properties and converts images to tables; the representativeness of this subset is not established.
  • Error tree structure = not fully disclosed
    The error taxonomy is said to be based on empirical data but the tree is only shown as a figure; its categories and thresholds are not specified.
  • Evaluation rubric = 10 dimensions, 100 total points
    The dimensions are listed in Appendix C but the scoring instructions are not given, so human and LLM scores are not independently reproducible.
assumptions (4)
  • domain assumption GPT-4 can reliably perform the role-based planning, tool calling, and reflection steps required by ROMAS when given the designed prompts.
    The system's behavior depends entirely on an opaque proprietary model; no evidence is provided that the prompting strategy transfers to other models.
  • ad hoc to paper The predefined error tree (Appendix D) is complete enough to classify all errors the monitor will encounter.
    The tree is described as based on empirical data but no data or validation is presented.
  • ad hoc to paper The 100-case subsets are representative of the FAMMA and HotpotQA benchmarks and the image-to-table conversion does not change task difficulty.
    No evidence of representativeness is given, and the conversion materially changes the input format.
  • domain assumption Success rate, LLM evaluation, and human evaluation as operationalized in the paper are valid measures of system quality for these tasks.
    Success criteria are not precisely defined (e.g., string matching rules), and the LLM evaluator is the same model family as the system.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ROMAS: A Role-Based Multi-Agent System for Database monitoring and Planning." pith.science (2026). https://pith.science/paper/7M7GLYO3

@misc{pith2026241213520,
  author       = {Pith},
  title        = {Pith review of: ROMAS: A Role-Based Multi-Agent System for Database monitoring and Planning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7M7GLYO3}},
  note         = {Machine review of arXiv:2412.13520}
}
read the original abstract

In recent years, Large Language Models (LLMs) have demonstrated remarkable capabilities in data analytics when integrated with Multi-Agent Systems (MAS). However, these systems often struggle with complex tasks that involve diverse functional requirements and intricate data processing challenges, necessitating customized solutions that lack broad applicability. Furthermore, current MAS fail to emulate essential human-like traits such as self-planning, self-monitoring, and collaborative work in dynamic environments, leading to inefficiencies and resource wastage. To address these limitations, we propose ROMAS, a novel Role-Based M ulti-A gent System designed to adapt to various scenarios while enabling low code development and one-click deployment. ROMAS has been effectively deployed in DB-GPT [Xue et al., 2023a, 2024b], a well-known project utilizing LLM-powered database analytics, showcasing its practical utility in real-world scenarios. By integrating role-based collaborative mechanisms for self-monitoring and self-planning, and leveraging existing MAS capabilities to enhance database interactions, ROMAS offers a more effective and versatile solution. Experimental evaluations of ROMAS demonstrate its superiority across multiple scenarios, highlighting its potential to advance the field of multi-agent data analytics.

Figures

Figures reproduced from arXiv: 2412.13520 by the authors.

Figure 1
Figure 1. ROMAS framework. The blue lines represent key message exchanged between agents and [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Initialization phase of ROMAS, the planner is primarily responsible for two steps: self [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Execution phase of ROMAS, workers execute tasks based on the strategies formulated by [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Re-plannig phase of ROMAS, planner receives global critical information and modification [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Figure 1 shows the average proportions of different subtask types in the FAMMA and [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: We divided the evaluation criteria into 10 dimensions based on common standards for large [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Error tree search, we classified common errors in MAS into two categories based on [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SQL-Factory: A Multi-Agent Framework for High-Quality and Large-Scale SQL Generation

    cs.DB 2025-04 conditional novelty 6.0 of 10

    A three-team multi-agent framework (GPT-4o exploration, lightweight local expansion, adaptive management) generates over 300,000 diverse SQL queries for under $200 and improves downstream text-to-SQL and clustering baselines.

  2. Perspective on Utilizing Foundation Models for Laboratory Automation in Materials Research

    cs.RO 2025-06 accept novelty 1.0 of 10

    A perspective article reviews the state of using foundation models for laboratory automation and proposes a roadmap for fully autonomous experiments.

Reference graph

Works this paper leans on

64 extracted references · 15 canonical work pages · cited by 2 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

    work page

  2. [2]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  3. [3]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  4. [4]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

    work page

  5. [5]

    On evaluation of embodied navigation agents

    Peter Anderson, Angel Chang, Devendra Singh Chaplot, Alexey Dosovitskiy, Saurabh Gupta, Vladlen Koltun, Jana Kosecka, Jitendra Malik, Roozbeh Mottaghi, Manolis Savva, et al. On evaluation of embodied navigation agents. arXiv preprint arXiv:1807.06757, 2018. URL https://arxiv.org/abs/1807.06757

    arXiv 2018

  6. [6]

    Browne, Edward Powley, Daniel Whitehouse, Simon M

    Cameron B. Browne, Edward Powley, Daniel Whitehouse, Simon M. Lucas, Peter I. Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyridon Samothrakis, and Simon Colton. A survey of monte carlo tree search methods. IEEE Transactions on Computational Intelligence and AI in Games, 4 0 (1): 0 1--43, 2012. doi:10.1109/TCIAIG.2012.2186810

    arXiv 2012

  7. [7]

    Lang C hain, 2022

    Harrison Chase. Lang C hain, 2022. URL https://github.com/hwchase17/langchain

    work page 2022

  8. [8]

    Karlsson, Jie Fu, and Yemin Shi

    Guangyao Chen, Siwei Dong, Yu Shu, Ge Zhang, Jaward Sesay, Börje F. Karlsson, Jie Fu, and Yemin Shi. Autoagents: A framework for automatic agent generation, 2024. URL https://arxiv.org/abs/2309.17288

    arXiv 2024

Show all 64 references
  1. [9]

    Chatcot: Tool-augmented chain-of-thought reasoning on chat-based large language models, 2023

    Zhipeng Chen, Kun Zhou, Beichen Zhang, Zheng Gong, Wayne Xin Zhao, and Ji-Rong Wen. Chatcot: Tool-augmented chain-of-thought reasoning on chat-based large language models, 2023. URL https://arxiv.org/abs/2305.14323

    2023 arXiv

  2. [10]

    Leveraging large language models for pre-trained recommender systems

    Zhixuan Chu, Hongyan Hao, Xin Ouyang, Simeng Wang, Yan Wang, Yue Shen, Jinjie Gu, Qing Cui, Longfei Li, Siqiao Xue, et al. Leveraging large language models for pre-trained recommender systems. arXiv preprint arXiv:2308.10837, 2023

    2023 arXiv

  3. [11]

    documentqa

    documentqa. documentqa. see https://huggingface.co/tasks/document-question-answering, 2023

    2023

  4. [12]

    Leibo, Usman Islam, Richard Willis, and Peter Sunehag

    Yali Du, Joel Z. Leibo, Usman Islam, Richard Willis, and Peter Sunehag. A review of cooperation in multi-agent learning, 2023. URL https://arxiv.org/abs/2312.05162

    2023 arXiv

  5. [13]

    Codebert: A pre-trained model for programming and natural languages, 2020

    Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. Codebert: A pre-trained model for programming and natural languages, 2020. URL https://arxiv.org/abs/2002.08155

    2020 arXiv

  6. [14]

    Agentscope: A flexible yet robust multi-agent platform, 2024

    Dawei Gao, Zitao Li, Xuchen Pan, Weirui Kuang, Zhijian Ma, Bingchen Qian, Fei Wei, Wenhao Zhang, Yuexiang Xie, Daoyuan Chen, Liuyi Yao, Hongyi Peng, Zeyu Zhang, Lin Zhu, Chen Cheng, Hongzhu Shi, Yaliang Li, Bolin Ding, and Jingren Zhou. Agentscope: A flexible yet robust multi-...

    2024 arXiv

  7. [15]

    Meta GPT : Meta programming for a multi-agent collaborative framework

    Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and J \"u rgen Schmidhuber. Meta GPT : Meta programming for a multi-agent collaborat...

    2024

  8. [16]

    Understanding the planning of llm agents: A survey, 2024

    Xu Huang, Weiwen Liu, Xiaolong Chen, Xingmei Wang, Hao Wang, Defu Lian, Yasheng Wang, Ruiming Tang, and Enhong Chen. Understanding the planning of llm agents: A survey, 2024. URL https://arxiv.org/abs/2402.02716

    2024 arXiv

  9. [17]

    Zhang, Jun Zhou, Defu Lian, and Ying Wei

    Gangwei Jiang, Caigao Jiang, Siqiao Xue, James Y. Zhang, Jun Zhou, Defu Lian, and Ying Wei. Towards anytime fine-tuning: Continually pre-trained language models with hypernetwork prompt. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing ...

    2023 arXiv

  10. [18]

    Interpretable catastrophic forgetting of large language model fine-tuning via instruction vector

    Gangwei Jiang, Caigao Jiang, Zhaoyi Li, Siqiao Xue, Jun Zhou, Linqi Song, Defu Lian, and Ying Wei. Interpretable catastrophic forgetting of large language model fine-tuning via instruction vector. https://arxiv.org/abs/2406.12227, 2024. URL https://arxiv.org/abs/2406.12227

    2024 arXiv

  11. [19]

    Tseng, Yu Zheng, Lei Chen, and Hui Xiong

    Ming Jin, Qingsong Wen, Yuxuan Liang, Chaoli Zhang, Siqiao Xue, Xue Wang, James Zhang, Yi Wang, Haifeng Chen, Xiaoli Li, Shirui Pan, Vincent S. Tseng, Yu Zheng, Lei Chen, and Hui Xiong. Large models for time series and spatio-temporal data: A survey and outlook, 2023

    2023

  12. [20]

    Siegel, Nitya Nadgir, and Arvind Narayanan

    Sayash Kapoor, Benedikt Stroebl, Zachary S. Siegel, Nitya Nadgir, and Arvind Narayanan. Ai agents that matter, 2024. URL https://arxiv.org/abs/2407.01502

    2024 arXiv

  13. [21]

    Agent-oriented planning in multi-agent systems, 2024 a

    Ao Li, Yuexiang Xie, Songze Li, Fugee Tsung, Bolin Ding, and Yaliang Li. Agent-oriented planning in multi-agent systems, 2024 a . URL https://arxiv.org/abs/2410.02189

    2024 arXiv

  14. [22]

    Optimizing language models with fair and stable reward composition in reinforcement learning

    Jiahui Li, Hanlin Zhang, Fengda Zhang, Tai-Wei Chang, Kun Kuang, Long Chen, and Jun Zhou. Optimizing language models with fair and stable reward composition in reinforcement learning. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Confer...

    2024 doi

  15. [23]

    Personal llm agents: Insights and survey about the capability, efficiency and security, 2024 c

    Yuanchun Li, Hao Wen, Weijun Wang, Xiangyu Li, Yizhen Yuan, Guohong Liu, Jiacheng Liu, Wenxing Xu, Xiang Wang, Yi Sun, Rui Kong, Yile Wang, Hanfei Geng, Jian Luan, Xuefeng Jin, Zilong Ye, Guanjing Xiong, Fan Zhang, Xiang Li, Mengwei Xu, Zhijun Li, Peng Li, Yang Liu, Ya-Qin Zha...

    2024 arXiv

  16. [24]

    A review: Data pre-processing and data augmentation techniques

    Kiran Maharana, Surajit Mondal, and Bhushankumar Nemade. A review: Data pre-processing and data augmentation techniques. Global Transitions Proceedings, 3 0 (1): 0 91--99, 2022. ISSN 2666-285X. doi:https://doi.org/10.1016/j.gltp.2022.04.020. URL https://www.sciencedirect.com/s...

    2022 doi

  17. [25]

    The landscape of emerging ai agent architectures for reasoning, planning, and tool calling: A survey, 2024

    Tula Masterman, Sandi Besen, Mason Sawtell, and Alex Chao. The landscape of emerging ai agent architectures for reasoning, planning, and tool calling: A survey, 2024. URL https://arxiv.org/abs/2404.11584

    2024 arXiv

  18. [26]

    Microsoft. bing. see https://www.microsoft.com/zh-cn/bing/search-app-desktop?rtc=1, 2023

    2023

  19. [27]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff ...

    2024 arXiv

  20. [28]

    Deep optimal timing strategies for time series

    Chen Pan, Fan Zhou, Xuanwei Hu, Xinxin Zhu, Wenxin Ning, Zi Zhuang, Siqiao Xue, James Zhang, and Yunhua Hu. Deep optimal timing strategies for time series. In ICDM, 2023

    2023

  21. [29]

    O'Brien, Carrie J

    Joon Sung Park, Joseph C. O'Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. Generative agents: Interactive simulacra of human behavior, 2023. URL https://arxiv.org/abs/2304.03442

    2023 arXiv

  22. [30]

    Graph retrieval-augmented generation: A survey, 2024

    Boci Peng, Yun Zhu, Yongchao Liu, Xiaohe Bo, Haizhou Shi, Chuntao Hong, Yan Zhang, and Siliang Tang. Graph retrieval-augmented generation: A survey, 2024. URL https://arxiv.org/abs/2408.08921

    2024 arXiv

  23. [31]

    Chatdev: Communicative agents for software development, 2024

    Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. Chatdev: Communicative agents for software development, 2024. URL https://arxiv.org/abs/2307.07924

    2024 arXiv

  24. [32]

    Taskweaver: A code-first agent framework, 2024

    Bo Qiao, Liqun Li, Xu Zhang, Shilin He, Yu Kang, Chaoyun Zhang, Fangkai Yang, Hang Dong, Jue Zhang, Lu Wang, Minghua Ma, Pu Zhao, Si Qin, Xiaoting Qin, Chao Du, Yong Xu, Qingwei Lin, Saravan Rajmohan, and Dongmei Zhang. Taskweaver: A code-first agent framework, 2024. URL https...

    2024 arXiv

  25. [33]

    RasaHQ. Rasa. https://github.com/RasaHQ/rasa, 2023

    2023

  26. [34]

    Depth-first search and linear graph algorithms

    Robert Tarjan. Depth-first search and linear graph algorithms. SIAM Journal on Computing, 1 0 (2): 0 146--160, 1972. doi:10.1137/0201010. URL https://doi.org/10.1137/0201010

    1972 doi

  27. [35]

    Llama 2: Open foundation and fine-tuned chat models, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, W...

    2023 arXiv

  28. [36]

    Appworld: A controllable world of apps and people for benchmarking interactive coding agents

    Harsh Trivedi, Tushar Khot, Mareike Hartmann, Ruskin Manku, Vinty Dong, Edward Li, Shashank Gupta, Ashish Sabharwal, and Niranjan Balasubramanian. Appworld: A controllable world of apps and people for benchmarking interactive coding agents. In Proceedings of the Annual Meeting...

    2024 arXiv

  29. [37]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2023. URL https://arxiv.org/abs/1706.03762

    2023 arXiv

  30. [38]

    Voyager: A n open-ended embodied agent with large language models

    Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: A n open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291, 2023 a . URL https://arxiv.org/abs/2305.16291

    2023 arXiv

  31. [39]

    A survey on large language model based autonomous agents

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Jirong Wen. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18 0 (6), March 2024 a . I...

    2024 doi

  32. [40]

    Large language models are not fair evaluators, 2023 b

    Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. Large language models are not fair evaluators, 2023 b . URL https://arxiv.org/abs/2305.17926

    2023 arXiv

  33. [41]

    Self-consistency improves chain of thought reasoning in language models, 2023 c

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models, 2023 c . URL https://arxiv.org/abs/2203.11171

    2023 arXiv

  34. [42]

    Llmrg: Improving recommendations through large language model reasoning graphs

    Yan Wang, Zhixuan Chu, Xin Ouyang, Simeng Wang, Hongyan Hao, Yue Shen, Jinjie Gu, Siqiao Xue, James Y Zhang, Qing Cui, Longfei Li, Jun Zhou, and Sheng Li. Llmrg: Improving recommendations through large language model reasoning graphs. In Proceedings of the AAAI Conference on A...

    2024

  35. [43]

    Learning to prompt for continual learning

    Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 139--149, 2022

    2022

  36. [44]

    Chain of thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems (NeurIPS), 2022. URL https://arxiv.org/abs/2201.11903.pdf

    2022 arXiv

  37. [45]

    Autogen: Enabling next-gen llm applications via multi-agent conversation, 2023 a

    Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, Ahmed Hassan Awadallah, Ryen W White, Doug Burger, and Chi Wang. Autogen: Enabling next-gen llm applications via multi-agent conversation, 2023 a . URL ht...

    2023 arXiv

  38. [46]

    Bloomberggpt: A large language model for finance

    Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. Bloomberggpt: A large language model for finance. arXiv preprint arXiv:2303.17564, 2023 b . URL https://arxiv.org/abs/2303.17564

    2023 arXiv

  39. [47]

    A meta reinforcement learning approach for predictive autoscaling in the cloud

    Siqiao Xue, Chao Qu, Xiaoming Shi, Cong Liao, Shiyi Zhu, Xiaoyu Tan, Lintao Ma, Shiyu Wang, Shijun Wang, Yun Hu, Lei Lei, Yangfei Zheng, Jianguo Li, and James Zhang. A meta reinforcement learning approach for predictive autoscaling in the cloud. In KDD '22: The 28th ACM SIGKDD...

    2022

  40. [48]

    Hypro: A hybridly normalized probabilistic model for long-horizon prediction of event sequences

    Siqiao Xue, Xiaoming Shi, Y James Zhang, and Hongyuan Mei. Hypro: A hybridly normalized probabilistic model for long-horizon prediction of event sequences. In Advances in Neural Information Processing Systems (NeurIPS), 2022 b . URL https://arxiv.org/abs/2210.01753

    2022 arXiv

  41. [49]

    Db-gpt: Empowering database interactions with private large language models

    Siqiao Xue, Caigao Jiang, Wenhui Shi, Fangyin Cheng, Keting Chen, Hongjun Yang, Zhiping Zhang, Jianshan He, Hongyang Zhang, Ganglin Wei, Wang Zhao, Fan Zhou, Danrui Qi, Hong Yi, Shaodong Liu, and Faqiang Chen. Db-gpt: Empowering database interactions with private large languag...

    2023 arXiv

  42. [50]

    Prompt-augmented temporal point process for streaming event sequence

    Siqiao Xue, Yan Wang, Zhixuan Chu, Xiaoming Shi, Caigao Jiang, Hongyan Hao, Gangwei Jiang, Xiaoyun Feng, James Zhang, and Jun Zhou. Prompt-augmented temporal point process for streaming event sequence. In Advances in Neural Information Processing Systems (NeurIPS), 2023 b . UR...

    2023 arXiv

  43. [51]

    Weaverbird: Empowering financial decision-making with large language model, knowledge base, and search engine

    Siqiao Xue, Fan Zhou, Yi Xu, Ming Jin, Qingsong Wen, Hongyan Hao, Qingyang Dai, Caigao Jiang, Hongyu Zhao, Shuo Xie, Jianshan He, James Zhang, and Hongyuan Mei. Weaverbird: Empowering financial decision-making with large language model, knowledge base, and search engine. arXiv...

    2023 arXiv

  44. [52]

    Famma: A benchmark for financial domain multilingual multimodal question answering

    Siqiao Xue, Tingting Chen, Fan Zhou, Qingyang Dai, Zhixuan Chu, and Hongyuan Mei. Famma: A benchmark for financial domain multilingual multimodal question answering. arXiv preprint arXiv:2410.04526, 2024 a . URL https://arxiv.org/abs/2410.04526

    2024 arXiv

  45. [53]

    Demonstration of db-gpt: Next generation data interaction system empowered by large language models

    Siqiao Xue, Danrui Qi, Caigao Jiang, Wenhui Shi, Fangyin Cheng, Keting Chen, Hongjun Yang, Zhiping Zhang, Jianshan He, Hongyang Zhang, Ganglin Wei, Wang Zhao, Fan Zhou, Hong Yi, Shaodong Liu, Hongjun Yang, and Faqiang Chen. Demonstration of db-gpt: Next generation data interac...

    2024

  46. [54]

    Zhang, Qingsong Wen, Jun Zhou, and Hongyuan Mei

    Siqiao Xue, Xiaoming Shi, Zhixuan Chu, Yan Wang, Hongyan Hao, Fan Zhou, Caigao Jiang, Chen Pan, James Y. Zhang, Qingsong Wen, Jun Zhou, and Hongyuan Mei. Easytpp: Towards open benchmarking temporal point processes. In International Conference on Learning Representations (ICLR)...

    2024 arXiv

  47. [55]

    Qwen2 technical report

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze...

    2024 arXiv

  48. [56]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. H otpot QA : A dataset for diverse, explainable multi-hop question answering. In Proceedings of the Conference on Empirical Methods in Natural Language Process...

    2018

  49. [57]

    Griffiths, Yuan Cao, and Karthik Narasimhan

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of Thoughts : Deliberate problem solving with large language models. In Advances in Neural Information Processing Systems (NeurIPS), 2023 a . URL https://arxiv.org/abs...

    2023 arXiv

  50. [58]

    React: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In Proceedings of the International Conference on Learning Representations (ICLR), 2023 b . URL https://arxiv.org/abs/2210.03629

    2023 arXiv

  51. [59]

    A survey on multimodal large language models

    Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. A survey on multimodal large language models. National Science Review, page nwae403, 11 2024. ISSN 2095-5138. doi:10.1093/nsr/nwae403. URL https://doi.org/10.1093/nsr/nwae403

    2024 doi

  52. [60]

    Coevolution of role-based cooperation in multiagent systems

    Chern Han Yong and Risto Miikkulainen. Coevolution of role-based cooperation in multiagent systems. IEEE Transactions on Autonomous Mental Development, 1 0 (3): 0 170--186, 2009. doi:10.1109/TAMD.2009.2037732

    2009

  53. [61]

    An overview on feedback mechanisms with minimum adjustment or cost in consensus reaching in group decision making: Research paradigms and challenges

    Hengjie Zhang, Sihai Zhao, Gang Kou, Cong-Cong Li, Yucheng Dong, and Francisco Herrera. An overview on feedback mechanisms with minimum adjustment or cost in consensus reaching in group decision making: Research paradigms and challenges. Information Fusion, 60: 0 65--79, 2020....

    2020 doi

  54. [62]

    A survey on the memory mechanism of large language model based agents, 2024

    Zeyu Zhang, Xiaohe Bo, Chen Ma, Rui Li, Xu Chen, Quanyu Dai, Jieming Zhu, Zhenhua Dong, and Ji-Rong Wen. A survey on the memory mechanism of large language model based agents, 2024. URL https://arxiv.org/abs/2404.13501

    2024 arXiv

  55. [63]

    Gmp-ar: Granularity message passing and adaptive reconciliation for temporal hierarchy forecasting

    Fan Zhou, Chen Pan, Lintao Ma, Yu Liu, Siqiao Xue, James Zhang, Jun Zhou, Hongyuan Mei, Weitao Lin, Zi Zhuang, Wenxin Ning, and Yunhua Hu. Gmp-ar: Granularity message passing and adaptive reconciliation for temporal hierarchy forecasting. In Proceedings of the AAAI Conference ...

    2024

  56. [64]

    Db-gpt-hub: Towards open benchmarking text-to-sql empowered by large language models

    Fan Zhou, Siqiao Xue, Danrui Qi, Wenhui Shi, Wang Zhao, Ganglin Wei, Hongyang Zhang, Caigai Jiang, Gangwei Jiang, Zhixuan Chu, and Faqiang Chen. Db-gpt-hub: Towards open benchmarking text-to-sql empowered by large language models. arXiv preprint arXiv:2406.11434, 2024 b . URL ...

    2024 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.