Pith. sign in

REVIEW 11 cited by

Multiagent Finetuning: Self Improvement with Diverse Reasoning Chains

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2501.05707 v2 pith:KX6HCDZL submitted 2025-01-10 cs.CL cs.AIcs.LG

Vighnesh Subramaniam , Yilun Du , Joshua B. Tenenbaum , Antonio Torralba , Shuang Li , Igor Mordatch This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords modelsdataself-improvementapproachlanguagemultiagentreasoningtraining
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large language models (LLMs) have achieved remarkable performance in recent years but are fundamentally limited by the underlying training data. To improve models beyond the training data, recent works have explored how LLMs can be used to generate synthetic data for autonomous self-improvement. However, successive steps of self-improvement can reach a point of diminishing returns. In this work, we propose a complementary approach towards self-improvement where finetuning is applied to a multiagent society of language models. A group of language models, all starting from the same base model, are independently specialized by updating each one using data generated through multiagent interactions among the models. By training each model on independent sets of data, we illustrate how this approach enables specialization across models and diversification over the set of models. As a result, our overall system is able to preserve diverse reasoning chains and autonomously improve over many more rounds of fine-tuning than single-agent self-improvement methods. We quantitatively illustrate the efficacy of the approach across a wide suite of reasoning tasks.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Models

    cs.LG 2025-06 conditional novelty 7.0 of 10

    Online self-play between attacker and defender roles of a single LLM improves safety robustness and attack diversity across Llama and Qwen models.

  2. Learning Decentralized LLM Collaboration with Multi-Agent Actor Critic

    cs.AI 2026-01 unverdicted novelty 6.0 of 10

    Multi-agent actor-critic methods with a centralized critic improve decentralized LLM collaboration over Monte Carlo baselines in long-horizon and sparse-reward settings.

  3. MaskSearch: A Universal Pre-Training Framework to Enhance Agentic Search Capability

    cs.CL 2025-05 conditional novelty 6.0 of 10

    A pre-training task called RAMP, where models practice searching to fill masked text spans, improves downstream agentic open-domain QA performance across Qwen and LLaMA models.

  4. When One LLM Drools, Multi-LLM Collaboration Rules

    cs.CL 2025-02 conditional novelty 6.0 of 10

    A position paper that introduces a four-level taxonomy of multi-LLM collaboration (API, text, logit, weight) and argues it is essential for reliability, pluralism, and democratization.

  5. Vision-Language Model Dialog Games for Self-Improvement

    cs.LG 2025-02 conditional novelty 6.0 of 10

    Self-play dialog games between two VLMs generate filtered synthetic data that, when fine-tuned on, improves VQA and robotics success detection.

  6. OSC: Cognitive Orchestration through Dynamic Knowledge Alignment in Multi-Agent LLM Collaboration

    cs.AI 2025-09 reject novelty 5.0 of 10

    OSC uses learned Collaborator Knowledge Models and RL-trained communication policies to make LLM agents communicate adaptively, claiming gains on AlpacaEval 2.0 and MT-Bench.

  7. OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automation

    cs.AI 2025-05 conditional novelty 5.0 of 10

    A hierarchical multi-agent framework with a domain-agnostic planner trained by SFT and DPO achieves 69.70% on GAIA and lifts a Qwen2.5-32B planner by 16.37 points.

  8. The Avengers: A Simple Recipe for Uniting Smaller Language Models to Challenge Proprietary Giants

    cs.CL 2025-05 reject novelty 5.0 of 10

    Clustering-based routing plus self-consistency voting among ten 7B open models reportedly outranks GPT-4.1 and GPT-4.5 on average over 15 diverse benchmarks.

  9. SWIRL: A Staged Workflow for Interleaved Reinforcement Learning in Mobile GUI Control

    cs.AI 2025-08 conditional novelty 4.0 of 10

    A multi-agent RL workflow that interleaves single-agent updates, applied to mobile GUI control, achieves SOTA zero-shot performance and a +14.8 MATH500 gain.

  10. Language Games as the Pathway to Artificial Superhuman Intelligence

    cs.AI 2025-01 conditional novelty 4.0 of 10

    A position paper arguing that open-ended language games with fluid roles, varied rewards, and evolving rules can drive expanded data reproduction and thus a path to artificial superhuman intelligence.

  11. Grounding Natural Language for Multi-agent Decision-Making with Multi-agentic LLMs

    cs.AI 2025-08 unverdicted novelty 3.0 of 10

    A design framework for multi-agent LLMs combining prompting, memory, multimodal input, and fine-tuning, with promised ablations on social-dilemma games that are absent from the supplied text.

Pith tools