REVIEW 16 cited by
Multi-Agent Verification: Scaling Test-Time Compute with Multiple Verifiers
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Multi-Agent Verification: Scaling Test-Time Compute with Multiple Verifiers
read the original abstract
By utilizing more computational resources at test-time, large language models (LLMs) can improve without additional training. One common strategy uses verifiers to evaluate candidate outputs. In this work, we propose a novel scaling dimension for test-time compute: scaling the number of verifiers. We introduce Multi-Agent Verification (MAV) as a test-time compute paradigm that combines multiple verifiers to improve performance. We propose using Aspect Verifiers (AVs), off-the-shelf LLMs prompted to verify different aspects of outputs, as one possible choice for the verifiers in a MAV system. AVs are a convenient building block for MAV since they can be easily combined without additional training. Moreover, we introduce BoN-MAV, a simple multi-agent verification algorithm that combines best-of-n sampling with multiple verifiers. BoN-MAV demonstrates stronger scaling patterns than self-consistency and reward model verification, and we demonstrate both weak-to-strong generalization, where combining weak verifiers improves even stronger LLMs, and self-improvement, where the same base model is used to both generate and verify outputs. Our results establish scaling the number of verifiers as a promising new dimension for improving language model performance at test-time.
Forward citations
Cited by 16 Pith papers
-
Cherry-pick Override: Unsafe Directional Commitment in LLM Judges under Mixed Evidence
The paper defines Cherry-pick Override (CCO) as unauthorized directional commitment by LLM judges under mixed evidence and quantifies its prevalence (>84% on AVeriTeC conflicting subset) while testing intervention lad...
-
IdleSpec: Exploiting Idle Time via Speculative Planning for LLM Agents
IdleSpec improves LLM agent accuracy by generating and aggregating speculative plans during idle time between tool calls and observations using complementary drafting strategies.
-
Refute-or-Promote: An Adversarial Stage-Gated Multi-Agent Review Methodology for High-Precision LLM-Assisted Defect Discovery
Refute-or-Promote applies adversarial multi-agent review with kill gates and empirical verification to filter LLM defect candidates, killing 79-83% before disclosure and yielding 4 CVEs plus multiple accepted fixes ac...
-
World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry
WAV self-improves action-conditioned world models by cycle-consistent verification of state plausibility and sparse action reachability, doubling sample efficiency and lifting policy reward by over 22% on nine tasks.
-
Oracle Gap and Signal Fidelity: A Fixed-Pool Diagnostic for Test-Time Collaboration
Training-free LLM collaboration gains are bounded by the fixed pool's oracle gap and then by signal coverage, fidelity, and harm, measurable with a small labeled audit.
-
Economy of Minds: Emerging Multi-Agent Intelligence with Economic Interactions
An economy of agents using auctions and wealth accumulation produces emergent multi-step reasoning that outperforms monolithic baselines on five agentic tasks.
-
Efficient Agentic Reasoning Through Self-Regulated Simulative Planning
SR²AM achieves competitive Pass@1 accuracy on diverse tasks with 25.8-95.3% fewer reasoning tokens than much larger models by using self-regulated simulative planning trained via supervised learning and RL.
-
FUSE: Ensembling Verifiers with Zero Labeled Data
FUSE ensembles verifiers unsupervisedly by controlling their conditional dependencies to improve spectral ensembling algorithms, matching or exceeding semi-supervised baselines on benchmarks including GPQA Diamond and...
-
Complementing Self-Consistency with Cross-Model Disagreement for Uncertainty Quantification
Cross-model semantic disagreement adds an epistemic uncertainty term that improves total uncertainty estimation over self-consistency alone, helping flag confident errors in LLMs.
-
When and How Much to Imagine: Adaptive Test-Time Scaling with World Models for Visual Spatial Reasoning
Adaptively gating and scaling world-model imagination at test time matches or outperforms always-on imagination on spatial reasoning benchmarks while using substantially fewer world-model calls and tokens.
-
Learning Decentralized LLM Collaboration with Multi-Agent Actor Critic
Multi-agent actor-critic methods with a centralized critic improve decentralized LLM collaboration over Monte Carlo baselines in long-horizon and sparse-reward settings.
-
EVE: A Generator-Verifier System for Generative Policies
Zero-shot VLM verifiers, ensembled and fused via guided diffusion, improve frozen generative robot policies' success rates by 1-2 percentage points on simulated manipulation tasks.
-
On the Generalization Gap in Self-Evolving Language Model Reasoning
Closed-loop self-evolution on LLMs improves reasoning on Knights and Knaves tasks but plateaus short of oracle-supervised levels, with multi-turn revision nearly matching it for large models.
-
Cut Your Losses! Learning to Prune Paths Early for Efficient Parallel Reasoning
STOP is a new learnable internal path-pruning technique that improves efficiency and accuracy of parallel reasoning in LRMs under fixed compute budgets.
-
Cut Your Losses! Learning to Prune Paths Early for Efficient Parallel Reasoning
STOP, a learnable internal super-token pruner, is claimed to raise parallel-reasoning accuracy and efficiency on LRMs from 1.5B to 20B, e.g. AIME25 from 84% to ~90% under fixed compute.
-
Learning Decentralized LLM Collaboration with Multi-Agent Actor Critic
Centralized-critic actor-critic training (CoLLM-CC) improves sample efficiency and stability over Monte-Carlo multi-agent RL for training decentralized LLM collaboration.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.