AlphaAgents: Large Language Model based Multi-Agents for Equity Portfolio Constructions

Dhagash Mehta; Harrison Garber; Jingrao Lyu; Stefano Pasquali; Stokes Jones; Tianjiao Zhao

REVIEW 3 major objections 3 minor 8 cited by

AlphaAgents: Large Language Model based Multi-Agents for Equity Portfolio Constructions

T0 review · 3 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Role-based LLM agent teams can pick stocks that beat established benchmarks at varying risk tolerance levels.

desk verdict You can't evaluate this paper from the abstract, and the contamination worry is the one that matters. read the letter →

arxiv 2508.11152 v1 pith:CRHHFOH3 submitted 2025-08-15 q-fin.ST cs.AI

Tianjiao Zhao , Jingrao Lyu , Stokes Jones , Harrison Garber , Stefano Pasquali , Dhagash Mehta This is my paper

classification q-fin.STcs.AI

keywords multi-agentsystemslargelanguagemodelsequityportfolioconstructionstockselectionbenchmarkcomparisonrisktoleranceLLMagentsmanagement

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a multi-agent system built from large language models can handle equity research and stock selection, constructing portfolios that outperform established benchmarks across different risk tolerance levels. The goal is to show that role-based LLM agents, each handling a specialized part of the investment process, are practically useful for portfolio construction. If the claim holds, asset managers and individual investors could use such agent teams as a systematic stock-picking tool.

What carries the argument

The central object is the role-based multi-agent system: a set of large language model agents, each with a specialized role in equity research and portfolio management, that collaborate to produce stock selections. The mechanism is the division of labor and collaboration among agents, where each agent contributes analysis or recommendations that are then combined into portfolio decisions. The system is evaluated against established benchmarks at several levels of risk tolerance.

What would settle it

Run the identical multi-agent system on market data from a period after the LLM's training cutoff, using paper trading, and compare its portfolio returns against the same benchmarks; the central claim would be refuted if the outperformance disappears or turns negative out of sample.

Watch

Extended reading notes

Core claim

The central claim is that a team of specialized, role-based LLM agents can select stocks and build equity portfolios that outperform established benchmarks under varying levels of risk tolerance. The study assigns distinct roles to different agents, has them collaborate on equity analysis, and aggregates their outputs into portfolio decisions. The paper reports that this multi-agent system performed favorably against the benchmarks and discusses the practical advantages and implementation challenges of using multi-agent frameworks in equity analysis.

Load-bearing premise

The evaluation setup is a fair and unbiased test: the historical data, benchmark choices, and agent design were not selected to flatter the multi-agent system, and no information about future returns leaked into the agents' selections.

Editorial extensions

If this is right

Multi-agent LLM systems become a credible, benchmark-aware method for constructing equity portfolios automatically.
The role-based division of labor among agents can be reused as a template for other financial analysis workflows.
Risk tolerance can be dialed into the same agent team without redesigning the system.
The benchmark-comparison framework offers a starting point for evaluating future LLM-based investment agents.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The reported edge may shrink once transaction costs, slippage, and market impact are included, since the abstract does not say whether the backtest accounts for them.
The specific LLM and prompt design likely drive much of the result; swapping models or roles would be a natural test of whether the multi-agent structure itself is the source of performance.
If the LLM's training data includes the backtest period, the stock selections could reflect memorized outcomes rather than genuine forecasting; testing on post-cutoff data would resolve this.
The approach could extend beyond equities to other asset classes, but fixed costs and liquidity differences would require separate empirical validation.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: the abstract is all there is, and it's too thin to judge. The central claim—that role-based LLM agents beat benchmarks in stock picking—is plausible but not verifiable from what's shown. The one thing you should know is that the validity of that claim depends entirely on whether the backtest is truly out-of-sample for the LLMs. The abstract doesn't say, so the result is unverifiable.

What's actually new: a multi-agent team of LLM roles (researcher, risk officer, etc.) selecting stocks. That's a reasonable extension of known multi-agent debate and memory methods. The paper's potential contribution is in the evaluation design, not the architecture. The abstract also mentions 'limitations' of the framework, which is a good sign of honest intent.

Where the soft spots are: they're everywhere, but in proportion to the information available. There are no data, no error bars, no benchmark names, no evaluation window, no LLM version or knowledge cutoff. The stress-test concern about training-data contamination is on point. For an LLM, the default hypothesis is that it has seen historical stock prices and news up to its cutoff. If the test period overlaps that cutoff, the agents can recall what happened, and 'outperformance' is just memorization. The abstract doesn't rule this out, and the problem is load-bearing—it zeroes out the whole result.

Also missing: transaction costs, risk adjustment, and a comparison to simple models like a momentum or value factor. Without those, beating a broad index is weak evidence. The phrase 'under varying risk tolerance' suggests they construct portfolios, but we don't know if the risk model is meaningful or just cosmetic.

Is it a serious thinker? The abstract reads like a standard system paper, no internal contradictions. But there's too little substance to tell if the reasoning holds. I'd want to see the full text before giving credit.

Verdict: this is a desk reject as submitted. The abstract gives no reason to believe the performance claim is real. If the full paper provides a proper out-of-sample evaluation with the cutoffs disclosed and compares against standard baselines, it becomes a viable application paper worth a referee. But as it stands, you'd be refereeing a marketing summary.

Referee Report

3 major / 3 minor

Summary. The paper proposes 'AlphaAgents,' a role-based multi-agent LLM system for stock selection and equity portfolio construction. The abstract claims that a team of specialized agents can analyze equities and outperform established benchmarks under varying risk tolerance, and that the study provides insights into the practical efficacy of multi-agent frameworks. However, the abstract presents no quantitative results, no evaluation window, no benchmark definitions, no model versions, and no methodology. The full text is not available; this review is abstract-only.

Significance. If the performance claims were substantiated, the work would be significant: it would provide evidence that multi-agent LLM systems can add value in quantitative finance, an area where LLM adoption is growing but rigorous validation is scarce. The multi-agent role specialization idea is interesting and plausible. However, the current abstract provides no verifiable evidence, no reproducibility artifacts, and no falsifiable predictions. The significance of the claimed result is thus entirely conditional on unpublished evaluation details. There is no machine-checked proof, code, or data to assess, so the paper's contribution cannot currently be evaluated.

major comments (3)

[Abstract] The central claim—that the multi-agent system outperforms established benchmarks—is stated without any supporting data. There are no numerical results, no performance metrics, no benchmark list, and no evaluation period. As written, the claim is unverifiable. For a portfolio-construction paper, the abstract should at least report the main comparison (e.g., Sharpe ratio, alpha, or return) and the test window; otherwise the stated conclusion is unsupported.
[Abstract] The evaluation setup is not described, so a load-bearing contamination risk remains. LLM agents may have been pretrained on data that includes the evaluation period, making 'predictions' reflect memorization rather than reasoning. The abstract does not disclose the LLM versions, their knowledge cutoffs, or whether the evaluation window postdates those cutoffs. This is not a peripheral detail: if contamination occurred, any reported outperformance is artifactual. The authors must disclose the evaluation period, model versions, and a contamination-mitigation strategy.
[Abstract] The claim of practical efficacy for portfolio construction is incomplete without accounting for real-world frictions. The abstract mentions 'varying levels of risk tolerance' but gives no indication of whether the backtest includes transaction costs, liquidity constraints, or survivorship-bias controls. These factors are first-order for equity portfolio construction and can easily reverse an apparent outperformance. The absence of any such detail in the abstract makes the practical-effectiveness claim premature.

minor comments (3)

[Abstract] The phrase 'varying levels of risk tolerance' is vague; it is unclear whether this refers to different utility functions, portfolio constraints, or benchmark comparisons. A concrete specification would help readers understand the scope.
[Abstract] The abstract does not cite any prior work on LLM-based trading agents or multi-agent finance, making it difficult to place the claimed contribution in context. At least representative references should be added in the full text.
[Abstract] The abstract says 'We present a comprehensive analysis' but no analysis details are shown. Minor wording such as 'preliminary findings' might be more appropriate until the evaluation is disclosed, though this is a presentation issue relative to the major concerns.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity detected; abstract-only lacks equations or citations.

full rationale

The paper is represented only by an abstract. It claims that a team of role-based LLM agents performs stock selection and that their performance is evaluated against established benchmarks, but no derivation, equations, fitted parameters, or cited prior work are presented. There is therefore no load-bearing step that reduces to its own inputs, no self-citation, and no fitted-input-called-prediction. The training-data contamination concern raised in the review is an external validity threat, not a circularity of the paper's internal reasoning. Since no specific reduction can be quoted from the manuscript, the circularity score is 0.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The abstract does not reveal any explicit free parameters or invented entities. The main assumptions are that LLM agents can meaningfully analyze financial information and that the backtest setup is unbiased.

assumptions (2)

domain assumption Large language models can produce valid financial analysis and stock-picking decisions when organized in a role-based multi-agent framework.
This is the core assumption of the study. It is not proven by the abstract; it is the hypothesis being tested.
domain assumption The historical market data and benchmarks used in the evaluation are representative and sufficient to assess real-world portfolio performance.
The abstract does not specify the data source or evaluation period. Any backtest result depends on this assumption, which is unverifiable without full text.

how reviews work

0 comments

Cite this review

Pith. "Pith review of AlphaAgents: Large Language Model based Multi-Agents for Equity Portfolio Constructions." pith.science (2026). https://pith.science/paper/CRHHFOH3

@misc{pith2026250811152,
  author       = {Pith},
  title        = {Pith review of: AlphaAgents: Large Language Model based Multi-Agents for Equity Portfolio Constructions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CRHHFOH3}},
  note         = {Machine review of arXiv:2508.11152}
}

read the original abstract

The field of artificial intelligence (AI) agents is evolving rapidly, driven by the capabilities of Large Language Models (LLMs) to autonomously perform and refine tasks with human-like efficiency and adaptability. In this context, multi-agent collaboration has emerged as a promising approach, enabling multiple AI agents to work together to solve complex challenges. This study investigates the application of role-based multi-agent systems to support stock selection in equity research and portfolio management. We present a comprehensive analysis performed by a team of specialized agents and evaluate their stock-picking performance against established benchmarks under varying levels of risk tolerance. Furthermore, we examine the advantages and limitations of employing multi-agent frameworks in equity analysis, offering critical insights into their practical efficacy and implementation challenges.

Discussion (0). Sign in to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Eco3S: Complex Socio-Economic System Simulation via Agent-Based Models
cs.AI 2026-07 conditional novelty 6.0 of 10

Eco3S packages co-evolving environments, checkpoint counterfactuals, and auto-refinement into one LLM agent-based simulation platform, demonstrated on canal-rebellion, state-formation, and information-spread cases.
Signal or Noise in Multi-Agent LLM-based Stock Recommendations?
q-fin.PM 2026-04 unverdicted novelty 6.0 of 10

A multi-agent LLM equity system produces statistically significant outperformance on S&P 500 stocks, with strong-buy portfolios returning +2.18% monthly versus +1.15% for the equal-weight benchmark over 19 months.
Generation of Ultrabrilliant Positron Beam via Superponderomotive Injection in Laser Wakefield Acceleration
physics.plasm-ph 2025-08 unverdicted novelty 6.0 of 10

A superponderomotive injection scheme traps positrons in a laser-driven wakefield, producing quasi-monoenergetic beams with six-dimensional brightness near 10^15 A/m^2/0.1%, according to the abstract's PIC simulations.
AgentSteerTTS: A Multi-Agent Closed-Loop Framework for Composite-Instruction Text-to-Speech
cs.CV 2026-05 unverdicted novelty 5.0 of 10

AgentSteerTTS proposes a multi-agent framework with adversarial disentanglement, dual-stream anchoring via acoustic prototypes, and fast-slow feedback to achieve intent-faithful expressive TTS for composite instructions.
Macro Economists in the Machine: A Multi-Agent LLM Framework for Commodity-Related ETF Portfolio Construction
q-fin.PM 2026-06 conditional novelty 4.0 of 10

LLM agents (hawkish, dovish, debate) outperform a deterministic z-score rule agent in Sharpe ratio for commodity ETF portfolios by 0.04-0.044, with advantage concentrated in the soft-landing sub-period and preserved u...
MadEvolve: Evolutionary Optimization of Trading Systems with Large Language Models
q-fin.TR 2026-05 unverdicted novelty 4.0 of 10

MadEvolve uses LLMs for evolutionary optimization of trading strategies and reports significant backtest improvements on Bitcoin tasks including signal feature evolution and joint strategy optimization.
Beyond Agent Architecture: Execution Assumptions and Reproducibility in LLM-Based Trading Systems
cs.AI 2026-06 unverdicted novelty 3.0 of 10

Reproducibility audit of 30 LLM trading papers shows execution assumptions under-reported relative to agent architectures, illustrated by a 10-equity example where frictions compress returns.
A Review of Large Language Models for Stock Price Forecasting from a Hedge-Fund Perspective
q-fin.PR 2026-04 unverdicted novelty 3.0 of 10

This review synthesizes LLM uses in stock forecasting and catalogs key practical pitfalls from a hedge-fund viewpoint.

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.