Signal or Noise in Multi-Agent LLM-based Stock Recommendations?

Paper: arXiv 2604.17327

Authors: George Fatouros, Kostas Metaxas

Abstract

We present the first portfolio-level validation of MarketSenseAI, a deployed multi-agent LLM equity system. All signals are generated live at each observation date, eliminating look-ahead bias. The system routes four specialist agents (News, Fundamentals, Dynamics, and Macro) through a synthesis agent that issues a monthly equity thesis and recommendation for each stock in its coverage universe, and we ask two questions: do its buy recommendations add value over both passive benchmarks and random selection, and what does the internal agent structure reveal about the source of the edge? On the S&P 500 cohort (19 months) the strong-buy equal-weight portfolio earns +2.18%/month against a passive equal-weight benchmark of +1.15% (approximating RSP), a +25.2% compound excess, and ranks at the 99.7th percentile of 10,000 Monte Carlo portfolios (p=0.003). The S&P 100 cohort (35 months) delivers a +30.5% compound excess over EQWL with consistent direction but formal significance not reached, limited by the small average selection of ~10 stocks per month. Non-negative least-squares projection of thesis embeddings onto agent embeddings reveals an adaptive-integration mechanism. Agent contributions rotate with market regime (Fundamentals leads on S&P 500, Macro on S&P 100, Dynamics acts as an episodic momentum signal) and this agent rotation moves in lockstep with both the sector composition of strong-buy selections and identifiable macro-calendar events, three independent views of the same underlying adaptation. The recommendation’s cross-sectional Information Coefficient is statistically significant on S&P 500 (ICIR=+0.489, p=0.024). These results suggest that multi-agent LLM equity systems can identify sources of alpha beyond what classical factor models capture, and that the buy signal functions as an effective universe-filter that can sit upstream of any portfolio-construction process.

Complexity vs Empirical Score

• Math Complexity: 3.2/10
• Empirical Rigor: 6.0/10 (capped from 8.5: transaction costs not modelled; see penalty flags)
• Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper focuses on the practical validation of an LLM-based production pipeline using robust empirical techniques like Monte Carlo permutation testing, NNLS attribution, and Information Coefficient analysis, rather than derivation of novel quantitative theory. Its strength lies in the rigorous backtesting methodology and the handling of look-ahead bias in an operational, live-data environment.

Research Flowchart

  flowchart TD
    A[<b>Goal:</b> Validate Multi-Agent LLM Alpha Generation] --> B[<b>Input:</b> Live Data (No Look-ahead Bias)]
    B --> C{MarketSenseAI: Multi-Agent System}
    C --> D[Synthesis: Specialist Routing & Adaptive Integration]
    D --> E[<b>Analysis:</b> Portfolio Backtesting vs Benchmarks]
    E --> F[<b>Outcomes:</b> Significant Alpha & Adaptive Mechanism]
    F --> G[<b>Key Finding:</b> LLMs capture alpha beyond classical factors]

Cited References (39)

1. Can Large Language Models beat wall street? Evaluating GPT-4’s impact on financial decision-making with MarketSenseAI (2025)
2. Transforming sentiment analysis in the financial domain with ChatGPT (2023)
3. AI in investment analysis: LLMs for equity stock ratings (2024)
4. MarketSenseAI 2.0: Enhancing Stock Analysis Through LLM Agents (2025) — link
5. Vibe-Trading: Your Personal Trading Agent (2026)
6. Can LLM-based Financial Investing Strategies Outperform the Market in Long Run? — → 2505.07078
7. TradingAgents : Multi-Agents LLM Financial Trading Framework — → 2412.20138
8. Can ChatGPT Forecast Stock Price Movements? R eturn Predictability and Large Language Models — → 2304.07619

11 of 39 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.