Paper: arXiv 2603.17692

Authors: Joohyoung Jeon, Hongchul Lee

Abstract

For LLM trading agents to be genuinely trustworthy, they must demonstrate understanding of market dynamics rather than exploitation of memorized ticker associations. Building responsible multi-agent systems demands rigorous signal validation: proving that predictions reflect legitimate patterns, not pre-trained recall. We address two sources of spurious performance: memorization bias from ticker-specific pre-training, and survivorship bias from flawed backtesting. Our approach is to blindfold the agents–anonymizing all identifiers–and verify whether meaningful signals persist. BlindTrade anonymizes tickers and company names, and four LLM agents output scores along with reasoning. We construct a GNN graph from reasoning embeddings and trade using PPO-DSR policy. On 2025 YTD (through 2025-08-01), we achieved Sharpe 1.40 +/- 0.22 across 20 seeds and validated signal legitimacy through negative control experiments. To assess robustness beyond a single OOS window, we additionally evaluate an extended period (2024–2025), revealing market-regime dependency: the policy excels in volatile conditions but shows reduced alpha in trending bull markets.

Complexity vs Empirical Score

  • Math Complexity: 5.5/10
  • Empirical Rigor: 7.5/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper presents an innovative anonymization-first methodology to address LLM memorization and look-ahead biases, combining text reasoning embeddings into GNNs with PPO-DSR reinforcement learning. It features solid empirical evaluation using backtests across multiple market regimes and includes crucial negative control experiments.

Research Flowchart

  flowchart TD
    A[Research Goal: Validate LLM Trading Signals <br>Without Memorization or Survivorship Bias] --> B[Data Prep: Anonymize Tickers & Company Names <br>2024-2025 Market Data]
    B --> C[4 LLM Agents Output <br>Predictive Scores & Natural Language Reasoning]
    C --> D[Construct GNN Graph from Reasoning Embeddings <br>& Train PPO-DSR Trading Policy]
    D --> E[Outcome: Sharpe 1.40 ± 0.22 on 2025 YTD <br>High Volatility Alpha vs. Regime Sensitivity]

Cited References (27)

  1. RockAlpha Arena : Real-Time LLM Trading Performance Leaderboard (2025) — link
  2. Your AI , Not Your View: The Bias of LLMs in Investment Analysis (2025) — link
  3. FinGPT : Open-Source Financial Large Language Models (2023) — link
  4. FinMem : A Performance-Enhanced LLM Trading Agent with Layered Memory and Character Design (2023) — link
  5. TradingAgents : Multi-Agent LLM Financial Trading Framework (2024) — link
  6. LiveTradeBench : Seeking Real-World Alpha with Large Language Models (2025) — link
  7. AI-Trader : Benchmarking Autonomous Agents in Real-Time Financial Markets (2025) — link
  8. A Distributional Perspective on Reinforcement Learning (2017) — link
and 19 more
  1. FinBERT : Financial Sentiment Analysis with Pre-trained Language Models (2019) — link
  2. BloombergGPT : A Large Language Model for Finance (2023) — link
  3. Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models (2023) — link
  4. The New Quant: A Survey of Large Language Models in Financial Prediction and Trading (2025) — link
  5. Assessing Look-Ahead Bias in Stock Return Predictions Generated by GPT Sentiment Analysis — → 2309.17322
  6. Survivorship Bias and Mutual Fund Performance (1996) — link
  7. Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance (2014) — link
  8. A comprehensive survey on deep neural networks for stock market: The need, challenges, and future directions (2021) — link
  9. Temporal Relational Ranking for Stock Prediction (2019) — link
  10. Graph Attention Networks (2018) — link
  11. How Attentive are Graph Attention Networks? (2022) — link
  12. Do Industries Explain Momentum? (1999) — link
  13. Sentence- BERT : Sentence Embeddings using Siamese BERT -Networks (2019) — link
  14. A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem (2017) — link
  15. Deep Reinforcement Learning for Trading (2020) — link
  16. Learning to Trade via Direct Reinforcement (2001) — link
  17. Does the Stock Market Overreact? (1985) — link
  18. Returns to Buying Winners and Selling Losers: Implications for Stock Market Efficiency (1993) — link
  19. EODHD Financial Data API : Historical Data, Fundamentals, and News (2024) — link

1 of 27 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.