Paper: arXiv 2502.17967

Authors: Tianmi Ma, Jiawei Du, Wenxin Huang, Wenjie Wang, Liang Xie, Xian Zhong, Joey Tianyi Zhou

Abstract

Large language models (LLMs) have demonstrated remarkable capabilities in natural language tasks, yet their performance in dynamic, real-world financial environments remains underexplored. Existing approaches are limited to historical backtesting, where trading actions cannot influence market prices and agents train only on static data. To address this limitation, we present the Agent Trading Arena, a virtual zero-sum stock market in which LLM-based agents engage in competitive multi-agent trading and directly impact price dynamics. By simulating realistic bid-ask interactions, our platform enables training in scenarios that closely mirror live markets, thereby narrowing the gap between training and evaluation. Experiments reveal that LLMs struggle with numerical reasoning when given plain-text data, often overfitting to local patterns and recent values. In contrast, chart-based visualizations significantly enhance both numerical reasoning and trading performance. Furthermore, incorporating a reflection module yields additional improvements, especially with visual inputs. Evaluations on NASDAQ and CSI datasets demonstrate the superiority of our method, particularly under high volatility. All code and data are available at https://github.com/wekjsdvnm/Agent-Trading-Arena.

Complexity vs Empirical Score

  • Math Complexity: 4.0/10
  • Empirical Rigor: 8.5/10
  • Quadrant: Street Traders — empirical and applied, lower math complexity

Why this score: The paper focuses on an experimental framework (Agent Trading Arena) with extensive empirical evaluation on real datasets (NASDAQ/CSI), providing code, and reporting performance metrics, indicating high empirical rigor. The mathematics is relatively low in complexity, primarily involving market microstructure simulation logic and basic reasoning patterns rather than advanced derivations or dense formulas.

Research Flowchart

  flowchart TD
  A["Research Goal<br>Understand & improve numerical reasoning<br>of LLM-based trading agents"] --> B["Agent Trading Arena<br>Virtual zero-sum market<br>Agents impact price dynamics"]
  B --> C["Inputs & Setup<br>Dataset: NASDAQ & CSI stocks<br>Inputs: Plain-text vs. Chart-based data<br>Agents: Base LLM & + Reflection Module"]
  C --> D["Core Process<br>Competitive Multi-Agent Trading<br>Reinforcement Learning & Simulation"]
  D --> E["Key Findings & Outcomes"]
  E --> F["1. Visual Input Superiority<br>Charts > Plain-text for numerical reasoning"]
  E --> G["2. Reflection Enhances Gains<br>Reflection module improves performance, especially with charts"]
  E --> H["3. Robustness in Volatility<br>Method excels in high-volatility markets"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes the Agent Trading Arena, a virtual zero‑sum stock market where LLM‑based agents trade competitively and directly affect price dynamics.

Source sentence
we present the Agent Trading Arena, a virtual zero-sum stock market in which LLM-based agents engage in competitive multi-agent trading and directly impact price dynamics.

Assumes 6

  • The environment is a closed‑loop system that isolates external knowledge and avoids reliance on outside information. other
    Source sentence
    The Agent Trading Arena is a closed-loop virtual economy that isolates external knowledge and avoids reliance…
  • Asset prices are generated exclusively through a bid‑ask process with no external benchmarks. market
    Source sentence
    Asset prices arise exclusively from a bid-ask process: each agent’s orders directly influence the market price, with no external benchmarks.
  • Agents receive a daily wealth fee proportional to their total wealth, encouraging active trading. execution
    Source sentence
    Daily Wealth Fee: A cost proportional to each agent’s total wealth is charged daily.
  • Dividends and capital gains are paid proportionally to holdings, anchoring prices and rewarding low‑cost positions. market
    Source sentence
    Dividends & Capital Gains: Agents earn income from both price appreciation and dividends, distributed proportionally to holdings.
Show 2 more
  • Simulations exclude weekends and holidays, using only trading days. data
    Source sentence
    excluding weekends and holidays.
  • The market friction and slippage are modeled via the bid‑ask order matching mechanism. execution
    Source sentence
    By simulating bid-ask order matching in real time, the Arena faithfully reproduces liquidity dynamics and price impact, enabling agents to experience authentic market friction and slippage.

Data

Backtest sample: NASDAQ and CSI · equities · daily

  • NASDAQ · Yahoo Finance and Baostock market-prices public
    September 3–October 29, 2024
    Source sentence
    For NASDAQ, we run a two-month simulation (September 3–October 29, 2024) with an initial capital of 100,000 units, excluding weekends and holidays.
  • CSI · Yahoo Finance and Baostock market-prices public
    January 2–December 31, 2024
    Source sentence
    For CSI, we conduct a one-year simulation (January 2–December 31, 2024) with the same capital, also excluding non-trading days.

Recognised sources: BaoStock Yahoo Finance

Costs

Modeled Costs are part of the reported headline results.

A daily wealth fee proportional to each agent’s total wealth is charged daily.

Source sentence
Daily Wealth Fee: A cost proportional to each agent’s total wealth is charged daily.

Finds 5

Reported performance (ArenaTrader w/ GPT-4o): Sharpe 0.35 (benchmark 0.19) · total return 16.0%

  • The proposed method achieves a Sharpe Ratio of 0.348 on NASDAQ‑100 (an improvement of 0.159) and 0.123 on CSI‑300 (an improvement of 0.069). performance
    Source sentence
    our model achieves 0.348 on NASDAQ-100 (an improvement of 0.159) and 0.123 on CSI 300 (an improvement of 0.069).
  • Agents using both textual and visual inputs with a 10‑day window obtain the highest return (TR 15.99%) and Sharpe Ratio (0.348). performance
    Source sentence
    Textual + Visual consistently achieves the best performance, particularly with a 10-day window
  • LLMs struggle with numerical reasoning when given plain‑text data, often overfitting to local patterns and recent values. negative
    Source sentence
    LLMs struggle with numerical reasoning when given plain-text data, often overfitting to local patterns and recent values.
  • Chart‑based visualizations significantly enhance both numerical reasoning and trading performance compared to plain‑text inputs. performance
    Source sentence
    In contrast, chart-based visualizations significantly enhance both numerical reasoning and trading performance.
  • Incorporating a reflection module yields additional improvements, especially when visual inputs are used. performance
    Source sentence
    incorporating a reflection module yields additional improvements, especially with visual inputs.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 26 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 2 papers in the archive (2 by independent authors)

Code

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (60)

  1. ChatDev: Communicative Agents for Software Development (2024)
  2. MemoryBank: Enhancing Large Language Models with Long-Term Memory (2024)
  3. Generative Agents: Interactive Simulacra of Human Behavior (2023)
  4. Social simulacra: Creating populated prototypes for social computing systems (2022)
  5. AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors in Agents (2023) — link
  6. AgentSims: An Open-Source Sandbox for Large Language Model Evaluation (2023) — link
  7. Voyager: An Open-Ended Embodied Agent with Large Language Models (2024)
  8. Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Tasks (2024)
and 52 more
  1. Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory (2023) — link
  2. ChatDB: Augmenting LLMs with Databases as Their Symbolic Memory (2023) — link
  3. MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework (2024)
  4. When ai meets finance (stockagent): Large language model-based stock trading in simulated real-world environments — → 2407.18957
  5. TradingAgents : Multi-Agents LLM Financial Trading Framework — → 2412.20138
  6. C rypto T rade: A Reflective LLM -based Agent to Guide Zero-shot Cryptocurrency Trading (2024)
  7. TransLLaMa : LLM -based Simultaneous Translation System (2024)
  8. A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models (2024)
  9. Exploring the Limits of ChatGPT for Query or Aspect-based Text Summarization (2023) — link
  10. LLM-ARC: Enhancing LLMs with an Automated Reasoning Critic (2024) — link
  11. LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers (2023)
  12. PaLM 2 Technical Report (2023) — link
  13. GPT Can Solve Mathematical Problems Without a Calculator (2023) — link
  14. Building Cooperative Embodied Agents Modularly with Large Language Models (2024)
  15. Improving Factuality and Reasoning in Language Models through Multiagent Debate (2024)
  16. Self-Collaboration Code Generation via ChatGPT (2024)
  17. SocioDojo: Building Lifelong Analytical Agents with Real-world Text and Time Series (2024)
  18. The LLaMa 3 herd of models (2024) — link
  19. GPT-4o System Card (2024) — link
  20. GPT-3.5 model card (2023) — link
  21. Qwen2.5 Technical Report (2024) — link
  22. Qwen2 Technical Report (2024) — link
  23. GPT-4 Technical Report (2023) — link
  24. Palm 2 technical report (2023) — link
  25. Llama: Open and efficient foundation language models (2023) — link
  26. GROOT: Learning to Follow Instructions by Watching Gameplay Videos (2024)
  27. SportsMetrics: Blending Text and Numerical Data to Understand Information Fusion in LLMs (2024)
  28. Reflect, Not Reflex: Inference-Based Common Ground Improves Dialogue Response Quality (2022)
  29. Reflexion: language agents with verbal reinforcement learning (2023)
  30. A Survey of Deep Learning for Mathematical Reasoning (2023)
  31. Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models (2023)
  32. Reasoning with Language Model Prompting: A Survey (2023)
  33. Analyzing Temporal Complex Events with Large Language Models? A Benchmark towards Temporal, Long Context Understanding (2024)
  34. Qwen2-VL: Enhancing Vision-Language Model’s Perception of the World at Any Resolution (2024) — link
  35. Qwen Technical Report (2023) — link
  36. ChatGLM : A Family of Large Language Models from GLM-130B to GLM-4 All Tools (2024) — link
  37. DeepSeek-V3 Technical Report (2024) — link
  38. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context (2024) — link
  39. Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? (2024)
  40. Economics arena for large language models (2024) — link
  41. ChatEval: Towards Better LLM -based Evaluators through Multi-Agent Debate (2024)
  42. How Language Model Hallucinations Can Snowball (2024)
  43. Non-linear prediction of security returns with moving average rules (1996)
  44. Mean reversion in stock prices: Evidence and implications (1988)
  45. Technical analysis and the London Stock exchange: Testing the MACD and RSI rules using the FT30 (2008)
  46. StockFormer: Learning Hybrid Trading Machines with Predictive Coding (2023)
  47. TimesNet: Temporal 2D -Variation Modeling for General Time Series Analysis (2023)
  48. A Time Series is Worth 64 Words: Long-term Forecasting with Transformers (2023)
  49. PDFormer : Propagation delay-aware dynamic long-range transformer for traffic flow prediction (2023)
  50. iTransformer: Inverted Transformers Are Effective for Time Series Forecasting (2024)
  51. StockMixer : A Simple Yet Strong MLP-Based Architecture for Stock Price Forecasting (2024)
  52. StoryLLaVA: Enhancing Visual Storytelling with Multi-Modal Large Language Models (2025)

2 of 60 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.