Paper: arXiv 2502.17967
Authors: Tianmi Ma, Jiawei Du, Wenxin Huang, Wenjie Wang, Liang Xie, Xian Zhong, Joey Tianyi Zhou
Abstract
Large language models (LLMs) have demonstrated remarkable capabilities in natural language tasks, yet their performance in dynamic, real-world financial environments remains underexplored. Existing approaches are limited to historical backtesting, where trading actions cannot influence market prices and agents train only on static data. To address this limitation, we present the Agent Trading Arena, a virtual zero-sum stock market in which LLM-based agents engage in competitive multi-agent trading and directly impact price dynamics. By simulating realistic bid-ask interactions, our platform enables training in scenarios that closely mirror live markets, thereby narrowing the gap between training and evaluation. Experiments reveal that LLMs struggle with numerical reasoning when given plain-text data, often overfitting to local patterns and recent values. In contrast, chart-based visualizations significantly enhance both numerical reasoning and trading performance. Furthermore, incorporating a reflection module yields additional improvements, especially with visual inputs. Evaluations on NASDAQ and CSI datasets demonstrate the superiority of our method, particularly under high volatility. All code and data are available at https://github.com/wekjsdvnm/Agent-Trading-Arena.
Complexity vs Empirical Score
- Math Complexity: 4.0/10
- Empirical Rigor: 8.5/10
- Quadrant: Street Traders — empirical and applied, lower math complexity
Why this score: The paper focuses on an experimental framework (Agent Trading Arena) with extensive empirical evaluation on real datasets (NASDAQ/CSI), providing code, and reporting performance metrics, indicating high empirical rigor. The mathematics is relatively low in complexity, primarily involving market microstructure simulation logic and basic reasoning patterns rather than advanced derivations or dense formulas.
Research Flowchart
flowchart TD A["Research Goal<br>Understand & improve numerical reasoning<br>of LLM-based trading agents"] --> B["Agent Trading Arena<br>Virtual zero-sum market<br>Agents impact price dynamics"] B --> C["Inputs & Setup<br>Dataset: NASDAQ & CSI stocks<br>Inputs: Plain-text vs. Chart-based data<br>Agents: Base LLM & + Reflection Module"] C --> D["Core Process<br>Competitive Multi-Agent Trading<br>Reinforcement Learning & Simulation"] D --> E["Key Findings & Outcomes"] E --> F["1. Visual Input Superiority<br>Charts > Plain-text for numerical reasoning"] E --> G["2. Reflection Enhances Gains<br>Reflection module improves performance, especially with charts"] E --> H["3. Robustness in Volatility<br>Method excels in high-volatility markets"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes the Agent Trading Arena, a virtual zero‑sum stock market where LLM‑based agents trade competitively and directly affect price dynamics.
Source sentence
we present the Agent Trading Arena, a virtual zero-sum stock market in which LLM-based agents engage in competitive multi-agent trading and directly impact price dynamics.
Assumes 6
- The environment is a closed‑loop system that isolates external knowledge and avoids reliance on outside information. other
Source sentence
The Agent Trading Arena is a closed-loop virtual economy that isolates external knowledge and avoids reliance…
- Asset prices are generated exclusively through a bid‑ask process with no external benchmarks. market
Source sentence
Asset prices arise exclusively from a bid-ask process: each agent’s orders directly influence the market price, with no external benchmarks.
- Agents receive a daily wealth fee proportional to their total wealth, encouraging active trading. execution
Source sentence
Daily Wealth Fee: A cost proportional to each agent’s total wealth is charged daily.
- Dividends and capital gains are paid proportionally to holdings, anchoring prices and rewarding low‑cost positions. market
Source sentence
Dividends & Capital Gains: Agents earn income from both price appreciation and dividends, distributed proportionally to holdings.
Show 2 more
- Simulations exclude weekends and holidays, using only trading days. data
Source sentence
excluding weekends and holidays.
- The market friction and slippage are modeled via the bid‑ask order matching mechanism. execution
Source sentence
By simulating bid-ask order matching in real time, the Arena faithfully reproduces liquidity dynamics and price impact, enabling agents to experience authentic market friction and slippage.
Data
NASDAQ and CSI · equities · daily
- NASDAQ market-prices public
Source sentence
For NASDAQ, we run a two-month simulation (September 3–October 29, 2024) with an initial capital of 100,000 units, excluding weekends and holidays.
- CSI market-prices public
Source sentence
For CSI, we conduct a one-year simulation (January 2–December 31, 2024) with the same capital, also excluding non-trading days.
BaoStock Yahoo Finance
Costs
Modeled
A daily wealth fee proportional to each agent’s total wealth is charged daily.
Source sentence
Daily Wealth Fee: A cost proportional to each agent’s total wealth is charged daily.
Finds 5
Sharpe 0.35 (benchmark 0.19) · total return 16.0%
- The proposed method achieves a Sharpe Ratio of 0.348 on NASDAQ‑100 (an improvement of 0.159) and 0.123 on CSI‑300 (an improvement of 0.069). performance
Source sentence
our model achieves 0.348 on NASDAQ-100 (an improvement of 0.159) and 0.123 on CSI 300 (an improvement of 0.069).
- Agents using both textual and visual inputs with a 10‑day window obtain the highest return (TR 15.99%) and Sharpe Ratio (0.348). performance
Source sentence
Textual + Visual consistently achieves the best performance, particularly with a 10-day window
- LLMs struggle with numerical reasoning when given plain‑text data, often overfitting to local patterns and recent values. negative
Source sentence
LLMs struggle with numerical reasoning when given plain-text data, often overfitting to local patterns and recent values.
- Chart‑based visualizations significantly enhance both numerical reasoning and trading performance compared to plain‑text inputs. performance
Source sentence
In contrast, chart-based visualizations significantly enhance both numerical reasoning and trading performance.
- Incorporating a reflection module yields additional improvements, especially when visual inputs are used. performance
Source sentence
incorporating a reflection module yields additional improvements, especially with visual inputs.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 26 related papers). Coverage is partial, so this is neither replication nor refutation.
2 papers in the archive
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (60)
- ChatDev: Communicative Agents for Software Development (2024)
- MemoryBank: Enhancing Large Language Models with Long-Term Memory (2024)
- Generative Agents: Interactive Simulacra of Human Behavior (2023)
- Social simulacra: Creating populated prototypes for social computing systems (2022)
- AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors in Agents (2023) — link
- AgentSims: An Open-Source Sandbox for Large Language Model Evaluation (2023) — link
- Voyager: An Open-Ended Embodied Agent with Large Language Models (2024)
- Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Tasks (2024)
and 52 more
- Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory (2023) — link
- ChatDB: Augmenting LLMs with Databases as Their Symbolic Memory (2023) — link
- MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework (2024)
- When ai meets finance (stockagent): Large language model-based stock trading in simulated real-world environments — → 2407.18957
- TradingAgents : Multi-Agents LLM Financial Trading Framework — → 2412.20138
- C rypto T rade: A Reflective LLM -based Agent to Guide Zero-shot Cryptocurrency Trading (2024)
- TransLLaMa : LLM -based Simultaneous Translation System (2024)
- A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models (2024)
- Exploring the Limits of ChatGPT for Query or Aspect-based Text Summarization (2023) — link
- LLM-ARC: Enhancing LLMs with an Automated Reasoning Critic (2024) — link
- LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers (2023)
- PaLM 2 Technical Report (2023) — link
- GPT Can Solve Mathematical Problems Without a Calculator (2023) — link
- Building Cooperative Embodied Agents Modularly with Large Language Models (2024)
- Improving Factuality and Reasoning in Language Models through Multiagent Debate (2024)
- Self-Collaboration Code Generation via ChatGPT (2024)
- SocioDojo: Building Lifelong Analytical Agents with Real-world Text and Time Series (2024)
- The LLaMa 3 herd of models (2024) — link
- GPT-4o System Card (2024) — link
- GPT-3.5 model card (2023) — link
- Qwen2.5 Technical Report (2024) — link
- Qwen2 Technical Report (2024) — link
- GPT-4 Technical Report (2023) — link
- Palm 2 technical report (2023) — link
- Llama: Open and efficient foundation language models (2023) — link
- GROOT: Learning to Follow Instructions by Watching Gameplay Videos (2024)
- SportsMetrics: Blending Text and Numerical Data to Understand Information Fusion in LLMs (2024)
- Reflect, Not Reflex: Inference-Based Common Ground Improves Dialogue Response Quality (2022)
- Reflexion: language agents with verbal reinforcement learning (2023)
- A Survey of Deep Learning for Mathematical Reasoning (2023)
- Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models (2023)
- Reasoning with Language Model Prompting: A Survey (2023)
- Analyzing Temporal Complex Events with Large Language Models? A Benchmark towards Temporal, Long Context Understanding (2024)
- Qwen2-VL: Enhancing Vision-Language Model’s Perception of the World at Any Resolution (2024) — link
- Qwen Technical Report (2023) — link
- ChatGLM : A Family of Large Language Models from GLM-130B to GLM-4 All Tools (2024) — link
- DeepSeek-V3 Technical Report (2024) — link
- Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context (2024) — link
- Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? (2024)
- Economics arena for large language models (2024) — link
- ChatEval: Towards Better LLM -based Evaluators through Multi-Agent Debate (2024)
- How Language Model Hallucinations Can Snowball (2024)
- Non-linear prediction of security returns with moving average rules (1996)
- Mean reversion in stock prices: Evidence and implications (1988)
- Technical analysis and the London Stock exchange: Testing the MACD and RSI rules using the FT30 (2008)
- StockFormer: Learning Hybrid Trading Machines with Predictive Coding (2023)
- TimesNet: Temporal 2D -Variation Modeling for General Time Series Analysis (2023)
- A Time Series is Worth 64 Words: Long-term Forecasting with Transformers (2023)
- PDFormer : Propagation delay-aware dynamic long-range transformer for traffic flow prediction (2023)
- iTransformer: Inverted Transformers Are Effective for Time Series Forecasting (2024)
- StockMixer : A Simple Yet Strong MLP-Based Architecture for Stock Price Forecasting (2024)
- StoryLLaVA: Enhancing Visual Storytelling with Multi-Modal Large Language Models (2025)
2 of 60 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.