{"paper_id":"2502.17967","title":"Agent Trading Arena: A Study on Numerical Understanding in LLM-Based Agents","authors":["Tianmi Ma","Jiawei Du","Wenxin Huang","Wenjie Wang","Liang Xie","Xian Zhong","Joey Tianyi Zhou"],"paper_date":"2025-02-25","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2502.17967v2","page_url":"https://thequant.space/flowcharts/agent-trading-arena-a-study-on-numerical-understanding-in-l/","scores":{"empirical_rigor":8.5,"math_complexity":4,"quadrant":"Street Traders"},"proposes":{"status":"available","statement":"The paper proposes the Agent Trading Arena, a virtual zero‑sum stock market where LLM‑based agents trade competitively and directly affect price dynamics.","quote":"we present the Agent Trading Arena, a virtual zero-sum stock market in which LLM-based agents engage in competitive multi-agent trading and directly impact price dynamics.","summary":"Large language models (LLMs) have demonstrated remarkable capabilities in natural language tasks, yet their performance in dynamic, real-world financial environments remains underexplored. Existing approaches are limited to historical backtesting, where trading actions cannot influence market…","strategy":"ArenaTrader w/ GPT-4o"},"assumptions":{"status":"available","items":[{"assumption":"The environment is a closed‑loop system that isolates external knowledge and avoids reliance on outside information.","type":"other","stated":"explicit","quote":"The Agent Trading Arena is a closed-loop virtual economy that isolates external knowledge and avoids reliance…"},{"assumption":"Asset prices are generated exclusively through a bid‑ask process with no external benchmarks.","type":"market","stated":"explicit","quote":"Asset prices arise exclusively from a bid-ask process: each agent’s orders directly influence the market price, with no external benchmarks."},{"assumption":"Agents receive a daily wealth fee proportional to their total wealth, encouraging active trading.","type":"execution","stated":"explicit","quote":"Daily Wealth Fee: A cost proportional to each agent’s total wealth is charged daily."},{"assumption":"Dividends and capital gains are paid proportionally to holdings, anchoring prices and rewarding low‑cost positions.","type":"market","stated":"explicit","quote":"Dividends & Capital Gains: Agents earn income from both price appreciation and dividends, distributed proportionally to holdings."},{"assumption":"Simulations exclude weekends and holidays, using only trading days.","type":"data","stated":"explicit","quote":"excluding weekends and holidays."},{"assumption":"The market friction and slippage are modeled via the bid‑ask order matching mechanism.","type":"execution","stated":"implied","quote":"By simulating bid-ask order matching in real time, the Arena faithfully reproduces liquidity dynamics and price impact, enabling agents to experience authentic market friction and slippage."}]},"datasets":{"status":"available","items":[{"name":"NASDAQ","provider":"Yahoo Finance and Baostock","kind":"market-prices","access":"public","period":"September 3–October 29, 2024","universe":null,"quote":"For NASDAQ, we run a two-month simulation (September 3–October 29, 2024) with an initial capital of 100,000 units, excluding weekends and holidays.","known_id":"yahoo-finance"},{"name":"CSI","provider":"Yahoo Finance and Baostock","kind":"market-prices","access":"public","period":"January 2–December 31, 2024","universe":null,"quote":"For CSI, we conduct a one-year simulation (January 2–December 31, 2024) with the same capital, also excluding non-trading days.","known_id":"yahoo-finance"}],"known":[{"id":"baostock","name":"BaoStock","provider":"BaoStock","kind":"market-prices","access":"public","quote":"Both datasets, collected from Yahoo Finance and Baostock, pro-vide daily records of opening, closing, high, and low prices, trading volume, and associated tech-nical indicators, offering a holistic and up-to-date depiction of market behavior.","mentions":1,"used_in":1},{"id":"yahoo-finance","name":"Yahoo Finance","provider":"Yahoo","kind":"market-prices","access":"public","quote":"Both datasets, collected from Yahoo Finance and Baostock, pro-vide daily records of opening, closing, high, and low prices, trading volume, and associated tech-nical indicators, offering a holistic and up-to-date depiction of market behavior.","mentions":1,"used_in":1}],"sample":{"start":null,"end":null,"market":"NASDAQ and CSI","asset_class":"equities","frequency":"daily"},"data_resolution":"daily"},"transaction_costs":{"status":"available","modeled":"yes","headline_net_of_costs":null,"detail":"A daily wealth fee proportional to each agent’s total wealth is charged daily.","quote":"Daily Wealth Fee: A cost proportional to each agent’s total wealth is charged daily.","source":"evidence_card","rule":"R2","candidates":{"front_matter":"unclear","evidence_card":"yes","evidence_card_quoted":true},"card_detail":"A daily wealth fee proportional to each agent’s total wealth is charged daily.","card_quote":"Daily Wealth Fee: A cost proportional to each agent’s total wealth is charged daily.","meaning":"costs are part of the reported headline results"},"results":{"status":"available","evaluation_type":"trading-pnl","findings":[{"finding":"The proposed method achieves a Sharpe Ratio of 0.348 on NASDAQ‑100 (an improvement of 0.159) and 0.123 on CSI‑300 (an improvement of 0.069).","kind":"performance","quote":"our model achieves 0.348 on NASDAQ-100 (an improvement of 0.159) and 0.123 on CSI 300 (an improvement of 0.069)."},{"finding":"Agents using both textual and visual inputs with a 10‑day window obtain the highest return (TR 15.99%) and Sharpe Ratio (0.348).","kind":"performance","quote":"Textual + Visual consistently achieves the best performance, particularly with a 10-day window"},{"finding":"LLMs struggle with numerical reasoning when given plain‑text data, often overfitting to local patterns and recent values.","kind":"negative","quote":"LLMs struggle with numerical reasoning when given plain-text data, often overfitting to local patterns and recent values."},{"finding":"Chart‑based visualizations significantly enhance both numerical reasoning and trading performance compared to plain‑text inputs.","kind":"performance","quote":"In contrast, chart-based visualizations significantly enhance both numerical reasoning and trading performance."},{"finding":"Incorporating a reflection module yields additional improvements, especially when visual inputs are used.","kind":"performance","quote":"incorporating a reflection module yields additional improvements, especially with visual inputs."}],"reported_performance":{"strategy":"ArenaTrader w/ GPT-4o","sharpe":0.348,"sharpe_benchmark":0.189,"annual_return_pct":null,"total_return_pct":15.99,"max_drawdown_pct":null},"out_of_sample":{"method":"holdout","headline_out_of_sample":true}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":26,"citations":{"cited_by_in_corpus":2,"by_independent_authors":2,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":1,"unclassified":1},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":true,"code_url":"https://github.com/MTMQuantAI/Agent-Trading-Arena","note":null},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T17:44:29Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}