{"paper_id":"2512.10971","title":"AI-Trader: Benchmarking Autonomous Agents in Real-Time Financial Markets","authors":["Tianyu Fan","Yuhao Yang","Yangqin Jiang","Yifei Zhang","Yuxuan Chen","Chao Huang"],"paper_date":"2025-12-01","paper_type":"dataset-benchmark","paper_url":"https://arxiv.org/pdf/2512.10971v1","page_url":"https://thequant.space/flowcharts/ai-trader-benchmarking-autonomous-agents-in-real-time-finan/","scores":{"empirical_rigor":6,"math_complexity":3,"quadrant":"Street Traders"},"proposes":{"status":"available","statement":"The paper introduces AI‑Trader, a fully autonomous, live, data‑uncontaminated benchmark for evaluating LLM agents in real‑time financial markets across U.S. stocks, A‑shares and cryptocurrencies.","quote":"We propose AI‑Trader, a dynamic, real‑time, and data‑contamination‑free benchmark specifically designed for evaluating LLM agents in financial environments.","summary":"Large Language Models (LLMs) have demonstrated remarkable potential as autonomous agents, approaching human-expert performance through advanced reasoning and tool orchestration. However, decision-making in fully dynamic and live environments remains highly challenging, requiring real-time…","strategy":"MiniMax-M2"},"assumptions":{"status":"available","items":[{"assumption":"Agents operate with only essential context and must independently search, verify, and synthesize live market information without human intervention.","type":"execution","stated":"explicit","quote":"Our benchmark implements a revolutionary fully autonomous minimal information paradigm where agents receive only essential context and must independently search, verify, and synthesize live market information without human intervention."},{"assumption":"The evaluation environment is data‑uncontaminated, i.e., no pre‑collected or leaked data is used during live trading.","type":"data","stated":"explicit","quote":"AI‑Trader, the first fully‑automated, live, and data‑uncontaminated evaluation benchmark for LLM agents in financial decision‑making."},{"assumption":"Market data is provided in real‑time with strict temporal filtering to ensure information isolation.","type":"market","stated":"explicit","quote":"AI‑Trader operates in real‑time market conditions with strict temporal filtering and carefully designed tool interfaces."},{"assumption":"Trading frequencies (hourly and daily) are used to simulate realistic decision‑making cadences.","type":"execution","stated":"explicit","quote":"markets ... and provides both hourly and daily trading granularities to comprehensively simulate the decision‑making cadence."},{"assumption":"Risk control capability determines cross‑market robustness of agents.","type":"model","stated":"explicit","quote":"We demonstrate that risk control capability determines cross‑market robustness."},{"assumption":"Highly liquid markets (U.S.) are more conducive to generating excess returns than policy‑driven environments (A‑shares).","type":"market","stated":"explicit","quote":"AI trading strategies achieve excess returns more readily in highly liquid markets (U.S.) compared to policy‑driven environments (A‑shares)."}]},"datasets":{"status":"available","items":[{"name":"Nasdaq‑100 Index constituents","provider":"Nasdaq","kind":"market-prices","access":"public","period":null,"universe":"100 largest non‑financial Nasdaq stocks","quote":"To construct our U.S. portfolio, we select all constituents of the Nasdaq‑100 Index1. This index comprises the 100 largest non‑financial companies listed on the Nasdaq Stock Market."},{"name":"SSE‑50 Index constituents","provider":"Shanghai Stock Exchange","kind":"market-prices","access":"public","period":null,"universe":"50 largest and most liquid blue‑chip Chinese stocks","quote":"To construct our A‑share portfolio, we select all 50 constituent stocks of the SSE‑50 Index2. This index comprises the largest and most liquid blue‑chip companies listed on the Shanghai Stock Exchange."}],"known":[],"sample":{"start":"2025-10","end":"2025-11","market":"U.S. stocks (NASDAQ-100)","asset_class":"equities","frequency":"intraday"},"data_resolution":"intraday-bars"},"transaction_costs":{"status":"available","modeled":"no","headline_net_of_costs":false,"detail":null,"quote":null,"source":"front_matter","rule":"R1","candidates":{"front_matter":"no","evidence_card":"unclear","evidence_card_quoted":false},"meaning":"a trading or execution result is reported without costs"},"results":{"status":"available","evaluation_type":"trading-pnl","findings":[{"finding":"General intelligence does not automatically translate to effective trading; most agents exhibit poor returns and weak risk management.","kind":"negative","quote":"Our analysis reveals striking findings: general intelligence does not automatically translate to effective trading capability, with most agents exhibiting poor returns and weak risk management."},{"finding":"AI agents achieve excess returns more readily in the highly liquid U.S. market; MiniMax‑M2 (+7.69% excess), DeepSeek‑v3.1 (+6.52% excess), Claude‑3.7‑Sonnet (+1.24% excess) all outperformed QQQ.","kind":"performance","quote":"In the U.S. market, multiple AI agents achieved positive excess returns: MiniMax‑M2 (+7.69% excess), DeepSeek‑v3.1 (+6.52% excess), and even Claude‑3.7‑Sonnet (+1.24% excess) all outperformed QQQ."},{"finding":"No agent outperformed the A‑share SSE‑50 benchmark (1.65%); the best, MiniMax‑M2, returned 1.31% with the lowest volatility (6.72%) and drawdown (-2.15%).","kind":"performance","quote":"MiniMax‑M2 still came closest to the benchmark with a return of 1.31%, and achieved the lowest volatility (6.72%) and smallest maximum drawdown (-2.15%) among all AI agents."},{"finding":"In the cryptocurrency experiment, DeepSeek‑v3.1 was the only agent to outperform the baseline, achieving -12.18% vs. -14.30% for the CD5 Index by maintaining high cash exposure.","kind":"performance","quote":"DeepSeek‑v3.1 is the sole agent that outperforms the baseline (-12.18% vs. -14.30% for CD5 Index). Its success stemmed from maintaining a high cash position (peaking at ~41 %) during the drawdown and executing strategic \"buy‑the‑dip\" operations."},{"finding":"Model generalization is limited; optimization in a single market does not transfer to others, leading to significant cross‑market performance gaps.","kind":"negative","quote":"Model generalization capabilities exhibit significant cross‑market limitations when operating without human guidance."}],"reported_performance":{"strategy":"MiniMax-M2","sharpe":null,"sharpe_benchmark":null,"annual_return_pct":null,"total_return_pct":9.56,"max_drawdown_pct":4.92},"out_of_sample":{"method":"unclear","headline_out_of_sample":null}},"independent_evidence":{"status":"available","supports":[],"contradicts":[],"qualifies":[],"compared_with":35,"citations":{"cited_by_in_corpus":4,"by_independent_authors":3,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":1,"mentions":1,"unclassified":2},"independent_papers":[{"paper":{"paper_id":"2603.17692","title":"Can Blindfolded LLMs Still Trade? An Anonymization-First Framework for Portfolio Optimization","paper_date":"2026-03-18","authors":["Joohyoung Jeon","Hongchul Lee"],"empirical_rigor":7.5,"page_url":"https://thequant.space/flowcharts/can-blindfolded-llms-still-trade-an-anonymization-first-framework-for-portfolio/"},"how":"discusses_findings","quote":"AI-Trader [CITED] reaches a similar conclusion. In tests spanning U.S. stocks, Chinese A-shares, and cryptocurrencies, agents without risk management perform poorly in practice."}],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":true,"code_url":"https://github.com/HKUDS/AI-Trader","note":null},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T17:20:27Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}