{"paper_id":"2412.18174","title":"INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agent","authors":["Haohang Li","Yupeng Cao","Yangyang Yu","Shashidhar Reddy Javaji","Zhiyang Deng","Yueru He","Yuechen Jiang","Zining Zhu","Koduvayur Subbalakshmi","Guojun Xiong","Jimin Huang","Lingfei Qian","Xueqing Peng","Qianqian Xie","Jordan W. Suchow"],"paper_date":"2024-12-24","paper_type":"dataset-benchmark","paper_url":"https://arxiv.org/pdf/2412.18174v1","page_url":"https://thequant.space/flowcharts/investorbench-a-benchmark-for-financial-decision-making-tas/","scores":{"empirical_rigor":6,"math_complexity":2.5,"quadrant":"Street Traders"},"proposes":{"status":"available","statement":"The paper proposes InvestorBench, a benchmark for evaluating LLM-based agents in diverse financial decision‑making contexts.","quote":"we introduce INVESTORBENCH, the first benchmark specifically designed for evaluating LLM-based agents in diverse financial decision-making contexts.","summary":"Recent advancements have underscored the potential of large language model (LLM)-based agents in financial decision-making. Despite this progress, the field currently encounters two main challenges: (1) the lack of a comprehensive LLM agent framework adaptable to a variety of financial tasks, and…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"The benchmark focuses on single‑asset financial decision‑making tasks only.","type":"other","stated":"implied","quote":"First, INVESTORBENCH is currently focusing on single-asset financial decision-making task, without addressing multi-asset tasks such as portfolio management."}]},"datasets":{"status":"available","items":[{"name":"Stock market environment","provider":"Yahoo Finance, SEC EDGAR, Refinitiv, Zhou et al.","kind":"market-prices","access":"public","period":"2020-07-01 to 2021-05-06","universe":"seven stocks (MSFT, JNJ, UVV, HON, TSLA, AAPL, NIO)","quote":"Stock market environment integrates information from multiple sources, including: 1) Daily stock open, high, low, close, and volume (OHLCV) data acquired from Yahoo Finance. 2) Summarized insights from company quarterly and annual reports (Form 10-Q and 10-K) downloaded from the SEC EDGAR database. 3) News articles for","known_id":"refinitiv"},{"name":"Cryptocurrency market environment","provider":"CoinMarketCap, cryptonews, cryptopotato, cointelegraph, Zhou","kind":"crypto-onchain","access":"public","period":"2023-02-13 to 2023-11-05","universe":"cryptocurrency assets","quote":"Cryptocurrency market environment encompasses 1) the daily stock open-high-low-close-volume (OHLCV) acquired from CoinMarketCap; 2) the multisource cryptocurrency news data collected from cryptonews, cryptopotato, and cointelegraph","known_id":"coinmarketcap"},{"name":"ETF market environment","provider":"Saqur et al.","kind":"text-news","access":"public","period":"2019-07-29 to 2020-09-21","universe":"ETF assets","quote":"ETF market environment is constructed using News-Informed Financial Trend Yield (NIFTY) dataset (Saqur et al., 2024). It contains the processed and curated daily news headlines from 2019-07-29 to 2020-09-21 and generated sentiment categories for each news headline."}],"known":[{"id":"edgar","name":"SEC EDGAR","provider":"U.S. Securities and Exchange Commission","kind":"text-news","access":"public","quote":"2) Summarized insights from company quarterly and annual reports (Form 10-Q and 10-K) downloaded from the SEC EDGAR database.","mentions":2,"used_in":1},{"id":"yahoo-finance","name":"Yahoo Finance","provider":"Yahoo","kind":"market-prices","access":"public","quote":"4 Stock market environment integrates information from multiple sources, including: 1) Daily stock open, high, low, close, and volume (OHLCV) data acquired from Yahoo Finance.","mentions":2,"used_in":1},{"id":"coinmarketcap","name":"CoinMarketCap","provider":"CoinMarketCap","kind":"market-prices","access":"public","quote":"Cryptocurrency market environment encom-passes 1) the daily stock open-high-low-close-volume (OHLCV) acquired from CoinMarketCap; 2) the multisource cryptocurrency news data collected from cryptonews, cryptopotato, and cointelegraph(Vanhoucke, 2023); 3) news spanning from 2023-02-13 to 2023-11-05 collected by (Zhou et al., 2021) in daily frequency.","mentions":1,"used_in":1},{"id":"refinitiv","name":"Refinitiv / LSEG (Datastream, Eikon, Tick History)","provider":"LSEG","kind":"market-prices","access":"commercial","quote":"these companies—Microsoft Corporation (MSFT), Johnson & Johnson (JNJ), UVV Corporation (UVV), and Honeywell International Inc. (HON)-are randomly selected from the pool with the most new records (over five hundred) from the open-access dataset provided by Zhou et al. (Zhou et al., 2021), while the news data for the remaining three companies—Tesla, Inc. (TSLA), Apple Inc. (AAPL), and NIO Inc. (NIO)-are obtained from R","mentions":1,"used_in":1}],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"daily"},"transaction_costs":{"status":"available","modeled":"no","headline_net_of_costs":null,"detail":null,"quote":null,"source":"front_matter","rule":"R1","candidates":{"front_matter":"no","evidence_card":"unclear","evidence_card_quoted":false},"meaning":"a trading or execution result is reported without costs"},"results":{"status":"available","evaluation_type":"trading-pnl","findings":[{"finding":"Mid‑sized and small‑sized open‑source LLM backbones generally yield weaker performance than the market baseline for cryptocurrency trading (lower Cumulative Return and Sharpe Ratio).","kind":"performance","quote":"using mid-sized and small-sized open-source models as the decision-making agent backbone generally results in weaker performance than the market baseline with respect to CR and SR."},{"finding":"Proprietary LLMs significantly outperform open‑source and financial‑domain models in ETF trading, achieving higher Cumulative Return and Sharpe Ratio.","kind":"performance","quote":"proprietary models significantly outperform open-source and financial domain-specific models in this task."}],"reported_performance":null,"out_of_sample":{"method":"holdout","headline_out_of_sample":null}},"independent_evidence":{"status":"available","supports":[],"contradicts":[],"qualifies":[],"compared_with":15,"citations":{"cited_by_in_corpus":15,"by_independent_authors":12,"how_cited":{"used_as_baseline":0,"uses_method_or_data":1,"extends":0,"discusses_findings":0,"mentions":10,"unclassified":4},"independent_papers":[{"paper":{"paper_id":"2609.33470","title":"LiveOption: Evaluating LLM Agents in Structured Option Trading with Nonlinear Payoffs","paper_date":"2026-09-27","authors":["Haochen Luo","Yifan Li","Binh Minh An","Xiaolong Luo","Zhengzhao Lai","Yuan Zhang"],"empirical_rigor":8,"page_url":"https://thequant.space/flowcharts/liveoption-evaluating-llm-agents-in-structured-option-trading-with-nonlinear/"},"how":"uses_method_or_data","quote":null}],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T17:48:50Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}