{"paper_id":"2505.07078","title":"Can LLM-based Financial Investing Strategies Outperform the Market in Long Run?","authors":["Weixian Waylon Li","Hyeonjun Kim","Mihai Cucuringu","Tiejun Ma"],"paper_date":"2025-05-11","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2505.07078v4","page_url":"https://thequant.space/flowcharts/can-llm-based-financial-investing-strategies-outperform-the/","scores":{"empirical_rigor":8,"math_complexity":4.5,"quadrant":"Street Traders"},"proposes":{"status":"available","statement":"The paper proposes FINSABER, a comprehensive backtesting framework for evaluating LLM-based investing strategies over long periods and a broad symbol universe.","quote":"We propose FINSABER, the first comprehensive evaluation framework for LLM-based investing strategies that supports 20 years of multi-source data, including unstructured inputs such as news and filings, expands symbol coverage via unbiased selection, and mitigates survivorship, look-ahead, and data-snooping biases.","summary":"Large Language Models (LLMs) have recently been leveraged for asset pricing tasks and stock trading applications, enabling AI agents to generate investment decisions from unstructured financial data. However, most evaluations of LLM timing-based investing strategies are conducted on narrow…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"A historical average risk‑free rate of 0.03 is used for performance calculations.","type":"market","stated":"explicit","quote":"Thus, we set a historical average risk-free rate of 0.03 and use Moomoo’s3 standard US commission fee ($0.0049/share, minimum $0.99/order), comparable to HSBC and TradeUp4."},{"assumption":"Transaction costs are modeled using a flat commission of $0.0049 per share with a $0.99 minimum per order.","type":"execution","stated":"explicit","quote":"Thus, we set a historical average risk-free rate of 0.03 and use Moomoo’s3 standard US commission fee ($0.0049/share, minimum $0.99/order), comparable to HSBC and TradeUp4."}]},"datasets":{"status":"available","items":[{"name":"20‑year multi‑source data (prices, news, filings)","provider":null,"kind":"text-news","access":"unknown","period":"20 years","universe":"100+ symbols","quote":"We propose FINSABER, the first comprehensive evaluation framework for LLM-based investing strategies that supports 20 years of multi-source data, including unstructured inputs such as news and filings, expands symbol coverage via unbiased selection..."}],"known":[],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"daily"},"transaction_costs":{"status":"available","modeled":"partial","headline_net_of_costs":null,"detail":"Flat commission $0.0049 per share with $0.99 minimum per order; used for analysis but not incorporated into headline performance numbers.","quote":"Thus, we set a historical average risk-free rate of 0.03 and use Moomoo’s3 standard US commission fee ($0.0049/share, minimum $0.99/order), comparable to HSBC and TradeUp4.","source":"evidence_card","rule":"R2","candidates":{"front_matter":"unclear","evidence_card":"partial","evidence_card_quoted":true},"card_detail":"Flat commission $0.0049 per share with $0.99 minimum per order; used for analysis but not incorporated into headline performance numbers.","card_quote":"Thus, we set a historical average risk-free rate of 0.03 and use Moomoo’s3 standard US commission fee ($0.0049/share, minimum $0.99/order), comparable to HSBC and TradeUp4.","meaning":"costs appear only as a robustness check, a flat fee without spread or impact, or outside the headline numbers"},"results":{"status":"available","evaluation_type":"trading-pnl","findings":[{"finding":"LLM timing‑based strategies lose their reported advantage when evaluated over longer horizons and a broader cross‑section of assets.","kind":"performance","quote":"Systematic backtests over two decades and 100+ symbols reveal that previously reported LLM advantages deteriorate significantly under broader cross-section and over a longer-term evaluation."},{"finding":"LLM strategies are overly conservative in bull markets and overly aggressive in bear markets, leading to underperformance and heavy losses.","kind":"performance","quote":"Our market regime analysis further demonstrates that LLM strategies are overly conservative in bull markets, underperforming passive benchmarks, and overly aggressive in bear markets, incurring heavy losses."},{"finding":"FinMem only consistently outperforms the benchmark for TSLA; traditional rule‑based methods remain competitive or superior for other symbols.","kind":"performance","quote":"Specifically, FinMem only consistently outperforms for TSLA, while traditional benchmarks remain competitive or superior for other symbols."},{"finding":"In unbiased selection setups, traditional methods achieve higher Sharpe ratios than LLM methods (e.g., Buy and Hold Sharpe 0.703 vs. FinAgent Sharpe 0.241).","kind":"performance","quote":"In the Volatility-based selection, traditional methods dominate even more clearly: Buy and Hold achieves the highest Sharpe (0.703)"},{"finding":"Statistical tests show LLM agents do not generate significant alpha; p‑values exceed 0.34, supporting the claim that their superiority does not hold under rigorous evaluation.","kind":"statistical","quote":"neither LLM agent generates statistically significant alpha, with all measured p‑values exceeding 0.34."}],"reported_performance":null,"out_of_sample":{"method":"unclear","headline_out_of_sample":null}},"independent_evidence":{"status":"available","supports":[{"paper":{"paper_id":"2411.00856","title":"AI in Investment Analysis: LLMs for Equity Stock Ratings","paper_date":"2024-10-30","authors":["Kassiani Papasotiriou","Srijan Sood","Shayleen Reynolds","Tucker Balch"],"empirical_rigor":8,"page_url":"https://thequant.space/flowcharts/ai-in-investment-analysis-llms-for-equity-stock-ratings/"},"question":"Do LLM‑based investment methods retain their advantage over longer investment horizons?","this_paper_finding":"LLM timing‑based strategies lose their reported advantage when evaluated over longer horizons and a broader cross‑section of assets.","other_paper_finding":"LLMs perform better for short‑term horizons, while traditional analysts perform better over longer horizons.","other_paper_quote":"News summaries are more beneficial for short-term predictions, while traditional analysts perform better over longer horizons.","note":"Both findings show the LLM advantage vanishes for longer horizons: A finds LLMs beat analysts only short‑term, while B reports timing‑based LLM strategies lose their edge over longer periods and a broader asset cross‑sec","confidence":0.726}],"contradicts":[],"qualifies":[],"compared_with":43,"citations":{"cited_by_in_corpus":9,"by_independent_authors":9,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":2,"mentions":4,"unclassified":3},"independent_papers":[{"paper":{"paper_id":"2611.07464","title":"FINCOSMOS: An Open-World Evaluation Paradigm for Agents in Finance","paper_date":"2026-11-05","authors":["Yupeng Cao","Xueqing Peng","Lingfei Qian","Haohang Li","Yangyang Yu","Anke Xu"],"empirical_rigor":9,"page_url":"https://thequant.space/flowcharts/fincosmos-an-open-world-evaluation-paradigm-for-agents-in-finance/"},"how":"discusses_findings","quote":"short evaluation windows overstate LLM strategies [CITED]"},{"paper":{"paper_id":"2512.23515","title":"Alpha-R1: Alpha Screening with LLM Reasoning via Reinforcement Learning","paper_date":"2025-12-29","authors":["Zuoyou Jiang","Li Zhao","Rui Sun","Ruohan Sun","Zhongjian Li","Jing Li"],"empirical_rigor":8.5,"page_url":"https://thequant.space/flowcharts/alpha-r1-alpha-screening-with-llm-reasoning-via-reinforceme/"},"how":"discusses_findings","quote":"both report that LLM-driven strategies overfit to specific market phases and degrade under regime shifts"}],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":true,"code_url":"https://github.com/waylonli/FINSABER","note":null},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T17:38:45Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}