Paper: arXiv 2601.06088

Authors: Bohan Liang, Zijian Chen, Qi Jia, Kaiwei Zhang, Kaiyuan Ji, Guangtao Zhai

Abstract

Stock prediction, a subject closely related to people’s investment activities in fully dynamic and live environments, has been widely studied. Current large language models (LLMs) have shown remarkable potential in various domains, exhibiting expert-level performance through advanced reasoning and contextual understanding. In this paper, we introduce PriceSeer, a live, dynamic, and data-uncontaminated benchmark specifically designed for LLMs performing stock prediction tasks. Specifically, PriceSeer includes 110 U.S. stocks from 11 industrial sectors, with each containing 249 historical data points. Our benchmark implements both internal and external information expansion, where LLMs receive extra financial indicators, news, and fake news to perform stock price prediction. We evaluate six cutting-edge LLMs under different prediction horizons, demonstrating their potential in generating investment strategies after obtaining accurate price predictions for different sectors. Additionally, we provide analyses of LLMs’ suboptimal performance in long-term predictions, including the vulnerability to fake news and specific industries. The code and evaluation data will be open-sourced at https://github.com/BobLiang2113/PriceSeer.

Complexity vs Empirical Score

  • Math Complexity: 2.5/10
  • Empirical Rigor: 6.5/10
  • Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper introduces a novel, dynamic evaluation benchmark (PriceSeer) for assessing LLM capabilities in stock price prediction and information robustness without mathematical depth beyond basic standard indicators. The empirical framework is well-structured with multiple horizons, sectors, and adversarial news perturbations, backed by open-source code and data.

Research Flowchart

  flowchart TD
    A["Research Goal:<br>Evaluate LLMs in Real-Time Stock Prediction (PriceSeer)"] --> B["Benchmark Setup:<br>110 U.S. Stocks across 11 Sectors"]
    B --> C["Data Integration & Information Expansion:<br>Financial Indicators, Real News & Fake News"]
    C --> D["Evaluation:<br>6 LLMs tested over Short & Long Prediction Horizons"]
    D --> E["Key Findings & Outcomes:<br>• LLMs generate viable short-term strategies<br>• Suboptimal long-term prediction accuracy<br>• Vulnerability to fake news & specific sectors"]