Predictive Power of LLMs in Financial Markets

ArXiv ID: 2411.16569 “View on arXiv”

Authors: Unknown

Abstract

Predicting the movement of the stock market and other assets has been valuable over the past few decades. Knowing how the value of a certain sector market may move in the future provides much information for investors, as they use that information to develop strategies to maximize profit or minimize risk. However, market data are quite noisy, and it is challenging to choose the right data or the right model to create such predictions. With the rise of large language models, there are ways to analyze certain data much more efficiently than before. Our goal is to determine whether the GPT model provides more useful information compared to other traditional transformer models, such as the BERT model. We shall use data from the Federal Reserve Beige Book, which provides summaries of economic conditions in different districts in the US. Using such data, we then employ the LLM’s to make predictions on the correlations. Using these correlations, we then compare the results with well-known strategies and determine whether knowing the economic conditions improves investment decisions. We conclude that the Beige Book does contain information regarding correlations amongst different assets, yet the GPT model has too much look-ahead bias and that traditional models still triumph.

Keywords: Large Language Models (LLMs), Sentiment Analysis, Economic Forecasting, Portfolio Optimization, GPT vs BERT

Complexity vs Empirical Score

• Math Complexity: 3.0/10
• Empirical Rigor: 7.0/10
• Quadrant: Street Traders
• Why: The paper uses standard statistical methods (linear regression) and treats LLM outputs as classification/continuous signals, lacking advanced mathematical derivations. However, it involves real financial data, backtesting portfolio strategies, and implementing ML models with specific prompts, indicating significant implementation and empirical validation efforts.

  flowchart TD
    A["Research Goal:<br>Predict Market Movement<br>using Beige Book"] --> B["Data Input:<br>Federal Reserve Beige Book<br>Economic Summaries"]
    B --> C["Methodology:<br>Apply GPT vs BERT Models<br>for Sentiment Analysis"]
    C --> D["Computational Process:<br>Extract Asset Correlations<br>Test for Predictive Power"]
    D --> E{"Analysis of Outcomes"}
    E -->|Look-ahead Bias| F["Outcome: GPT Model<br>High Bias, Limited Utility"]
    E -->|Traditional Analysis| G["Outcome: BERT/Traditional Models<br>Outperform for Strategy"]
    F & G --> H["Conclusion:<br>Economic Conditions Contain Signal,<br>but LLMs Require Rigorous Validation"]