The Wall Street Neophyte: A Zero-Shot Analysis of ChatGPT Over MultiModal Stock Movement Prediction Challenges

Paper: arXiv 2304.05351

Authors: Qianqian Xie, Weiguang Han, Yanzhao Lai, Min Peng, Jimin Huang

Abstract

Recently, large language models (LLMs) like ChatGPT have demonstrated remarkable performance across a variety of natural language processing tasks. However, their effectiveness in the financial domain, specifically in predicting stock market movements, remains to be explored. In this paper, we conduct an extensive zero-shot analysis of ChatGPT’s capabilities in multimodal stock movement prediction, on three tweets and historical stock price datasets. Our findings indicate that ChatGPT is a “Wall Street Neophyte” with limited success in predicting stock movements, as it underperforms not only state-of-the-art methods but also traditional methods like linear regression using price features. Despite the potential of Chain-of-Thought prompting strategies and the inclusion of tweets, ChatGPT’s performance remains subpar. Furthermore, we observe limitations in its explainability and stability, suggesting the need for more specialized training or fine-tuning. This research provides insights into ChatGPT’s capabilities and serves as a foundation for future work aimed at improving financial market analysis and prediction by leveraging social media sentiment and historical stock data.

Complexity vs Empirical Score

• Math Complexity: 2.5/10
• Empirical Rigor: 6.5/10
• Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: This paper presents an empirical evaluation of zero-shot ChatGPT for stock prediction using text and price data. While math complexity is low, the empirical analysis compares LLM performance against traditional baselines across multiple datasets, highlighting the current limitations of general-purpose LLMs in financial forecasting.

Research Flowchart

  flowchart TD
    A[Research Goal: Evaluate ChatGPT Zero-Shot Multimodal Stock Prediction] --> B[Input Data: Historical Stock Prices & Social Media Tweets]
    B --> C[Methodology: Apply CoT Prompting & Zero-Shot Analysis]
    C --> D[Comparison: Benchmarked against SOTA & Linear Regression]
    D --> E[Outcome: ChatGPT Underperforms SOTA & Traditional Price-based Models]
    E --> F[Conclusion: Limited Explainability/Stability Requires Specialized Fine-Tuning]