Paper: arXiv 2411.01368
Authors: Ali Elahi, Fatemeh Taghvaei
Abstract
Predicting financial markets and stock price movements requires analyzing a company’s performance, historic price movements, industry-specific events alongside the influence of human factors such as social media and press coverage. We assume that financial reports (such as income statements, balance sheets, and cash flow statements), historical price data, and recent news articles can collectively represent aforementioned factors. We combine financial data in tabular format with textual news articles and employ pre-trained Large Language Models (LLMs) to predict market movements. Recent research in LLMs has demonstrated that they are able to perform both tabular and text classification tasks, making them our primary model to classify the multi-modal data. We utilize retrieval augmentation techniques to retrieve and attach relevant chunks of news articles to financial metrics related to a company and prompt the LLMs in zero, two, and four-shot settings. Our dataset contains news articles collected from different sources, historic stock price, and financial report data for 20 companies with the highest trading volume across different industries in the stock market. We utilized recently released language models for our LLM-based classifier, including GPT- 3 and 4, and LLaMA- 2 and 3 models. We introduce an LLM-based classifier capable of performing classification tasks using combination of tabular (structured) and textual (unstructured) data. By using this model, we predicted the movement of a given stock’s price in our dataset with a weighted F1-score of 58.5% and 59.1% and Matthews Correlation Coefficient of 0.175 for both 3-month and 6-month periods.
Complexity vs Empirical Score
- Math Complexity: 3.0/10
- Empirical Rigor: 6.0/10
- Quadrant: Street Traders — empirical and applied, lower math complexity
Why this score: The paper applies established LLM techniques (zero/few-shot prompting, retrieval augmentation) with minimal novel mathematical formalization, but demonstrates strong empirical implementation through detailed data collection, model selection (GPT-3/4, LLaMA), and reported metrics (F1-score, MCC).
Research Flowchart
flowchart TD
A["Research Goal<br>Predict stock price movement<br>using multi-modal data"] --> B{"Data Collection"}
B --> C["Financial Data<br>Tabular: Income, Balance, Cash Flow"]
B --> D["Market Data<br>Tabular: Historical Prices"]
B --> E["News Articles<br>Textual: Sentiment & Events"]
C & D & E --> F["Preprocessing & RAG<br>Retrieve relevant news chunks"]
F --> G["LLM Classifier<br>GPT-3/4, LLaMA-2/3<br>Zero/Few-shot prompting"]
G --> H["Outcome<br>Weighted F1: 58.5% - 59.1%<br>MCC: 0.175"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes an LLM-based classifier that combines tabular financial data and textual news articles to predict stock price movements.
Source sentence
We introduce an LLM-based classifier capable of performing classification tasks using combination of tabular (structured) and textual (unstructured) data.
Assumes 2
- Financial reports, historical price data, and recent news articles together can represent the factors influencing stock price movements. data
Source sentence
We assume that financial reports (such as income statements, balance sheets, and cash flow statements), historical price data, and recent news articles can collectively represent aforementioned factors.
- The analysis can be performed in zero‑to‑four‑shot settings without fine‑tuning the LLMs. model
Source sentence
we will conduct this analysis in a zero to four-shot setting.
Data
- Combined financial and news dataset text-news unknown
Source sentence
Our dataset contains news articles collected from different sources, historic stock price, and financial report data for 20 companies with the highest trading volume across different industries in the stock market.
Costs
Not applicable
Finds 3
- The LLM‑based classifier achieved weighted F1‑scores of 58.5% (3‑month) and 59.1% (6‑month) with a Matthews Correlation Coefficient of 0.175 for both horizons. predictive-accuracy
Source sentence
By using this model, we predicted the movement of a given stock’s price in our dataset with a weighted F1-score of 58.5% and 59.1% and Matthews Correlation Coefficient of 0.175 for both 3-month and 6-month periods.
- The best‑performing models for both 3‑month and 6‑month predictions were the GPT and LLaMA3‑8B models. performance
Source sentence
The best-performing models for both 3-month and 6-month predictions were the GPT and LLaMA3-8B models.
- More advanced models (LLaMA3 and GPT) outperform LLaMA2, and GPT‑4 generally outperforms GPT‑3.5 except in zero‑shot 3‑month prediction where GPT‑3.5 has a better WF1 and MCC. performance
Source sentence
The results in Figure 3 demonstrate that more advanced models, such as LLaMA3 and GPT models, outperform LLaMA2 models. Additionally, LLaMA2 models show improved performance with an increase in the number of parameters. GPT-4 models generally exhibit better performance than GPT-3.5 models based in both WF1 and MCC, exc
Checked by others
Qualified by
- Assessing Consistency and Reproducibility in the Outputs of Large Language Models: Evidence Across Diverse Finance and Accounting Tasks
Compare the findings
This paper: More advanced models (LLaMA3 and GPT) outperform LLaMA2, and GPT‑4 generally outperforms GPT‑3.5 except in zero‑shot 3‑month prediction where GPT‑3.5 has a better WF1 and MCC.
That paper: More advanced models do not consistently outperform earlier models in consistency; GPT‑4o is better for numerical predictions but GPT‑3.5‑turbo outperforms GPT‑4o‑mini in multi‑class classification of FOMC statements.
2 papers in the archive
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (37)
- The Changing Landscape of Investors in the United States: A Report of the National Financial Capability Study. FINRA Investor Education Foundation (2022)
- Retrieval-augmented generation for knowledge-intensive NLP tasks (2020) — link
- A comparative analysis of sentence embedding techniques for document ranking (2022)
- Sentence similarity based on contexts (2022)
- RAPTOR : Recursive abstractive processing for tree-organized retrieval (2024) — link
- Specialized document embeddings for aspect-based similarity of research papers (2022)
- FinGPT : Instruction Tuning benchmark for open-source large language models in financial datasets — → 2310.04793
- FinGPT : Open-source financial large language models — → 2306.06031
and 29 more
- LoRA : Low-Rank Adaptation of large language models (2021) — link
- R e2 G : Retrieve, Rerank, Generate (2022) — link
- A Study on the Implementation of Generative AI Services Using an Enterprise Data-Based LLM Application Architecture (2023) — link
- Large language models in finance: A survey — → 2311.10723
- Ploutos: Towards interpretable stock movement prediction with financial large language model — → 2403.00782
- LLMFactor : Extracting profitable factors through prompts for explainable stock movement prediction (2024) — link
- Stock market one-day ahead movement prediction using disparate data sources (2017)
- Making LLMs worth every penny: Resource-limited text classification in banking (2023)
- Learning mutual fund categorization using natural language processing (2022)
- Enhancing financial sentiment analysis via retrieval augmented large language models — → 2310.04027
- LLMs for financial advisement: A fairness and efficacy study in personal decision making (2023)
- Efficient capital markets: A review of theory and empirical work (1970)
- Can ChatGPT forecast stock price movements? Return predictability and large language models — → 2304.07619
- Forecasting stock market indices using the recurrent neural network based hybrid models: CNN-LSTM , GRU-CNN , and ensemble models (2023)
- Predicting stock market time-series data using CNN-LSTM Neural Network model — → 2305.14378
- Stock market one-day ahead movement prediction using disparate data sources (2017)
- Stock market’s price movement prediction with LSTM neural networks (2017)
- Prediction of stock market index movement by ten data mining techniques (2009)
- SelfCheckGPT : Zero-resource black-box hallucination detection for generative large language models (2023)
- PromptRank : Unsupervised Keyphrase Extraction Using Prompt (2023) — link
- Data Augmentation for Abstractive Query-Focused Multi-Document Summarization (2021)
- Language Models are Few-Shot Learners (2020) — link
- TabLLM : Few-shot classification of tabular data with large language models (2022) — link
- Large language models(LLMs ) on tabular data: Prediction, generation, and understanding – A survey (2024) — link
- Text Classification via Large Language Models (2023) — link
- Enhancing financial sentiment analysis via retrieval augmented large language models — → 2310.04027
- Making llms worth every penny: Resource-limited text classification in banking (2023)
- Learning Mutual Fund Categorization using Natural Language Processing (2022)
- P rompt R ank: Unsupervised Keyphrase Extraction Using Prompt (2023) — link
8 of 37 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.