Paper: arXiv 2411.00856
Authors: Kassiani Papasotiriou, Srijan Sood, Shayleen Reynolds, Tucker Balch
Abstract
Investment Analysis is a cornerstone of the Financial Services industry. The rapid integration of advanced machine learning techniques, particularly Large Language Models (LLMs), offers opportunities to enhance the equity rating process. This paper explores the application of LLMs to generate multi-horizon stock ratings by ingesting diverse datasets. Traditional stock rating methods rely heavily on the expertise of financial analysts, and face several challenges such as data overload, inconsistencies in filings, and delayed reactions to market events. Our study addresses these issues by leveraging LLMs to improve the accuracy and consistency of stock ratings. Additionally, we assess the efficacy of using different data modalities with LLMs for the financial domain. We utilize varied datasets comprising fundamental financial, market, and news data from January 2022 to June 2024, along with GPT-4-32k (v0613) (with a training cutoff in Sep. 2021 to prevent information leakage). Our results show that our benchmark method outperforms traditional stock rating methods when assessed by forward returns, specially when incorporating financial fundamentals. While integrating news data improves short-term performance, substituting detailed news summaries with sentiment scores reduces token use without loss of performance. In many cases, omitting news data entirely enhances performance by reducing bias. Our research shows that LLMs can be leveraged to effectively utilize large amounts of multimodal financial data, as showcased by their effectiveness at the stock rating prediction task. Our work provides a reproducible and efficient framework for generating accurate stock ratings, serving as a cost-effective alternative to traditional methods. Future work will extend to longer timeframes, incorporate diverse data, and utilize newer models for enhanced insights.
Complexity vs Empirical Score
- Math Complexity: 2.5/10
- Empirical Rigor: 8.0/10
- Quadrant: Street Traders — empirical and applied, lower math complexity
Why this score: The paper focuses on applying off-the-shelf LLMs to financial data with minimal novel mathematical derivations, but includes a detailed empirical study using real market data over multiple years with robust backtesting on forward returns.
Research Flowchart
flowchart TD
A["Research Goal<br>Apply LLMs for Equity Stock Ratings"] --> B
subgraph B ["Key Methodology"]
direction LR
B1["Data Collection<br>Jan 2022 - Jun 2024"] --> B2["LLM Processing<br>GPT-4-32k"]
end
B --> C["Data Inputs"]
subgraph C ["Multimodal Datasets"]
C1["Fundamental Data"]
C2["Market Data"]
C3["News/Sentiment"]
end
C --> D["Computational Process<br>Multi-Horizon Stock Rating Generation"]
D --> E["Key Findings & Outcomes"]
subgraph E ["Results"]
direction LR
E1["↑ Accuracy vs Traditional Methods<br>Especially with Fundamentals"]
E2["News aids short-term<br>but increases bias"]
E3["Sentiment scores<br>reduce token usage"]
E4["Reproducible &<br>Cost-effective Framework"]
end
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes using a Large Language Model to predict equity stock ratings, presenting a reproducible framework that leverages multimodal financial data.
Source sentence
We demonstrate the novel use of an LLM for predicting stock ratings, addressing a significantly under-researched area.
Assumes 3
- The GPT-4-32k model is assumed to have no information leakage because its training cutoff is set to September 2021. model
Source sentence
We utilize GPT-4-32k (v0613), hosted on Azure, which features a context window of 32,000 tokens and is trained on data up to
- Evaluation of rating quality is based on forward returns quantiles, assuming that forward returns are a valid proxy for rating correctness. statistical
Source sentence
Our results show that our benchmark method outperforms traditional stock rating methods when assessed by forward returns.
- The method does not fine‑tune the LLM, assuming that zero‑shot prompting is sufficient for cost‑effective performance. model
Source sentence
Since we do not fine-tune the model, the process remains cost-effective.
Data
- Fundamental financial data fundamentals public
Source sentence
We aggregate quarterly company fundamentals from 10‑Q and 10‑K filings from January 2022 to March 2024 using finagg API
- Market price data market-prices unknown
Source sentence
Our analysis focuses on US equities, specifically the 500 constituents of the S&P 500 index, using data spanning from January 2022 to the end of June 2024.
- Financial news articles text-news unknown
Source sentence
on average per month, there are 39.63 articles, 187K characters, and 40K tokens.
Costs
Unclear
Finds 4
- The vanilla LLM model achieves a lower market‑relative MAE (1.447) than analyst ratings (1.570), indicating better predictive performance. performance
Source sentence
Vanilla method has a lower MAE of 1.447 compared to the Analyst predictions, which has a Return MAE of 1.570.
- Adding sentiment scores to fundamentals further improves MAE slightly (1.417) compared to fundamentals alone. predictive-accuracy
Source sentence
Fundamentals + Sentiment 1.417 ± 0.747
- Using news summaries versus sentiment scores provides only a very small performance difference when fundamentals are not included. performance
Source sentence
The performance difference between adding news as text versus news sentiment to the LLM is very small when other data is not included (i.e. Fundamentals).
- LLMs perform better for short‑term horizons, while traditional analysts perform better over longer horizons. performance
Source sentence
News summaries are more beneficial for short-term predictions, while traditional analysts perform better over longer horizons.
Checked by others
Supported by
- Can LLM-based Financial Investing Strategies Outperform the Market in Long Run?
Compare the findings
This paper: LLMs perform better for short‑term horizons, while traditional analysts perform better over longer horizons.
That paper: LLM timing‑based strategies lose their reported advantage when evaluated over longer horizons and a broader cross‑section of assets.
3 papers in the archive
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (82)
- Teaching machines to read and comprehend (2015) — link
- Language Models are Few-Shot Learners (2020) — link
- Table meets llm: Can large language models understand structured table data? a benchmark and empirical study (2024)
- Hallucination Leaderboard (2023)
- Evaluating GPT-4 Zero-Shot Summarization (2023)
- Medical Text Summarization using LLMs (2023)
- Fine-tuning and utilization methods of domain-specific llms (2024) — link
- State-of-the-art in stock prediction techniques (2013)
and 74 more
- Automated trading systems statistical and machine learning methods and hardware implementation: a survey (2019)
- InvestLM: A Large Language Model for Investment using Financial Domain Instruction Tuning — → 2309.13064
- TELeR: A General Taxonomy of LLM Prompts for Benchmarking Complex Tasks (2023) — link
- AlphaStock: A Buying-Winners-and-Selling-Losers Investment Strategy using Interpretable Deep Reinforcement Attention Networks (2019) — link
- Cryptocurrency Portfolio Management with Deep Reinforcement Learning (2017) — link
- Adversarial Deep Reinforcement Learning in Portfolio Management (2018) — link
- Model-based deep reinforcement learning for dynamic portfolio optimization (2019) — link
- Agent Inspired Trading Using Recurrent Reinforcement Learning and LSTM Neural Networks (2017) — link
- Deep Reinforcement Learning for Optimal Portfolio Allocation: A Comparative Study with Mean-Variance Optimization (2023)
- Adaptive quantitative trading: An imitative deep reinforcement learning approach (2020)
- FinRL: Deep reinforcement learning framework to automate trading in quantitative finance (2021)
- Sentiment and knowledge based algorithmic trading with deep reinforcement learning (2022)
- Can ChatGPT improve investment decisions? From a portfolio management perspective (2024)
- Chatgpt-based investment portfolio selection — → 2308.06260
- Portfolio performance based on LLM news scores and related economical analysis (2024)
- TradingGPT: Multi-Agent System with Layered Memory and Distinct Characters for Enhanced Financial Trading Performance — → 2309.03736
- FinMem: A performance-enhanced LLM trading agent with layered memory and character design — → 2311.13743
- QuantAgent: Seeking Holy Grail in Trading by Self-Improving Large Language Model — → 2402.03755
- Alpha-GPT: Human-AI Interactive Alpha Mining for Quantitative Investment — → 2308.00016
- Applications of deep learning in stock market prediction: recent progress (2021)
- A graph-based CNN-LSTM stock price prediction algorithm with leading indicators (2023)
- GCNET: graph-based prediction of stock price movement using graph convolutional network (2022)
- Transformer-based deep learning model for stock price prediction: A case study on Bangladesh stock market (2023)
- Hierarchical Multi-Scale Gaussian Transformer for Stock Movement Prediction. (2020)
- CNNpred: CNN-based stock market prediction using a diverse set of variables (2019)
- Stock price prediction using LSTM, RNN and CNN-sliding window model (2017)
- Deep learning networks for stock market analysis and prediction: Methodology, data representations, and case studies (2017)
- Stock prediction using deep learning (2017)
- Deep learning for stock market prediction from financial news articles (2017)
- Deep learning for stock prediction using numerical and textual information (2016)
- Deep learning for event-driven stock prediction (2015)
- Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models — → 2304.07619
- Learning to Generate Explainable Stock Predictions using Self-Reflective Large Language Models — → 2402.03659
- Can Large Language Models Beat Wall Street? Unveiling the Potential of AI in Stock Selection — → 2401.03737
- Ploutos: Towards interpretable stock movement prediction with financial large language model — → 2403.00782
- RiskLabs: Predicting Financial Risk Using Large Language Model Based on Multi-Sources Data (2024) — link
- A lpha F in: Benchmarking Financial Analysis with Retrieval-Augmented Stock-Chain Framework (2024) — link
- FundRecLLM: Fund recommendation based on financial news and research analyst report (2023) — link
- Do Brokerage Analysts’ Recommendations Have Investment Value? (1996)
- Financial analysts’ performance: Sector versus country specialization (2009)
- Information content of analyst recommendations in the banking industry (2017)
- When Are Analyst Recommendation Changes Influential? (2010) — link
- The profitability of following analyst recommendations on the Polish stock market (2015)
- The role and responsibilities of securities analysts (2001)
- Counselors, judges, or executioners: The role of financial analysts in capital markets’ responses to alleged FCPA violations (2021)
- Forecasting stock price trends by analyzing economic reports with analyst profiles (2022)
- Analyst recommendations, mutual fund herding, and overreaction in stock prices (2014)
- Analyzing the Analysts: When Do Recommendations Add Value? (2004)
- Value of analyst recommendations: International evidence (2006)
- Analyst following and credit ratings (2008)
- Ratings changes, ratings levels, and the predictive value of analysts’ recommendations (2010)
- Buys, holds, and sells: The distribution of investment banks’ stock ratings and the implications for the profitability of analysts’ recommendations (2006) — link
- Reassessing the returns to analysts’ stock recommendations (2003)
- Predicting stock market movements using neural networks: a review and application study (2024)
- Revolutionizing Finance with LLMs: An Overview of Applications and Insights (2024) — link
- Stock market prediction using deep learning (2022)
- Stock market prediction using deep learning algorithms (2023)
- BloombergGPT: A Large Language Model for Finance (2023) — link
- Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Models — → 2310.04027
- Extracting Structured Insights from Financial News: An Augmented LLM Driven Approach (2024) — link
- Evaluation Measures for Ordinal Regression (2009) — link
- A Survey of Large Language Models in Finance (FinLLMs) (2024) — link
- yfinance: Yahoo! Finance market data downloader (2023)
- LangChain (2022) — link
- SerpAPI: Google Search API (2023)
- GPT-4-32k (v0613) (2023)
- OpenAIPlatform — link
- finagg: A Financial Aggregation Library (2023)
- EDGAR Application Programming Interfaces — link
- Chain-of-thought prompting elicits reasoning in large language models (2024)
- Comparison of Prompt Engineering and Fine-Tuning Strategies in Large Language Models in the Classification of Clinical Notes (2024)
- Leancontext: Cost-efficient domain-specific question answering using llms (2024)
- Evaluating LLM s ’ Mathematical Reasoning in Financial Document Question Answering (2024) — link
- A comprehensive survey on process-oriented automatic text summarization with exploration of llm-based methods (2024) — link
11 of 82 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.