Paper: arXiv 2308.16771
Authors: Rick Steinert, Saskia Altmann
Abstract
This paper investigates the potential improvement of the GPT-4 Language Learning Model (LLM) in comparison to BERT for modeling same-day daily stock price movements of Apple and Tesla in 2017, based on sentiment analysis of microblogging messages. We recorded daily adjusted closing prices and translated them into up-down movements. Sentiment for each day was extracted from messages on the Stocktwits platform using both LLMs. We develop a novel method to engineer a comprehensive prompt for contextual sentiment analysis which unlocks the true capabilities of modern LLM. This enables us to carefully retrieve sentiments, perceived advantages or disadvantages, and the relevance towards the analyzed company. Logistic regression is used to evaluate whether the extracted message contents reflect stock price movements. As a result, GPT-4 exhibited substantial accuracy, outperforming BERT in five out of six months and substantially exceeding a naive buy-and-hold strategy, reaching a peak accuracy of 71.47 % in May. The study also highlights the importance of prompt engineering in obtaining desired outputs from GPT-4’s contextual abilities. However, the costs of deploying GPT-4 and the need for fine-tuning prompts highlight some practical considerations for its use.
Complexity vs Empirical Score
- Math Complexity: 2.0/10
- Empirical Rigor: 6.5/10
- Quadrant: Street Traders — empirical and applied, lower math complexity
Why this score: The paper employs a straightforward logistic regression model with minimal advanced mathematics, placing it in the low complexity range. However, it demonstrates high empirical rigor through backtesting on real historical data, comparison against benchmarks, and detailed reporting of accuracy metrics, aligning with the Street Traders quadrant.
Research Flowchart
flowchart TD
A["Research Goal<br>Assess GPT-4 vs. BERT for predicting<br>daily stock movements using microblogging sentiment"] --> B["Data Collection & Preparation"]
B --> C["Sentence Transformation &<br>LLM Prompt Engineering"]
C --> D["Sentiment Extraction"]
D --> E{"Compute Daily<br>Sentiment Scores"}
E --> F["Logistic Regression Modeling"]
F --> G["Key Findings & Outcomes"]
subgraph B ["Data Collection & Preparation"]
B1["Stock Prices<br>(Apple & Tesla 2017)"]
B2["Microblogging Messages<br>(Stocktwits)"]
end
subgraph D ["Sentiment Extraction"]
D1["GPT-4 Analysis"]
D2["BERT Analysis"]
end
G --> G1["Outcomes"]
G1 --> G2["GPT-4 outperformed BERT in 5/6 months"]
G1 --> G3["Peak accuracy: 71.47% (May 2017)"]
G1 --> G4["Better than Buy-and-Hold strategy"]
G1 --> G5["Prompt engineering is crucial for performance"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes a novel prompt engineering method for contextual sentiment analysis using GPT-4 to predict same‑day stock price movements.
Source sentence
We develop a novel method to engineer a comprehensive prompt for contextual sentiment analysis which unlocks the true capabilities of modern LLM.
Assumes 3
- A higher correlation between investor sentiment and same‑day stock return movements indicates a better interpretation of the social media feed. model
Source sentence
In our study, we assume that a higher correlation between investor sentiment and same-day stock return movements indicates a better interpretation of the social media feed.
- Predicting same‑day movements captures the most immediate and potent effects of sentiment. other
Source sentence
By predicting same-day movements, we capture the most immediate and potent effects of sentiment.
- Apple and Tesla markets are quite efficient, making same‑day reaction modeling feasible. market
Source sentence
Given that we have employed large and closely monitored companies like Apple and Tesla, we assume that the markets are quite efficient in these domains.
Data
- Daily adjusted closing prices market-prices unknown
Source sentence
We recorded daily adjusted closing prices and translated them into up-down movements.
- Stocktwits microblogging messages text-news public
Source sentence
Sentiment for each day was extracted from messages on the Stocktwits platform using both LLMs.
Costs
Not applicable
Finds 1
- GPT-4 achieved higher accuracy than BERT in five of six months and reached a peak accuracy of 71.47% in May, substantially exceeding a naive buy‑and‑hold strategy. predictive-accuracy
Source sentence
As a result, GPT-4 exhibited substantial accuracy, outperforming BERT in five out of six months and substantially exceeding a naive buy-and-hold strategy, reaching a peak accuracy of 71.47 % in May.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 3 related papers). Coverage is partial, so this is neither replication nor refutation.
4 papers in the archive
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.