Paper: arXiv 2308.16771

Authors: Rick Steinert, Saskia Altmann

Abstract

This paper investigates the potential improvement of the GPT-4 Language Learning Model (LLM) in comparison to BERT for modeling same-day daily stock price movements of Apple and Tesla in 2017, based on sentiment analysis of microblogging messages. We recorded daily adjusted closing prices and translated them into up-down movements. Sentiment for each day was extracted from messages on the Stocktwits platform using both LLMs. We develop a novel method to engineer a comprehensive prompt for contextual sentiment analysis which unlocks the true capabilities of modern LLM. This enables us to carefully retrieve sentiments, perceived advantages or disadvantages, and the relevance towards the analyzed company. Logistic regression is used to evaluate whether the extracted message contents reflect stock price movements. As a result, GPT-4 exhibited substantial accuracy, outperforming BERT in five out of six months and substantially exceeding a naive buy-and-hold strategy, reaching a peak accuracy of 71.47 % in May. The study also highlights the importance of prompt engineering in obtaining desired outputs from GPT-4’s contextual abilities. However, the costs of deploying GPT-4 and the need for fine-tuning prompts highlight some practical considerations for its use.

Complexity vs Empirical Score

  • Math Complexity: 2.0/10
  • Empirical Rigor: 6.5/10
  • Quadrant: Street Traders — empirical and applied, lower math complexity

Why this score: The paper employs a straightforward logistic regression model with minimal advanced mathematics, placing it in the low complexity range. However, it demonstrates high empirical rigor through backtesting on real historical data, comparison against benchmarks, and detailed reporting of accuracy metrics, aligning with the Street Traders quadrant.

Research Flowchart

  flowchart TD
  A["Research Goal<br>Assess GPT-4 vs. BERT for predicting<br>daily stock movements using microblogging sentiment"] --> B["Data Collection & Preparation"]
  B --> C["Sentence Transformation &<br>LLM Prompt Engineering"]
  C --> D["Sentiment Extraction"]
  D --> E{"Compute Daily<br>Sentiment Scores"}
  E --> F["Logistic Regression Modeling"]
  F --> G["Key Findings & Outcomes"]
  
  subgraph B ["Data Collection & Preparation"]
      B1["Stock Prices<br>(Apple & Tesla 2017)"]
      B2["Microblogging Messages<br>(Stocktwits)"]
  end
  
  subgraph D ["Sentiment Extraction"]
      D1["GPT-4 Analysis"]
      D2["BERT Analysis"]
  end
  
  G --> G1["Outcomes"]
  G1 --> G2["GPT-4 outperformed BERT in 5/6 months"]
  G1 --> G3["Peak accuracy: 71.47% (May 2017)"]
  G1 --> G4["Better than Buy-and-Hold strategy"]
  G1 --> G5["Prompt engineering is crucial for performance"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a novel prompt engineering method for contextual sentiment analysis using GPT-4 to predict same‑day stock price movements.

Source sentence
We develop a novel method to engineer a comprehensive prompt for contextual sentiment analysis which unlocks the true capabilities of modern LLM.

Assumes 3

  • A higher correlation between investor sentiment and same‑day stock return movements indicates a better interpretation of the social media feed. model
    Source sentence
    In our study, we assume that a higher correlation between investor sentiment and same-day stock return movements indicates a better interpretation of the social media feed.
  • Predicting same‑day movements captures the most immediate and potent effects of sentiment. other
    Source sentence
    By predicting same-day movements, we capture the most immediate and potent effects of sentiment.
  • Apple and Tesla markets are quite efficient, making same‑day reaction modeling feasible. market
    Source sentence
    Given that we have employed large and closely monitored companies like Apple and Tesla, we assume that the markets are quite efficient in these domains.

Data

  • Daily adjusted closing prices market-prices unknown
    2017 · Apple and Tesla
    Source sentence
    We recorded daily adjusted closing prices and translated them into up-down movements.
  • Stocktwits microblogging messages · Stocktwits text-news public
    2017 · Apple and Tesla
    Source sentence
    Sentiment for each day was extracted from messages on the Stocktwits platform using both LLMs.

Recognised sources: Twitter / X StockTwits

Costs

Not applicable The paper reports no trading or execution result; the cost the extractor found is a model parameter.

The paper-text extraction found a cost parameter inside a model (Costs of deploying GPT-4 are mentioned as a practical consideration but not quantified in the trading results.); the classifier found no reported trading result, so costs are not applicable.

Finds 1

  • GPT-4 achieved higher accuracy than BERT in five of six months and reached a peak accuracy of 71.47% in May, substantially exceeding a naive buy‑and‑hold strategy. predictive-accuracy
    Source sentence
    As a result, GPT-4 exhibited substantial accuracy, outperforming BERT in five out of six months and substantially exceeding a naive buy-and-hold strategy, reaching a peak accuracy of 71.47 % in May.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 3 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 4 papers in the archive (4 by independent authors)

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.