Paper: arXiv 2412.06837

Authors: Olamilekan Shobayo, Sidikat Adeyemi-Longe, Olusogo Popoola, Bayode Ogunleye

Abstract

This study explores the comparative performance of cutting-edge AI models, i.e., Finaance Bidirectional Encoder representations from Transsformers (FinBERT), Generatice Pre-trained Transformer GPT-4, and Logistic Regression, for sentiment analysis and stock index prediction using financial news and the NGX All-Share Index data label. By leveraging advanced natural language processing models like GPT-4 and FinBERT, alongside a traditional machine learning model, Logistic Regression, we aim to classify market sentiment, generate sentiment scores, and predict market price movements. This research highlights global AI advancements in stock markets, showcasing how state-of-the-art language models can contribute to understanding complex financial data. The models were assessed using metrics such as accuracy, precision, recall, F1 score, and ROC AUC. Results indicate that Logistic Regression outperformed the more computationally intensive FinBERT and predefined approach of versatile GPT-4, with an accuracy of 81.83% and a ROC AUC of 89.76%. The GPT-4 predefined approach exhibited a lower accuracy of 54.19% but demonstrated strong potential in handling complex data. FinBERT, while offering more sophisticated analysis, was resource-demanding and yielded a moderate performance. Hyperparameter optimization using Optuna and cross-validation techniques ensured the robustness of the models. This study highlights the strengths and limitations of the practical applications of AI approaches in stock market prediction and presents Logistic Regression as the most efficient model for this task, with FinBERT and GPT-4 representing emerging tools with potential for future exploration and innovation in AI-driven financial analytics

Complexity vs Empirical Score

  • Math Complexity: 3.5/10
  • Empirical Rigor: 7.0/10
  • Quadrant: Street Traders — empirical and applied, lower math complexity

Why this score: The paper relies heavily on empirical methods like model training, hyperparameter optimization with Optuna, cross-validation, and standard performance metrics (accuracy, AUC-ROC), but its mathematical depth is limited to basic machine learning concepts and statistical evaluation without advanced derivations or novel theoretical frameworks.

Research Flowchart

  flowchart TD
  A["Research Goal<br>Compare FinBERT, GPT-4, and Logistic Regression<br>for sentiment analysis & stock price prediction"] --> B["Data Input<br>Financial News & NGX All-Share Index"]

  subgraph C ["Computational Processing"]
      direction TB
      C1["FinBERT<br>Resource-Demanding<br>Moderate Performance"]
      C2["GPT-4 Predefined<br>Versatile but Lower Accuracy"]
      C3["Logistic Regression<br>Hyperparameter Optimization via Optuna"]
  end

  B --> C
  C1 --> D["Model Evaluation<br>Accuracy, Precision, Recall, F1, ROC AUC"]
  C2 --> D
  C3 --> D

  D --> E["Key Findings & Outcomes"]
  E --> E1["Logistic Regression<br>Best Performance: 81.83% Acc, 89.76% ROC AUC"]
  E --> E2["GPT-4<br>Low Accuracy (54.19%) but Complex Data Potential"]
  E --> E3["FinBERT<br>Sophisticated but Resource-Intensive"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a comparative evaluation of FinBERT, GPT‑4 and Logistic Regression for sentiment analysis and stock index prediction, showing Logistic Regression as the most efficient model.

Source sentence
Results indicate that Logistic Regression outperformed the more computationally intensive FinBERT and predefined approach of versatile GPT-4, with an accuracy of 81.83% and a ROC AUC of 89.76%.

Assumes 5

  • Temporal order of the news data must be preserved during model training and evaluation. data
    Source sentence
    Time series cross-validation (TSCV) was used to train and validate the model with folds (n = 5) on the news dataset. This method maintains the time dependency between data points.
  • The dataset is split into fixed proportions for training, validation and testing. data
    Source sentence
    The preprocessed news headlines datasets were split into 70% for training, 15% for validation, and 15% for testing.
  • TF‑IDF vectorization provides suitable features for Logistic Regression on financial news text. model
    Source sentence
    Logistic Regression: TF‑IDF (Term Frequency–Inverse Document Frequency) vectorization was used to transform the cleaned text data into features.
  • Hyperparameter optimization via Optuna yields robust model settings. model
    Source sentence
    Hyperparameter optimization using Optuna and cross‑validation techniques ensured the robustness of the models.
Show 1 more
  • Financial news sentiment can be captured by three classes (positive, negative, neutral). data
    Source sentence
    These models can classify sentiment (e.g., positive, negative, neutral) and predict how such sentiment may influence stock prices.

Data

  • NGX All‑Share Index data label market-prices unknown
    Source sentence
    Each model was trained on historical NGX All Share Index data labels and financial news

Costs

Not applicable The paper reports no trading or execution result.

Finds 5

  • Logistic Regression achieved an accuracy of 81.83% and ROC AUC of 89.76%, outperforming FinBERT and GPT‑4. performance
    Source sentence
    Results indicate that Logistic Regression outperformed the more computationally intensive FinBERT and predefined approach of versatile GPT-4, with an accuracy of 81.83% and a ROC AUC of 89.76%.
  • GPT‑4 predefined approach obtained a lower accuracy of 54.19% on the same task. performance
    Source sentence
    The GPT‑4 predefined approach exhibited a lower accuracy of 54.19% but demonstrated strong potential in handling complex data.
  • FinBERT delivered moderate performance and was resource‑demanding. performance
    Source sentence
    FinBERT, while offering more sophisticated analysis, was resource‑demanding and yielded a moderate performance.
  • Logistic Regression generalizes well, with training accuracy 80.93% and test accuracy 81.83%, indicating no significant overfitting. statistical
    Source sentence
    the model test accuracy is very close to the training accuracy of 80.93%, which indicates that the model generalizes well and there is no significant overfitting.
  • Advanced NLP models incur higher computational costs, limiting their real‑time applicability. other
    Source sentence
    FinBERT, while better equipped to handle financial language, faced challenges in terms of computational demands and resource usage, which limited its practical application in real‑time prediction scenarios.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 43 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 2 papers in the archive (2 by independent authors)

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.