Paper: arXiv 2412.06837
Authors: Olamilekan Shobayo, Sidikat Adeyemi-Longe, Olusogo Popoola, Bayode Ogunleye
Abstract
This study explores the comparative performance of cutting-edge AI models, i.e., Finaance Bidirectional Encoder representations from Transsformers (FinBERT), Generatice Pre-trained Transformer GPT-4, and Logistic Regression, for sentiment analysis and stock index prediction using financial news and the NGX All-Share Index data label. By leveraging advanced natural language processing models like GPT-4 and FinBERT, alongside a traditional machine learning model, Logistic Regression, we aim to classify market sentiment, generate sentiment scores, and predict market price movements. This research highlights global AI advancements in stock markets, showcasing how state-of-the-art language models can contribute to understanding complex financial data. The models were assessed using metrics such as accuracy, precision, recall, F1 score, and ROC AUC. Results indicate that Logistic Regression outperformed the more computationally intensive FinBERT and predefined approach of versatile GPT-4, with an accuracy of 81.83% and a ROC AUC of 89.76%. The GPT-4 predefined approach exhibited a lower accuracy of 54.19% but demonstrated strong potential in handling complex data. FinBERT, while offering more sophisticated analysis, was resource-demanding and yielded a moderate performance. Hyperparameter optimization using Optuna and cross-validation techniques ensured the robustness of the models. This study highlights the strengths and limitations of the practical applications of AI approaches in stock market prediction and presents Logistic Regression as the most efficient model for this task, with FinBERT and GPT-4 representing emerging tools with potential for future exploration and innovation in AI-driven financial analytics
Complexity vs Empirical Score
- Math Complexity: 3.5/10
- Empirical Rigor: 7.0/10
- Quadrant: Street Traders — empirical and applied, lower math complexity
Why this score: The paper relies heavily on empirical methods like model training, hyperparameter optimization with Optuna, cross-validation, and standard performance metrics (accuracy, AUC-ROC), but its mathematical depth is limited to basic machine learning concepts and statistical evaluation without advanced derivations or novel theoretical frameworks.
Research Flowchart
flowchart TD
A["Research Goal<br>Compare FinBERT, GPT-4, and Logistic Regression<br>for sentiment analysis & stock price prediction"] --> B["Data Input<br>Financial News & NGX All-Share Index"]
subgraph C ["Computational Processing"]
direction TB
C1["FinBERT<br>Resource-Demanding<br>Moderate Performance"]
C2["GPT-4 Predefined<br>Versatile but Lower Accuracy"]
C3["Logistic Regression<br>Hyperparameter Optimization via Optuna"]
end
B --> C
C1 --> D["Model Evaluation<br>Accuracy, Precision, Recall, F1, ROC AUC"]
C2 --> D
C3 --> D
D --> E["Key Findings & Outcomes"]
E --> E1["Logistic Regression<br>Best Performance: 81.83% Acc, 89.76% ROC AUC"]
E --> E2["GPT-4<br>Low Accuracy (54.19%) but Complex Data Potential"]
E --> E3["FinBERT<br>Sophisticated but Resource-Intensive"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes a comparative evaluation of FinBERT, GPT‑4 and Logistic Regression for sentiment analysis and stock index prediction, showing Logistic Regression as the most efficient model.
Source sentence
Results indicate that Logistic Regression outperformed the more computationally intensive FinBERT and predefined approach of versatile GPT-4, with an accuracy of 81.83% and a ROC AUC of 89.76%.
Assumes 5
- Temporal order of the news data must be preserved during model training and evaluation. data
Source sentence
Time series cross-validation (TSCV) was used to train and validate the model with folds (n = 5) on the news dataset. This method maintains the time dependency between data points.
- The dataset is split into fixed proportions for training, validation and testing. data
Source sentence
The preprocessed news headlines datasets were split into 70% for training, 15% for validation, and 15% for testing.
- TF‑IDF vectorization provides suitable features for Logistic Regression on financial news text. model
Source sentence
Logistic Regression: TF‑IDF (Term Frequency–Inverse Document Frequency) vectorization was used to transform the cleaned text data into features.
- Hyperparameter optimization via Optuna yields robust model settings. model
Source sentence
Hyperparameter optimization using Optuna and cross‑validation techniques ensured the robustness of the models.
Show 1 more
- Financial news sentiment can be captured by three classes (positive, negative, neutral). data
Source sentence
These models can classify sentiment (e.g., positive, negative, neutral) and predict how such sentiment may influence stock prices.
Data
- NGX All‑Share Index data label market-prices unknown
Source sentence
Each model was trained on historical NGX All Share Index data labels and financial news
Costs
Not applicable
Finds 5
- Logistic Regression achieved an accuracy of 81.83% and ROC AUC of 89.76%, outperforming FinBERT and GPT‑4. performance
Source sentence
Results indicate that Logistic Regression outperformed the more computationally intensive FinBERT and predefined approach of versatile GPT-4, with an accuracy of 81.83% and a ROC AUC of 89.76%.
- GPT‑4 predefined approach obtained a lower accuracy of 54.19% on the same task. performance
Source sentence
The GPT‑4 predefined approach exhibited a lower accuracy of 54.19% but demonstrated strong potential in handling complex data.
- FinBERT delivered moderate performance and was resource‑demanding. performance
Source sentence
FinBERT, while offering more sophisticated analysis, was resource‑demanding and yielded a moderate performance.
- Logistic Regression generalizes well, with training accuracy 80.93% and test accuracy 81.83%, indicating no significant overfitting. statistical
Source sentence
the model test accuracy is very close to the training accuracy of 80.93%, which indicates that the model generalizes well and there is no significant overfitting.
- Advanced NLP models incur higher computational costs, limiting their real‑time applicability. other
Source sentence
FinBERT, while better equipped to handle financial language, faced challenges in terms of computational demands and resource usage, which limited its practical application in real‑time prediction scenarios.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 43 related papers). Coverage is partial, so this is neither replication nor refutation.
2 papers in the archive
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.