Paper: arXiv 2409.11408

Authors: Baptiste Lefort, Eric Benhamou, Jean-Jacques Ohana, David Saltiel, Beatrice Guez

Abstract

In this paper, we demonstrate that non-generative, small-sized models such as FinBERT and FinDRoBERTa, when fine-tuned, can outperform GPT-3.5 and GPT-4 models in zero-shot learning settings in sentiment analysis for financial news. These fine-tuned models show comparable results to GPT-3.5 when it is fine-tuned on the task of determining market sentiment from daily financial news summaries sourced from Bloomberg. To fine-tune and compare these models, we created a novel database, which assigns a market score to each piece of news without human interpretation bias, systematically identifying the mentioned companies and analyzing whether their stocks have gone up, down, or remained neutral. Furthermore, the paper shows that the assumptions of Condorcet’s Jury Theorem do not hold suggesting that fine-tuned small models are not independent of the fine-tuned GPT models, indicating behavioural similarities. Lastly, the resulted fine-tuned models are made publicly available on HuggingFace, providing a resource for further research in financial sentiment analysis and text classification.

Complexity vs Empirical Score

  • Math Complexity: 4.0/10
  • Empirical Rigor: 7.0/10
  • Quadrant: Street Traders — empirical and applied, lower math complexity

Why this score: The paper introduces a novel market-based dataset, fine-tunes multiple models, and makes models publicly available, indicating significant empirical rigor. While it references the Condorcet Jury Theorem for conceptual analysis, the core contribution is methodological and data-driven rather than advancing complex mathematical theory.

Research Flowchart

  flowchart TD
  A["Research Goal: Assess fine-tuned small models vs. GPT in financial sentiment analysis"] --> B["Create Novel Dataset<br>Bloomberg News with<br>objective market scores"]
  B --> C["Methodology: Fine-tuning<br>FinBERT, FinDRoBERTa, GPT-3.5"]
  C --> D["Comparative Analysis<br>Zero-shot vs. Fine-tuned performance"]
  D --> E{"Key Findings & Outcomes"}
  E --> F["Fine-tuned small models outperform<br>zero-shot GPT-3.5/4"]
  E --> G["Small models achieve<br>comparable results to fine-tuned GPT"]
  E --> H["Models publicly available on<br>HuggingFace"]
  E --> I["Condorcet's Jury Theorem assumptions<br>do not hold<br>Models show behavioral similarities"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a market‑based dataset and fine‑tunes compact models (FinBERT, FinDRoBERTa) to outperform GPT‑3.5/4 on financial sentiment classification.

Source sentence
We fine‑tune Fin‑BERT, FinDRoBERTa and GPT‑3.5 on the market‑based dataset.

Assumes 4

  • The market‑based labels reflect the true market reaction, not human annotator bias. data
    Source sentence
    The label must reflect the market’s true behavior, not just the human annotator’s assessment.
  • All individual classifiers are better than random guessing. statistical
    Source sentence
    The key assumption beyond independence is that all the individual classifiers, Cn, are a better than random guessing.
  • Models are independent estimators for Condorcet’s theorem to hold. statistical
    Source sentence
    If several independent estimators, that are better than random decision, are combined into a majority vote classifier
  • Fine‑tuning uses identical parameters and the same dataset for all models. model
    Source sentence
    All models underwent fine‑tuning with identical parameters and the same dataset.

Data

Backtest sample: main equity markets · equities · daily

  • market‑based dataset (financial_headlines_market_based) · Bloomberg (via HuggingFace) text-news public
    2010‑2024
    Source sentence
    we developed a new and large dataset derived from all major Bloomberg news from 2010 to 2024 that assigns market reaction to news headlines while eliminating human bias

Recognised sources: Bloomberg

Costs

Not applicable The paper reports no trading or execution result.

Finds 3

  • Compact, non‑generative models fine‑tuned on the market‑based dataset surpass the zero‑shot capabilities of GPT‑3.5 and GPT‑4 in financial sentiment analysis. performance
    Source sentence
    we illustrate that with proper fine‑tuning, compact, non‑generative models such as FinBERT and FinDRoBERTa are able to surpass the zero‑shot learning capabilities of larger models like GPT‑3.5 and GPT‑4
  • Bagging (majority‑vote) does not improve performance, indicating the models are not independent. statistical
    Source sentence
    The bagging experiment demonstrates that there is no significant improvement, regardless of whether the models are fine‑tuned or not.
  • The independence condition of Condorcet’s theorem is violated, suggesting similar decision‑making patterns across models of different sizes. theoretical
    Source sentence
    Since the first condition is met, the failure to observe improvement suggests that the second condition is not fulfilled.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 18 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 1 paper in the archive

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (43)

  1. FinBERT: Financial Sentiment Analysis with Pre-trained Language Models (2019) — link
  2. FinBERT: A Pretrained Language Model for Financial Communications (2020) — link
  3. Big Data: Deep Learning for Financial Sentiment Analysis (2018)
  4. Deep Learning for Financial Sentiment Analysis on Finance News Providers (2016)
  5. BloombergGPT: A Large Language Model for Finance (2023) — link
  6. FinGPT: Open-Source Financial Large Language Models — → 2306.06031
  7. Extending Context Window of Large Language Models via Positional Interpolation (2023) — link
  8. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (2023) — link
and 35 more
  1. Prompt-Based Meta-Learning For Few-shot Text Classification (2022) — link
  2. Evaluation of Sentiment Analysis in Finance: From Lexicons to Transformers (2020) — link
  3. The Impact of Sentiment Indices on the Stock Exchange—The Connections between Quantitative Sentiment Indicators, Technical Analysis, and Stock Market (2023) — link
  4. Stock market reaction to news: Do tense and horizon matter? (2023) — link
  5. Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Texts (2013) — link
  6. Can ChatGPT Decipher Fedspeak? (2023) — link
  7. How to Learn and Teach Economics with Large Language Models, Including GPT (2023) — link
  8. Language Models and Cognitive Automation for Economic Research (2023) — link
  9. Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models — → 2304.07619
  10. Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence (2023) — link
  11. Can ChatGPT Compute Trustworthy Sentiment Scores from Bloomberg Market Wraps? — → 2401.05447
  12. Revolutionizing Finance with LLMs: An Overview of Applications and Insights (2024) — link
  13. FinQA: A Dataset of Numerical Reasoning over Financial Data (2022) — link
  14. Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Models — → 2310.04027
  15. Multimodal Gen-AI for Fundamental Investment Research — → 2401.06164
  16. Context-Based Interpretation of Financial Information (2023) — link
  17. Making LLMs Worth Every Penny: Resource-Limited Text Classification in Banking (2023) — link
  18. Sentiment Analysis in the Era of Large Language Models: A Reality Check (2023) — link
  19. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2021) — link
  20. On the probability of the Condorcet Jury Theorem or the Miracle of Aggregation (2022)
  21. Information aggregation, rationality, and the Condorcet jury theorem (1996)
  22. Essai sur l’application de l’analyse à la probabilité des décisions rendues à la pluralité des voix (1785)
  23. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter (2020) — link
  24. Optimal decision rules in uncertain dichotomous choice situations (1982)
  25. Generative AI Text Classification using Ensemble LLM Approaches (2023) — link
  26. Essai sur l’application de l’analyse à la probabilité des décisions rendues à la pluralité des voix (1785)
  27. New Bounds on the Accuracy of Majority Voting for Multi-Class Classification (2023) — link
  28. Breaking Ties of Plurality Voting in Ensembles of Distributed Neural Network Classifiers Using Soft Max Accumulations (2020)
  29. Probability and Measure (1986)
  30. Foundations of Modern Probability (2002)
  31. GPT-NER: Named Entity Recognition via Large Language Models (2023) — link
  32. FinBERT: Financial Sentiment Analysis with Pre-trained Language Models (2021)
  33. DistilROBERTA fine-tuned on Financial News for Sentiment Analysis (2021)
  34. Stress index strategy enhanced with financial news sentiment analysis for the equity markets — → 2404.00012
  35. Good debt or bad debt: Detecting semantic orientations in economic texts (2014)

6 of 43 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.