Paper: arXiv 2310.04027

Authors: Boyu Zhang, Hongyang Yang, Tianyu Zhou, Ali Babar, Xiao-Yang Liu

Abstract

Financial sentiment analysis is critical for valuation and investment decision-making. Traditional NLP models, however, are limited by their parameter size and the scope of their training datasets, which hampers their generalization capabilities and effectiveness in this field. Recently, Large Language Models (LLMs) pre-trained on extensive corpora have demonstrated superior performance across various NLP tasks due to their commendable zero-shot abilities. Yet, directly applying LLMs to financial sentiment analysis presents challenges: The discrepancy between the pre-training objective of LLMs and predicting the sentiment label can compromise their predictive performance. Furthermore, the succinct nature of financial news, often devoid of sufficient context, can significantly diminish the reliability of LLMs’ sentiment analysis. To address these challenges, we introduce a retrieval-augmented LLMs framework for financial sentiment analysis. This framework includes an instruction-tuned LLMs module, which ensures LLMs behave as predictors of sentiment labels, and a retrieval-augmentation module which retrieves additional context from reliable external sources. Benchmarked against traditional models and LLMs like ChatGPT and LLaMA, our approach achieves 15% to 48% performance gain in accuracy and F1 score.

Complexity vs Empirical Score

  • Math Complexity: 3.5/10
  • Empirical Rigor: 7.5/10
  • Quadrant: Street Traders — empirical and applied, lower math complexity

Why this score: The paper applies established NLP techniques (RAG, instruction tuning) with minimal novel mathematical derivations, focusing on engineering a practical framework, but is heavily backed by benchmark results (15-48% gains) and comparisons against established models like ChatGPT and LLaMA, demonstrating strong empirical backing.

Research Flowchart

  flowchart TD
  A["Research Goal:<br>Enhance Financial Sentiment Analysis"] --> B{"Core Problem: Traditional NLP &<br>Direct LLM Application Limitations"}
  B --> C["Proposed Method:<br>Retrieval-Augmented LLM Framework"]
  
  subgraph C_Method ["Framework Components"]
      C1["Instruction-Tuned LLM Module<br>Acts as Label Predictor"]
      C2["Retrieval-Augmentation Module<br>Supplements Context from External Sources"]
  end

  C --> C_Method
  
  C_Method --> D["Input Data:<br>Financial News & Market Data"]
  D --> E["Computational Process:<br>Retrieval + LLM Inference"]
  E --> F["Output: Accurate<br>Sentiment Labels"]
  
  F --> G["Key Outcomes"]
  subgraph G_Results ["Benchmark Results"]
      G1["15% to 48% Gain in Accuracy & F1"]
      G2["Outperforms Baselines<br>(Traditional NLP, ChatGPT, LLaMA)"]
  end
  
  G --> G_Results

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a retrieval‑augmented large language model framework for financial sentiment analysis, combining instruction‑tuned LLMs with external knowledge retrieval.

Source sentence
retrieval‑augmented large language model framework tailored for financial sentiment analysis.

Assumes 4

  • Financial news and tweets are concise and often lack adequate background information. data
    Source sentence
    the typical subjects of financial sentiment analysis, such as news flashes and tweets, are characteristically concise and often lack adequate background information.
  • Instruction‑finetuning with a limited set of examples can align LLM predictions with user intentions and boost accuracy. model
    Source sentence
    ... aligning LLMs’ predictions with user intentions and significantly boosting their prediction accuracy.
  • Retrieving additional context based on textual similarity improves sentiment prediction. model
    Source sentence
    the integration of external knowledge retrieval has further enriched the depth and context of the information fed into the LLMs, enabling more nuanced predictions.
  • The approach relies exclusively on textual similarity for retrieval, ignoring macro‑ and micro‑economic data. other
    Source sentence
    a limitation of our approach is its exclusive reliance on textual similarity to retrieve relevant information.

Data

  • Twitter Financial News dataset text-news public
    Source sentence
    dataset ... comprises 9,540 samples for training, each annotated with one of three labels: Bearish, Bullish, or Neutral.
  • FiQA dataset text-news public
    Source sentence
    FiQA dataset: This dataset includes 961 samples. Each sample has been annotated with one of three labels: positive, neutral, or negative.
  • Financial PhraseBank (FPB) dataset benchmark public
    Source sentence
    comprises 4,840 samples randomly extracted from financial news articles

Recognised sources: Financial PhraseBank

Costs

Not applicable The paper reports no trading or execution result.

Finds 1

  • The proposed method achieves a 15% to 48% performance gain in accuracy and F1 score over traditional models and general‑purpose LLMs. performance
    Source sentence
    achieving a 15% to 48% performance gain in accuracy and F1 score.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 8 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 25 papers in the archive (17 by independent authors): Are Large Language Models Good In-context Learners for … (uses its method or data); A Survey of Large Language Models for Financial … (uses its method or data)

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.