Paper: arXiv 2407.01953

Authors: Yupeng Cao, Zhiyuan Yao, Zhi Chen, Zhiyang Deng

Abstract

The integration of Large Language Models (LLMs) into financial analysis has garnered significant attention in the NLP community. This paper presents our solution to IJCAI-2024 FinLLM challenge, investigating the capabilities of LLMs within three critical areas of financial tasks: financial classification, financial text summarization, and single stock trading. We adopted Llama3-8B and Mistral-7B as base models, fine-tuning them through Parameter Efficient Fine-Tuning (PEFT) and Low-Rank Adaptation (LoRA) approaches. To enhance model performance, we combine datasets from task 1 and task 2 for data fusion. Our approach aims to tackle these diverse tasks in a comprehensive and integrated manner, showcasing LLMs’ capacity to address diverse and complex financial tasks with improved accuracy and decision-making capabilities.

Complexity vs Empirical Score

  • Math Complexity: 1.5/10
  • Empirical Rigor: 6.0/10 (capped from 8.0: transaction costs not modelled; see penalty flags)
  • Quadrant: Street Traders — empirical and applied, lower math complexity (effective empirical rigor capped from 8.0)

Why this score: The paper is heavily implementation-focused, detailing specific model architectures (Llama3-8B, Mistral-7B), parameter-efficient fine-tuning (PEFT/LoRA), hardware specifications (RTX-A6000 GPUs), and data fusion strategies with clear validation metrics; however, it contains almost no advanced mathematical derivations or theoretical proofs, relying instead on empirical testing.

Research Flowchart

  flowchart TD
  A["Research Goal:<br>Evaluate LLMs in Financial Analysis"] --> B{"Base Models:<br>Llama3-8B & Mistral-7B"}
  B --> C{"Data Fusion Strategy"}
  C --> D["Parameter Efficient<br>Fine-Tuning PEFT"]
  D --> E["Low-Rank<br>Adaptation LoRA"]
  E --> F["Task Execution:<br>Classify, Summarize, Trade"]
  F --> G["Outcomes:<br>Enhanced Accuracy &<br>Financial Decision Making"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper fine‑tunes Llama3‑8B and Mistral‑7B with PEFT and LoRA, using data fusion of task‑1 and task‑2 datasets to address classification, summarization, and stock‑trading tasks.

Source sentence
We adopted Llama3-8B and Mistral-7B as base models, fine-tuning them through Parameter Efficient Fine-Tuning (PEFT) and Low-Rank Adaptation (LoRA) approaches. To enhance model performance, we combine datasets from task 1 and task 2 for data fusion.

Assumes 2

  • The task‑3 dataset is excluded because its narrow company‑specific content would not benefit the model. data
    Source sentence
    We excluded the dataset for task 3, which focuses on texts related to three specific stocks, due to its narrow company-specific content.
  • Models are fine‑tuned for only one epoch due to limited computational resources. execution
    Source sentence
    The models were trained and inferenced on two NVIDIA RTX-A6000 GPUs (each has 48GB DRAM) with one epoch.

Data

  • Task 1 dataset text-news unknown
    Source sentence
    We combine datasets from task 1 and task 2 for data fusion.
  • Task 2 dataset text-news unknown
    Source sentence
    We combine datasets from task 1 and task 2 for data fusion.

Costs

Not modeled A trading or execution result is reported without costs.

Finds 5

  • Fine‑tuned Mistral‑7B on the fused dataset achieved a 0.5634 F1 score on the classification task (Task 1). performance
    Source sentence
    where it achieves a 0.5634 F1 score.
  • On the summarization task (Task 2), the fused model improved Rouge‑1 to 0.4920, Rouge‑2 to 0.3015, and BertScore to 0.6946. performance
    Source sentence
    Task 1 + Task 2 0.4920 0.3015 0.6946
  • Mistral‑7B demonstrated superior predictive capabilities over Llama3‑8B, leading to its exclusive use in subsequent experiments. performance
    Source sentence
    Mistral-7B demonstrated superior predictive capabilities and produced well-formatted outputs that could be easily parsed to yield final predictions.
  • On the stock‑trading task (Task 3), the integrated Sharpe Ratio was -0.6199, indicating poor performance. negative
    Source sentence
    In Task 3, the integrated Sharp Ratio (SR) was -0.6199.
  • In the test set, the fine‑tuned model achieved ACC 0.711, F1 0.4199, and MCC 0.6818 on Task 1. performance
    Source sentence
    In Task 1, the model achieved an ACC of 0.711, an F1 score of 0.4199, and a Matthews correlation coefficient (MCC) of 0.6818.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 24 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 2 papers in the archive (2 by independent authors): HARLF: Hierarchical Reinforcement Learning and Lightweight … (uses its method or data)

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (17)

  1. A survey of fintech research and policy discussion (2021)
  2. Natural language processing in accounting, auditing and finance: A synthesis of the literature with a roadmap for future research (2016)
  3. The FinBen: An Holistic Financial Benchmark for Large Language Models (2024) — link
  4. Large language models in finance: A survey — → 2311.10723
  5. FinMem: A performance-enhanced LLM trading agent with layered memory and character design — → 2311.13743
  6. Sparks of artificial general intelligence: Early experiments with gpt-4 (2023) — link
  7. Fine-grained argument understanding with bert ensemble techniques: A deep dive into financial sentiment analysis (2023)
  8. PEFT: State-of-the-art Parameter-Efficient Fine-Tuning methods (2022)
and 9 more
  1. Mistral 7B (2023) — link
  2. Trade the event: Corporate events detection for news-based event-driven trading (2021) — link
  3. Gpt-4 technical report (2023) — link
  4. Llama 3 Model Card (2024) — link
  5. Gemini: a family of highly capable multimodal models (2023) — link
  6. Lora: Low-rank adaptation of large language models (2021) — link
  7. Quantization and training of neural networks for efficient integer-arithmetic-only inference (2018)
  8. An information fusion based approach to context-based fine-tuning of GPT models (2024)
  9. Multi-view fusion for instruction mining of large language model (2024)

2 of 17 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.