Paper: arXiv 2407.01953
Authors: Yupeng Cao, Zhiyuan Yao, Zhi Chen, Zhiyang Deng
Abstract
The integration of Large Language Models (LLMs) into financial analysis has garnered significant attention in the NLP community. This paper presents our solution to IJCAI-2024 FinLLM challenge, investigating the capabilities of LLMs within three critical areas of financial tasks: financial classification, financial text summarization, and single stock trading. We adopted Llama3-8B and Mistral-7B as base models, fine-tuning them through Parameter Efficient Fine-Tuning (PEFT) and Low-Rank Adaptation (LoRA) approaches. To enhance model performance, we combine datasets from task 1 and task 2 for data fusion. Our approach aims to tackle these diverse tasks in a comprehensive and integrated manner, showcasing LLMs’ capacity to address diverse and complex financial tasks with improved accuracy and decision-making capabilities.
Complexity vs Empirical Score
- Math Complexity: 1.5/10
- Empirical Rigor: 6.0/10 (capped from 8.0: transaction costs not modelled; see penalty flags)
- Quadrant: Street Traders — empirical and applied, lower math complexity (effective empirical rigor capped from 8.0)
Why this score: The paper is heavily implementation-focused, detailing specific model architectures (Llama3-8B, Mistral-7B), parameter-efficient fine-tuning (PEFT/LoRA), hardware specifications (RTX-A6000 GPUs), and data fusion strategies with clear validation metrics; however, it contains almost no advanced mathematical derivations or theoretical proofs, relying instead on empirical testing.
Research Flowchart
flowchart TD
A["Research Goal:<br>Evaluate LLMs in Financial Analysis"] --> B{"Base Models:<br>Llama3-8B & Mistral-7B"}
B --> C{"Data Fusion Strategy"}
C --> D["Parameter Efficient<br>Fine-Tuning PEFT"]
D --> E["Low-Rank<br>Adaptation LoRA"]
E --> F["Task Execution:<br>Classify, Summarize, Trade"]
F --> G["Outcomes:<br>Enhanced Accuracy &<br>Financial Decision Making"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper fine‑tunes Llama3‑8B and Mistral‑7B with PEFT and LoRA, using data fusion of task‑1 and task‑2 datasets to address classification, summarization, and stock‑trading tasks.
Source sentence
We adopted Llama3-8B and Mistral-7B as base models, fine-tuning them through Parameter Efficient Fine-Tuning (PEFT) and Low-Rank Adaptation (LoRA) approaches. To enhance model performance, we combine datasets from task 1 and task 2 for data fusion.
Assumes 2
- The task‑3 dataset is excluded because its narrow company‑specific content would not benefit the model. data
Source sentence
We excluded the dataset for task 3, which focuses on texts related to three specific stocks, due to its narrow company-specific content.
- Models are fine‑tuned for only one epoch due to limited computational resources. execution
Source sentence
The models were trained and inferenced on two NVIDIA RTX-A6000 GPUs (each has 48GB DRAM) with one epoch.
Data
- Task 1 dataset text-news unknown
Source sentence
We combine datasets from task 1 and task 2 for data fusion.
- Task 2 dataset text-news unknown
Source sentence
We combine datasets from task 1 and task 2 for data fusion.
Costs
Not modeled
Finds 5
- Fine‑tuned Mistral‑7B on the fused dataset achieved a 0.5634 F1 score on the classification task (Task 1). performance
Source sentence
where it achieves a 0.5634 F1 score.
- On the summarization task (Task 2), the fused model improved Rouge‑1 to 0.4920, Rouge‑2 to 0.3015, and BertScore to 0.6946. performance
Source sentence
Task 1 + Task 2 0.4920 0.3015 0.6946
- Mistral‑7B demonstrated superior predictive capabilities over Llama3‑8B, leading to its exclusive use in subsequent experiments. performance
Source sentence
Mistral-7B demonstrated superior predictive capabilities and produced well-formatted outputs that could be easily parsed to yield final predictions.
- On the stock‑trading task (Task 3), the integrated Sharpe Ratio was -0.6199, indicating poor performance. negative
Source sentence
In Task 3, the integrated Sharp Ratio (SR) was -0.6199.
- In the test set, the fine‑tuned model achieved ACC 0.711, F1 0.4199, and MCC 0.6818 on Task 1. performance
Source sentence
In Task 1, the model achieved an ACC of 0.711, an F1 score of 0.4199, and a Matthews correlation coefficient (MCC) of 0.6818.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 24 related papers). Coverage is partial, so this is neither replication nor refutation.
2 papers in the archive : HARLF: Hierarchical Reinforcement Learning and Lightweight …
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (17)
- A survey of fintech research and policy discussion (2021)
- Natural language processing in accounting, auditing and finance: A synthesis of the literature with a roadmap for future research (2016)
- The FinBen: An Holistic Financial Benchmark for Large Language Models (2024) — link
- Large language models in finance: A survey — → 2311.10723
- FinMem: A performance-enhanced LLM trading agent with layered memory and character design — → 2311.13743
- Sparks of artificial general intelligence: Early experiments with gpt-4 (2023) — link
- Fine-grained argument understanding with bert ensemble techniques: A deep dive into financial sentiment analysis (2023)
- PEFT: State-of-the-art Parameter-Efficient Fine-Tuning methods (2022)
and 9 more
- Mistral 7B (2023) — link
- Trade the event: Corporate events detection for news-based event-driven trading (2021) — link
- Gpt-4 technical report (2023) — link
- Llama 3 Model Card (2024) — link
- Gemini: a family of highly capable multimodal models (2023) — link
- Lora: Low-rank adaptation of large language models (2021) — link
- Quantization and training of neural networks for efficient integer-arithmetic-only inference (2018)
- An information fusion based approach to context-based fine-tuning of GPT models (2024)
- Multi-view fusion for instruction mining of large language model (2024)
2 of 17 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.