Paper: arXiv 2412.00549
Authors: Jebish Purbey, Siddhant Gupta, Nikhil Manali, Siddartha Pullakhandam, Drishti Sharma, Ashay Srivastava, Ram Mohan Rao Kadiyala
Abstract
This paper presents the system description of our entry for the COLING 2025 FMD challenge, focusing on misinformation detection in financial domains. We experimented with a combination of large language models, including Qwen, Mistral, and Gemma-2, and leveraged pre-processing and sequential learning for not only identifying fraudulent financial content but also generating coherent, and concise explanations that clarify the rationale behind the classifications. Our approach achieved competitive results with an F1-score of 0.8283 for classification, and ROUGE-1 of 0.7253 for explanations. This work highlights the transformative potential of LLMs in financial applications, offering insights into their capabilities for combating misinformation and enhancing transparency while identifying areas for future improvement in robustness and domain adaptation.
Complexity vs Empirical Score
- Math Complexity: 1.5/10
- Empirical Rigor: 7.5/10
- Quadrant: Street Traders — empirical and applied, lower math complexity
Why this score: The paper uses standard NLP metrics (F1, ROUGE) and detailed fine-tuning procedures with specific datasets and model configurations, indicating strong empirical implementation. The math is primarily focused on model application rather than novel theoretical developments.
Research Flowchart
flowchart TD
Start["Research Goal:<br>Financial Misinformation Detection<br>& Explanation Generation"] --> Methodology
subgraph Methodology ["Key Methodology"]
Data["Input: Financial Claims & Dataset<br>(FMD Challenge)"]
Proc["Sequential Learning Pipeline:<br>Preprocessing & Sequential Processing"]
Models["Model Ensemble:<br>Qwen, Mistral, Gemma-2"]
Data --> Proc --> Models
end
Models --> Computation["Computational Process:<br>Claim Verification &<br>Coherent Explanation Generation"]
Computation --> Outcomes["Key Outcomes:<br>Class. F1: 0.8283 | Expl. ROUGE-1: 0.7253<br>LLM Potential for Financial Transparency"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes a sequential fine‑tuning strategy (SeQwen) that first fine‑tunes open‑source LLMs for classification and then adapts them to generate explanations for financial misinformation detection.
Source sentence
Our approach achieved competitive results with an F1-score of 0.8283 for classification, and ROUGE-1 of 0.7253 for explanations.
Assumes
No explicit assumptions found in the text.
Data
- FIN-FACT benchmark unknown
Source sentence
The task leverages the FIN-FACT (Rangapur et al., 2024) dataset which includes claims categorized as True, False, or Not Enough Information (NEI) across diverse sectors
Costs
Not applicable
Finds 3
- SeQwen achieved a Micro F1 score of 0.8283 on the test set, a 1.4% relative gain over the baseline Qwen2.5 7B model fine‑tuned for 5 epochs. performance
Source sentence
Micro F1 score increasing from 0.8165 to 0.8283, representing a 1.4% relative gain.
- Explanation quality improved markedly, with ROUGE‑1 rising from 0.6337 to 0.7253 (14.5% increase), ROUGE‑2 from 0.5652 to 0.6763 (19.7% increase), and ROUGE‑L from 0.5885 to 0.6911 (17.4% increase). performance
Source sentence
ROUGE-1 rose from 0.6337 to 0.7253 (a 14.5% increase), ROUGE-2 increased from 0.5652 to 0.6763 (19.7% gain), and ROUGE-L improved from 0.5885 to 0.6911 (17.4% increase).
- The overall combined score improved from 0.7251 to 0.7768, a 7.1% improvement, highlighting the benefit of sequential fine‑tuning for both tasks. performance
Source sentence
overall score improved from 0.7251 to 0.7768, reflecting a 7.1% improvement
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 16 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (12)
- When FLUE Meets FLANG : Benchmarks and Large Pretrained Language Model for Financial Domain (2022) — link
- BloombergGPT: A Large Language Model for Finance (2023) — link
- PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance (2023) — link
- FMDLlama: Financial Misinformation Detection based on Large Language Models (2024) — link
- Fin-Fact: A Benchmark Dataset for Multimodal Financial Fact Checking and Explanation Generation (2024) — link
- LoRA: Low-Rank Adaptation of Large Language Models (2021) — link
- Qwen2.5: A Party of Foundation Models (2024) — link
- The Llama 3 Herd of Models (2024) — link
and 4 more
0 of 12 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.