Financial Text Classification Based On rLoRA Finetuning On Qwen3-8B model

ArXiv ID: 2512.00630 “View on arXiv”

Authors: Zhiming Lian

Abstract

Financial text classification has increasingly become an important aspect in quantitative trading systems and related tasks, such as financial sentiment analysis and the classification of financial news. In this paper, we assess the performance of the large language model Qwen3-8B on both tasks. Qwen3-8B is a state-of-the-art model that exhibits strong instruction-following and multilingual capabilities, and is distinct from standard models, primarily because it is specifically optimized for efficient fine tuning and high performance on reasoning-based benchmarks, making it suitable for financial applications. To adapt this model, we apply Noisy Embedding Instruction Finetuning and based on our previous work, this method increases robustness by injecting controlled noise into the embedding layers during supervised adaptation. We improve efficiency further with Rank-stabilized Low-Rank Adaptation low-rank optimization approach, and FlashAttention, which allow for faster training with lower GPU memory. For both tasks, we benchmark Qwen3-8B against standard classical transformer models, such as T5, BERT, and RoBERTa, and large models at scale, such as LLaMA1-7B, LLaMA2-7B, and Baichuan2-7B. The findings reveal that Qwen3-8B consistently surpasses these baselines by obtaining better classification accuracy and needing fewer training epochs. The synergy of instruction-based fine-tuning and memory-efficient optimization methods suggests Qwen3-8B can potentially serve as a scalable, economical option for real-time financial NLP applications. Qwen3-8B provides a very promising base for advancing dynamic quantitative trading systems in the future.

Keywords: Financial NLP, Large Language Models (LLMs), Quantitative Trading, Sentiment Analysis, Fintech

Complexity vs Empirical Score

• Math Complexity: 4.0/10
• Empirical Rigor: 7.5/10
• Quadrant: Street Traders
• Why: The paper applies standard deep learning architectures (LoRA, Qwen3) with minimal novel mathematical derivations, but is heavily data-driven with explicit benchmarking on financial datasets, hyperparameter tuning, and empirical comparisons against multiple baselines.

  flowchart TD
    Goal["Research Goal: Assess Qwen3-8B for Financial Text Classification"]
    
    SubMethod["Methodology<br>Noisy Embedding Instruction Finetuning<br>Rank-stabilized Low-Rank Adaptation<br>FlashAttention"]
    
    Inputs["Data/Inputs<br>Financial Sentiment Analysis &<br>Financial News Classification Tasks"]
    
    Processes["Computational Processes<br>Qwen3-8B Fine-tuning<br>vs. BERT, RoBERTa, T5, LLaMA, Baichuan2"]
    
    Outcomes["Key Findings<br>Qwen3-8B surpasses baselines<br>Fewer epochs needed<br>Scalable & economical solution"]
    
    Goal --> SubMethod
    SubMethod --> Inputs
    Inputs --> Processes
    Processes --> Outcomes