Paper: arXiv 2603.19286

Authors: Pei-Jun Liao, Hung-Shin Lee, Yao-Fei Cheng, Li-Wei Chen, Hung-yi Lee, Hsin-Min Wang

Abstract

Predicting stock prices presents challenges in financial forecasting. While traditional approaches such as ARIMA and RNNs are prevalent, recent developments in Large Language Models (LLMs) offer alternative methodologies. This paper introduces an approach that integrates LLMs with daily financial news for stock price prediction. To address the challenge of processing news data and identifying relevant content, we utilize stock name embeddings within attention mechanisms. Specifically, we encode news articles using a pre-trained LLM and implement three attention-based pooling techniques – self-attentive, cross-attentive, and position-aware self-attentive pooling – to filter news based on stock relevance. The filtered news embeddings, combined with historical stock prices, serve as inputs to the prediction model. Unlike prior studies that focus on individual stocks, our method trains a single generalized model applicable across multiple stocks. Experimental results demonstrate a 7.11% reduction in Mean Absolute Error (MAE) compared to the baseline, indicating the utility of stock name embeddings for news filtering and price forecasting within a generalized framework.

Complexity vs Empirical Score

  • Math Complexity: 4.5/10
  • Empirical Rigor: 6.5/10
  • Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper presents a solid empirical integration of Large Language Models (Time-LLM) and textual news data via stock-name-guided attention mechanisms for multi-stock price prediction. While the mathematical formulations are straightforward standard deep learning operations (attention pooling and cross-attention), the empirical design across both Taiwanese and US stock markets is relatively rigorous.

Research Flowchart

  flowchart TD
    A[Goal: Generalized Multi-Stock Prediction via LLM & News Fusion] --> B[Data Inputs: Daily Financial News & Historical Stock Prices]
    B --> C[LLM Encoding & Stock Name Embeddings]
    C --> D[Attention Pooling: Self, Cross, & Position-Aware]
    D --> E[Single Generalized Prediction Model]
    E --> F[Outcome: 7.11% MAE Reduction vs. Baseline]

Cited References (63)

  1. Time- LLM : Time Series Forecasting by Reprogramming Large Language Models (2024)
  2. One Fits All:Power General Time Series Analysis by Pretrained LM (2023)
  3. StockTime : A Time Series Specialized Large Language Model Architecture for Stock Price Prediction — → 2409.08281
  4. AutoTimes : Autoregressive Time Series Forecasters via Large Language Models (2024)
  5. BELT : A Pipeline for Stock Price Prediction Using News (2020)
  6. A CNN - BiLSTM - AM method for stock price prediction (2021)
  7. State-of-the-Art in Stock Prediction Techniques (2013)
  8. Financial statement analysis and the prediction of stock returns (1989)
and 55 more
  1. Short term stock price prediction using deep learning (2017)
  2. A LSTM -based method for stock returns prediction: A case study of China stock market (2015)
  3. Stock Price Prediction Using the ARIMA Model (2014)
  4. Prediction Models for Indian Stock Market (2016)
  5. Stock prediction: Integrating text mining approach using real-time news (2003)
  6. Stock Price Prediction Using News Sentiment Analysis (2019)
  7. Deep Time Series Models: A Comprehensive Survey and Benchmark (2024) — link
  8. Stock Movement Prediction from Tweets and Historical Prices (2018)
  9. A Survey of Large Language Models in Finance ( FinLLMs ) (2024) — link
  10. Hybrid Deep Sequential Modeling for Social Text-Driven Stock Prediction (2018)
  11. Stock Movement and Volatility Prediction from Tweets, Macroeconomic Factors and Historical Prices (2023)
  12. Accurate Stock Movement Prediction with Self-supervised Learning from Sparse Noisy Tweets (2022)
  13. A Survey of Forex and Stock Price Prediction Using Deep Learning (2021)
  14. Comparison of ARIMA and Artificial Neural Networks Models for Stock Price Prediction (2014)
  15. Sentiment analysis for stock price prediction (2020)
  16. Combining Technical Analysis with Sentiment Analysis for Stock Price Prediction (2011)
  17. Stock price prediction using LSTM (2020)
  18. Incorporating Corporation Relationship via Graph Convolutional Neural Networks for Stock Price Prediction (2018)
  19. Graph convolutional networks: a comprehensive review (2019)
  20. Attention is all you need (2017)
  21. Stock Movement Prediction That Integrates Heterogeneous Data Sources Using Dilated Causal Convolution Networks with Attention (2020)
  22. Stock Price Prediction Using CNN and LSTM -Based Deep Learning Models (2020)
  23. Stock price pattern recognition-a recurrent neural network approach (1990)
  24. MASTER : Market-Guided Stock Transformer for Stock Price Forecasting (2024)
  25. FinGPT : Open-Source Financial Large Language Models — → 2306.06031
  26. PIXIU: A Comprehensive Benchmark, Instruction Dataset and Large Language Model for Finance (2023)
  27. ALERTA-Net : A Temporal Distance-Aware Recurrent Networks for Stock Movement and Volatility Prediction (2023)
  28. Ploutos: Towards interpretable stock movement prediction with financial large language model — → 2403.00782
  29. AMA-LSTM : Pioneering Robust and Fair Financial Audio Analysis for Stock Volatility Prediction (2024)
  30. BloombergGPT : A Large Language Model for Finance (2023) — link
  31. StockMixer : A Simple Yet Strong MLP -Based Architecture for Stock Price Forecasting (2024)
  32. Combined deep learning classifiers for stock market prediction: integrating stock price and news sentiments (2023)
  33. CNNpred : CNN -based stock market prediction using a diverse set of variables (2019)
  34. A decoder-only foundation model for time-series forecasting (2024)
  35. BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding (2019)
  36. RoBERTa: A Robustly Optimized BERT Pretraining Approach (2019) — link
  37. Improving Language Understanding by Generative Pre-Training (2018)
  38. Language Models are Unsupervised Multitask Learners (2019)
  39. Language Models are Few-Shot Learners (2020) — link
  40. ChatGPT : A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope (2023)
  41. LLaMA : Open and Efficient Foundation Language Models (2023) — link
  42. LLaMA 2: Open Foundation and Fine-Tuned Chat Models (2023) — link
  43. The LLaMA 3 Herd of Models (2024) — link
  44. A Time Series is Worth 64 Words: Long-term Forecasting with Transformers (2023)
  45. Stock price prediction: comparison of different moving average techniques using deep learning model (2024)
  46. Stock trend prediction using simple moving average supported by news classification (2014)
  47. Time series analysis: forecasting and control (2015)
  48. Self-Attention Encoding and Pooling for Speaker Recognition (2020)
  49. Cross Attentive Pooling for Speaker Verification (2021)
  50. Bidirectional Attention Flow for Machine Comprehension (2016) — link
  51. Learning to Generate Explainable Stock Predictions using Self-Reflective Large Language Models — → 2402.03659
  52. Position-aware Joint Entity and Relation Extraction with Attention Mechanism. (2022)
  53. Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting (2021)
  54. LLaMA3-TAIDE-LX-8B-Chat-Alpha1 (2024) — link
  55. DeBERTa : Decoding-enhanced BERT with Disentangled Attention (2021)

4 of 63 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.