Support for Stock Trend Prediction Using Transformers and Sentiment Analysis

ArXiv ID: 2305.14368 “View on arXiv”

Authors: Unknown

Abstract

Stock trend analysis has been an influential time-series prediction topic due to its lucrative and inherently chaotic nature. Many models looking to accurately predict the trend of stocks have been based on Recurrent Neural Networks (RNNs). However, due to the limitations of RNNs, such as gradient vanish and long-term dependencies being lost as sequence length increases, in this paper we develop a Transformer based model that uses technical stock data and sentiment analysis to conduct accurate stock trend prediction over long time windows. This paper also introduces a novel dataset containing daily technical stock data and top news headline data spanning almost three years. Stock prediction based solely on technical data can suffer from lag caused by the inability of stock indicators to effectively factor in breaking market news. The use of sentiment analysis on top headlines can help account for unforeseen shifts in market conditions caused by news coverage. We measure the performance of our model against RNNs over sequence lengths spanning 5 business days to 30 business days to mimic different length trading strategies. This reveals an improvement in directional accuracy over RNNs as sequence length is increased, with the largest improvement being close to 18.63% at 30 business days.

Keywords: Stock Trend Prediction, Transformer, Sentiment Analysis, Recurrent Neural Networks (RNNs), Time Series Analysis, Equities

Complexity vs Empirical Score

• Math Complexity: 6.0/10
• Empirical Rigor: 6.0/10
• Quadrant: Holy Grail
• Why: The paper employs advanced deep learning concepts like Transformer self-attention and has detailed equations for technical indicators, yet it is grounded in a novel dataset with specific performance metrics against baseline models.

  flowchart TD
    A["Research Goal: Improve Stock Trend Prediction"] --> B["Data Collection"]
    
    subgraph B ["Data Inputs"]
        B1["Daily Technical Data"]
        B2["Top News Headlines"]
    end
    
    B --> C["Sentiment Analysis"]
    B --> D["Transformer Model Training"]
    
    C --> D
    
    subgraph D ["Model Architecture"]
        D1["Technical Features + Sentiment Scores"]
        D2["Multi-Head Attention"]
        D3["Feed-Forward Network"]
    end
    
    D --> E["Performance Evaluation"]
    
    subgraph E ["Comparison"]
        E1["Transformer vs RNNs"]
        E2["5-30 Business Days Sequence Lengths"]
    end
    
    E --> F["Key Findings"]
    
    subgraph F ["Outcomes"]
        F1["↑ 18.63% Directional Accuracy at 30 Days"]
        F2["Transformer Outperforms RNNs for Long Sequences"]
        F3["Sentiment Analysis Reduces News Lag"]
    end