A Novel Deep Reinforcement Learning Based Automated Stock Trading System Using Cascaded LSTM Networks

Paper: arXiv 2212.02721

Authors: Jie Zou, Jiashu Lou, Baohua Wang, Sixue Liu

Abstract

More and more stock trading strategies are constructed using deep reinforcement learning (DRL) algorithms, but DRL methods originally widely used in the gaming community are not directly adaptable to financial data with low signal-to-noise ratios and unevenness, and thus suffer from performance shortcomings. In this paper, to capture the hidden information, we propose a DRL based stock trading system using cascaded LSTM, which first uses LSTM to extract the time-series features from stock daily data, and then the features extracted are fed to the agent for training, while the strategy functions in reinforcement learning also use another LSTM for training. Experiments in DJI in the US market and SSE50 in the Chinese stock market show that our model outperforms previous baseline models in terms of cumulative returns and Sharp ratio, and this advantage is more significant in the Chinese stock market, a merging market. It indicates that our proposed method is a promising way to build a automated stock trading system.

Complexity vs Empirical Score

• Math Complexity: 5.5/10
• Empirical Rigor: 6.0/10
• Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper adapts deep reinforcement learning to financial time series using a cascaded LSTM architecture to handle low signal-to-noise ratios. It provides solid empirical evaluation across two distinct stock indices (DJI and SSE50) showing improved Sharpe ratios, though it relies on standard DRL formulations without deep mathematical proofs.

Research Flowchart

  flowchart TD
    A[Research Goal: Adapt DRL for noisy financial data in automated stock trading] --> B[Data Inputs: Daily stock data from DJI US & SSE50 Chinese markets]
    B --> C[Feature Extraction: 1st LSTM extracts temporal features from raw data]
    C --> D[Agent Training: 2nd LSTM processes features within DRL policy/strategy functions]
    D --> E[Backtesting & Evaluation: Compare model against baselines using Cumulative Returns & Sharpe Ratio]
    E --> F[Key Outcome: Cascaded LSTM-DRL outperforms baselines, especially in emerging markets like SSE50]