Paper: arXiv 2404.00424

Authors: Zhaofeng Zhang, Banghao Chen, Shengxin Zhu, Nicolas Langrené

Abstract

In traditional quantitative trading practice, navigating the complicated and dynamic financial market presents a persistent challenge. Fully capturing various market variables, including long-term information, as well as essential signals that may lead to profit remains a difficult task for learning algorithms. In order to tackle this challenge, this paper introduces quantformer, an enhanced neural network architecture based on transformer, to build investment factors. By transfer learning from sentiment analysis, quantformer not only exploits its original inherent advantages in capturing long-range dependencies and modeling complex data relationships, but is also able to solve tasks with numerical inputs and accurately forecast future returns over a given period. This work collects more than 5,000,000 rolling data of 4,601 stocks in the Chinese capital market from 2010 to 2023. The results of this study demonstrate the model’s superior performance in predicting stock trends compared with other 100-factor-based quantitative strategies. Notably, the model’s innovative use of transformer-like model to establish factors, in conjunction with market sentiment information, has been shown to enhance the accuracy of trading signals significantly, thereby offering promising implications for the future of quantitative trading strategies.

Complexity vs Empirical Score

  • Math Complexity: 5.5/10
  • Empirical Rigor: 6.0/10 (capped from 7.5: transaction costs only partially modelled; see penalty flags)
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper adapts a Transformer architecture for quantitative factor construction by leveraging transfer learning from sentiment analysis on a massive Chinese equity dataset. While mathematically moderate, its empirical evaluation against 100 baseline factors over a 13-year dataset provides solid rigor.

Research Flowchart

  flowchart TD
    A[Research Goal: Improve stock return forecasting using a novel Transformer strategy] --> B[Data Collection: 5M+ rolling data from 4,601 Chinese stocks 2010-2023]
    B --> C[Transfer Learning: Pre-trained on Sentiment Analysis tasks]
    C --> D[Quantformer Model: Process numerical inputs & capture long-range dependencies]
    D --> E[Factor Construction: Generate enhanced quantitative trading signals]
    E --> F[Outcomes: Superior trend prediction & profitability over 100-factor baselines]

Cited References (107)

  1. Attention is all you need (2017)
  2. Language models are few-shot learners (2020)
  3. BART : denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension (2020)
  4. Language models are unsupervised multitask learners (2019)
  5. Portfolio selection (1952)
  6. Temporal relational ranking for stock prediction (2019)
  7. Capital asset prices: a theory of market equilibrium under conditions of risk (1964)
  8. A five-factor asset pricing model (2015)
and 99 more
  1. Induction of decision trees (1986)
  2. Support vector machines (1998)
  3. Long short-term memory (1997)
  4. Empirical evaluation of gated recurrent neural networks on sequence modeling (2014)
  5. Finding structure in time (1990)
  6. Common risk factors in the returns on stocks and bonds (1993)
  7. Transformers are RNNs : fast autoregressive transformers with linear attention (2020)
  8. Financial time series forecasting using support vector machines (2003)
  9. Forecasting stock market movement direction with support vector machine (2005)
  10. Generating sequences with recurrent neural networks (2013)
  11. Computational intelligence and financial markets: a survey and future directions (2016)
  12. Pairs selection and outranking: an application to the S & P ~100 index (2009)
  13. Modelling high-frequency limit order book dynamics with support vector machines (2015)
  14. Deep learning with long short-term memory networks for financial market predictions (2018)
  15. A hybrid model based on convolutional neural networks and long short-term memory for ozone concentration prediction (2018)
  16. A deep learning framework for financial time series using stacked autoencoders and long-short term memory (2017)
  17. Predicting stock market index using fusion of machine learning techniques (2015)
  18. Stock price forecasting by hybrid machine learning techniques (2009)
  19. Algorithmic financial trading with deep convolutional neural networks: time series to image conversion approach (2018)
  20. Stock price prediction via discovering multi-frequency trading patterns (2017)
  21. Deep learning with long short-term memory networks for financial market predictions (2018)
  22. Fin BERT : a large language model for extracting information from financial text (2023)
  23. Fin BERT : financial sentiment analysis with pre-trained language models (2019)
  24. Bert: Pre-training of deep bidirectional transformers for language understanding (2019)
  25. GPT : origin, theory, application, and future (2021)
  26. Financial time series forecasting using CNN and transformer — → 2304.04912
  27. Hierarchical multi-scale G aussian transformer for stock movement prediction. (2020)
  28. Informer: beyond efficient transformer for long sequence time-series forecasting (2021)
  29. Adaptive long-short pattern transformer for stock investment selection (2022)
  30. Fin BERT : a pretrained language model for financial communications (2020)
  31. When is a liability not a liability? T extual analysis, dictionaries, and 10- K s (2011)
  32. Bloomberg GPT : a large language model for finance (2023)
  33. Relation-aware transformer for portfolio policy learning (2021)
  34. DA-Transformer: Distance-aware Transformer (2021)
  35. A na "\i ve SVM-KNN based stock market trend reversal analysis for I ndian benchmark indices (2015)
  36. Self-normalizing neural networks (2017)
  37. Py T orch: an imperative style, high-performance deep learning library (2019)
  38. GPT-4 Technical Report (2023)
  39. Llama 2: open foundation and fine-tuned chat models (2023)
  40. Risk management and financial institutions (2012)
  41. Random search for hyper-parameter optimization (2012)
  42. Adam: a method for stochastic optimization (2015)
  43. On embeddings for numerical features in tabular deep learning (2022)
  44. Empirical evaluation of gated recurrent neural networks on sequence modeling (2014)
  45. An introduction to convolutional neural networks (2015)
  46. A dynamic analysis of the relationship between investor sentiment and stock market realized volatility: evidence from C hina (2020)
  47. An empirical examination of investor sentiment and stock market volatility: evidence from I ndia (2020)
  48. The investor psychology and stock market behavior during the initial era of COVID-19 : a study of C hina, J apan, and the U nited S tates (2021)
  49. The power of past stock returns to explain future stock returns (1995)
  50. Deep learning of predicting closing price through historical adjustment closing price (2022)
  51. A model of price adjustment (1971)
  52. Generative adversarial networks: a survey on attack and defense perspective (2023)
  53. Deep learning for cyber security intrusion detection: approaches, datasets, and comparative study (2020)
  54. CMMLU : measuring massive multitask language understanding in C hinese (2023)
  55. Adversarial learning (2005)
  56. Practical portfolio performance measurement and attribution (2023)
  57. Comparative study of machine learning models for stock price prediction (2022)
  58. Advances in financial machine learning (2018)
  59. Deep learning in the stock market—a systematic survey of practice, backtesting, and applications (2023)
  60. Stock market index prediction using deep T ransformer model (2022)
  61. Stockformer: A price-volume factor stock selection model based on wavelet transform and multi-task self-attention networks (2025)
  62. Dynamic mean-variance portfolio selection under factor models (2024)
  63. Dynamic CVaR portfolio construction with attention-powered generative factor learning — → 2301.07318
  64. Worst-case conditional value-at-risk with application to robust portfolio management (2009)
  65. Machine learning and portfolio optimization (2018)
  66. Technical analysis as a sentiment barometer and the cross-section of stock returns (2023)
  67. Physics-informed convolutional transformer for predicting volatility surface (2024)
  68. The evolved transformer (2019)
  69. SSDNet : state space decomposition neural network for time series forecasting (2021)
  70. A transformer-based framework for multivariate time series representation learning (2021)
  71. Triformer: triangular, variable-specific attentions for long sequence multivariate time series forecasting (2022)
  72. Transformer H awkes process (2020)
  73. Ensemble of temporal transformers for financial time series (2024)
  74. Fx-spot predictions with state-of-the-art transformer and time embeddings (2024)
  75. Multi-transformer: A new neural network-based architecture for forecasting S & P volatility (2021)
  76. Tarnet: task-aware reconstruction for time-series transformer (2022)
  77. Scaleformer: iterative multi-scale refining transformers for time series forecasting (2022)
  78. Reformer: the efficient transformer (2020)
  79. Learning to rotate: quaternion transformer for complicated periodical time series forecasting (2022)
  80. Pyraformer: low-complexity pyramidal attention for long-range time series modeling and forecasting (2022)
  81. Probabilistic transformer for time series analysis (2021)
  82. A time series is worth 64 words: long-term forecasting with transformers (2022)
  83. Non-stationary transformers: exploring the stationarity in time series forecasting (2022)
  84. MQT ransformer: multi-horizon forecasts with context dependent and feedback-aware attention (2020)
  85. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting (2019)
  86. i T ransformer: inverted transformers are effective for time series forecasting (2024)
  87. Exploring the advantages of transformers for high-frequency trading — → 2302.13850
  88. Gated transformer networks for multivariate time series classification (2021)
  89. Fedformer: frequency enhanced decomposed transformer for long-term series forecasting (2022)
  90. Are transformers effective for time series forecasting? (2023)
  91. A stock price prediction method based on BiLSTM and improved transformer (2023)
  92. Autoformer: decomposition transformers with auto-correlation for long-term series forecasting (2021)
  93. Airformer: predicting nationwide air quality in C hina with transformers (2023)
  94. Adversarial sparse transformer for time series forecasting (2020)
  95. Rough transformers: lightweight and continuous time series modelling through signature patching (2024)
  96. Quantnet: Transferring learning across systematic trading strategies (2020)
  97. Fee schedule of the S henzhen stock exchange (2023) — link
  98. Fee schedule of the S hanghai stock exchange (2023) — link
  99. Notice on adjusting the standards for securities transaction commissions (2002) — link

3 of 107 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.