Paper: arXiv 2402.06638

Authors: Chu Myaet Thwal, Ye Lin Tun, Kitae Kim, Seong-Bae Park, Choong Seon Hong

Abstract

Recent innovations in transformers have shown their superior performance in natural language processing (NLP) and computer vision (CV). The ability to capture long-range dependencies and interactions in sequential data has also triggered a great interest in time series modeling, leading to the widespread use of transformers in many time series applications. However, being the most common and crucial application, the adaptation of transformers to time series forecasting has remained limited, with both promising and inconsistent results. In contrast to the challenges in NLP and CV, time series problems not only add the complexity of order or temporal dependence among input sequences but also consider trend, level, and seasonality information that much of this data is valuable for decision making. The conventional training scheme has shown deficiencies regarding model overfitting, data scarcity, and privacy issues when working with transformers for a forecasting task. In this work, we propose attentive federated transformers for time series stock forecasting with better performance while preserving the privacy of participating enterprises. Empirical results on various stock data from the Yahoo! Finance website indicate the superiority of our proposed scheme in dealing with the above challenges and data heterogeneity in federated learning.

Complexity vs Empirical Score

  • Math Complexity: 6.0/10
  • Empirical Rigor: 7.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced transformer architectures (multi-head self-attention) and federated learning with attentive aggregation (FedAtt), requiring significant mathematical foundations in neural networks and optimization, while demonstrating rigorous empirical evaluation on real-world stock data from Yahoo! Finance with comparative baselines.

Research Flowchart

  flowchart TD
  Goal["Research Goal: Enhance Time Series Stock Forecasting<br>with Privacy Preservation"] --> Method["Methodology: Attentive Federated Transformers"]
  Method --> Data["Data: Yahoo! Finance Equities<br>(Heterogeneous, Multi-Client)"]
  Data --> Process["Computational Process:<br>Local Transformer Training +<br>Attentive Federated Aggregation"]
  Process --> Outcome["Key Outcomes:<br>1. Superior Forecast Accuracy<br>2. Preserved Enterprise Privacy<br>3. Robustness to Data Heterogeneity"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes attentive federated transformers for time series stock forecasting, combining transformer models with attentive federated aggregation to improve performance while preserving privacy.

Source sentence
we propose attentive federated transformers for time series stock forecasting with better performance while preserving the privacy of participating enterprises.

Assumes 6

  • Historical daily stock data can be represented with a Time2Vec embedding of the day integer. data
    Source sentence
    we train our model on historical daily stock data where the input integer (day) is used as the time feature for Time2Vec representation
  • Applying a moving average smoothing window of 10 days improves forecasting results. model
    Source sentence
    We apply the moving average smoothing effect with a window size of 10 days to all the features for a better forecasting result.
  • Features are converted to daily volume changes and stock returns to enhance stationarity. data
    Source sentence
    We enhance the stationarity of our datasets by converting the volume and price features into the daily volume changes and stock returns.
  • Data are min‑max normalized and split 80/10/10 for training/validation/test. data
    Source sentence
    Then, the values are min‑max normalized and split into 80% for training, 10% for validation and 10% for test set.
Show 2 more
  • Each enterprise’s dataset represents a private local client with its own forecasting model. execution
    Source sentence
    Each dataset of an enterprise represents the private database of a local client with its own forecasting model.
  • Clients have heterogeneous data sizes, reflecting data heterogeneity in federated learning. data
    Source sentence
    As we use various sizes of datasets for the data heterogeneity purpose, each dataset con‑

Data

  • Yahoo! Finance stock data · Yahoo! Finance market-prices public
    various start dates to 2022-10-25 · 45 separate global enterprises
    Source sentence
    We retrieve the real‑time historical stock price data of 45 separate global enterprises from the Yahoo! Finance website [...]

Costs

Not applicable The paper reports no trading or execution result.

Finds 4

  • The proposed FedAtt scheme outperforms both the decentralized SOLO baseline and the FedAvg baseline on most of the tested enterprises. performance
    Source sentence
    From the table, we can observe that our proposed scheme outperforms the other two baselines in most cases.
  • FedAtt yields better forecasting accuracy than FedAvg on the TMUS dataset, which has few data points. predictive-accuracy
    Source sentence
    Fig. 6 indicates that our proposed FedAtt scheme outperforms the other baselines. It is because the TMUS dataset contains only a small number of data points [...]
  • On the IBM dataset, which has many data points, all approaches perform similarly, confirming that a large private dataset can train a high‑performing model alone. predictive-accuracy
    Source sentence
    ... which is a sufficient amount to train a high‑performing model even solely with its private data.
  • Attentive aggregation (FedAtt) optimizes the distance between the global model and client models in parameter space, leading to a well‑generalized global model. theoretical
    Source sentence
    By combing our time series transformer with the FedAtt scheme, it optimizes the distance between the global model and client models in parameter space to learn a well‑generalized global model for different enterprises.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 42 related papers). Coverage is partial, so this is neither replication nor refutation.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (43)

  1. An effective time series analysis for stock trend prediction using ARIMA model for nifty midcap-50 (2013)
  2. Notice of violation of ieee publication principles: A hybrid arima and neural network model for short-term price forecasting in deregulated market (2009)
  3. Predicting closed price time series data using ARIMA Model (2018)
  4. Stock price prediction using the ARIMA model (2014)
  5. Forecasting of Indian stock market using time-series ARIMA model (2014)
  6. Using artificial neural network models in stock market index prediction (2011)
  7. Predicting stock and stock price index movement using trend deterministic data preparation and machine learning techniques (2015)
  8. An ensemble of LSTM neural networks for high-frequency stock market classification (2019)
and 35 more
  1. Predictability of machine learning techniques to forecast the trends of market index prices: Hypothesis testing for the Korean stock markets (2017)
  2. Forecasting stock prices with long-short term memory neural network based on attention mechanism (2020)
  3. Predicting stock prices using LSTM (2017)
  4. ModAugNet: A new forecasting framework for stock market index value with an overfitting prevention LSTM module and a prediction LSTM module (2018)
  5. Ensemble of deep recurrent neural networks for identifying enhancers via dinucleotide physicochemical properties (2019)
  6. Stock market prediction using LSTM recurrent neural network (2020)
  7. CNNpred: CNN-based stock market prediction using a diverse set of variables (2019)
  8. Forecasting stock prices from the limit order book using convolutional neural networks (2017)
  9. Stock price prediction using LSTM, RNN and CNN-sliding window model (2017)
  10. Reinforcement learning for optimized trade execution (2006)
  11. Generative adversarial networks (2020)
  12. Stock market prediction on high-frequency data using generative adversarial nets. (2018)
  13. Communication-efficient learning of deep networks from decentralized data (2017)
  14. Advances and open problems in federated learning (2021)
  15. Federated learning: Challenges, methods, and future directions (2020)
  16. Attention is all you need (2017)
  17. Bert: Pre-training of deep bidirectional transformers for language understanding (2018) — link
  18. Time2vec: Learning a vector representation of time (2019) — link
  19. Stock market prediction based on generative adversarial network (2019)
  20. Deep learning for time series forecasting: a survey (2021)
  21. Machine learning advances for time series forecasting (2021)
  22. Time-series forecasting with deep learning: a survey (2021)
  23. Yahoo! Finance
  24. An image is worth 16x16 words: Transformers for image recognition at scale (2020) — link
  25. Swin transformer: Hierarchical vision transformer using shifted windows (2021)
  26. Transformers in time series: A survey (2022) — link
  27. Informer: Beyond efficient transformer for long sequence time-series forecasting (2021)
  28. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting (2021)
  29. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting (2022) — link
  30. ETSformer: Exponential Smoothing Transformers for Time-series Forecasting (2022) — link
  31. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting (2021)
  32. Learning private neural language modeling with attentive aggregation (2019)
  33. Enhancing stock movement prediction with adversarial training (2018) — link
  34. Alphastock: A buying-winners-and-selling-losers investment strategy using interpretable deep reinforcement attention networks (2019)
  35. CLVSA: a convolutional LSTM based variational sequence-to-sequence model with attention for predicting trends of financial markets (2021) — link

0 of 43 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.