Paper: arXiv 2402.06638
Authors: Chu Myaet Thwal, Ye Lin Tun, Kitae Kim, Seong-Bae Park, Choong Seon Hong
Abstract
Recent innovations in transformers have shown their superior performance in natural language processing (NLP) and computer vision (CV). The ability to capture long-range dependencies and interactions in sequential data has also triggered a great interest in time series modeling, leading to the widespread use of transformers in many time series applications. However, being the most common and crucial application, the adaptation of transformers to time series forecasting has remained limited, with both promising and inconsistent results. In contrast to the challenges in NLP and CV, time series problems not only add the complexity of order or temporal dependence among input sequences but also consider trend, level, and seasonality information that much of this data is valuable for decision making. The conventional training scheme has shown deficiencies regarding model overfitting, data scarcity, and privacy issues when working with transformers for a forecasting task. In this work, we propose attentive federated transformers for time series stock forecasting with better performance while preserving the privacy of participating enterprises. Empirical results on various stock data from the Yahoo! Finance website indicate the superiority of our proposed scheme in dealing with the above challenges and data heterogeneity in federated learning.
Complexity vs Empirical Score
- Math Complexity: 6.0/10
- Empirical Rigor: 7.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper employs advanced transformer architectures (multi-head self-attention) and federated learning with attentive aggregation (FedAtt), requiring significant mathematical foundations in neural networks and optimization, while demonstrating rigorous empirical evaluation on real-world stock data from Yahoo! Finance with comparative baselines.
Research Flowchart
flowchart TD Goal["Research Goal: Enhance Time Series Stock Forecasting<br>with Privacy Preservation"] --> Method["Methodology: Attentive Federated Transformers"] Method --> Data["Data: Yahoo! Finance Equities<br>(Heterogeneous, Multi-Client)"] Data --> Process["Computational Process:<br>Local Transformer Training +<br>Attentive Federated Aggregation"] Process --> Outcome["Key Outcomes:<br>1. Superior Forecast Accuracy<br>2. Preserved Enterprise Privacy<br>3. Robustness to Data Heterogeneity"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes attentive federated transformers for time series stock forecasting, combining transformer models with attentive federated aggregation to improve performance while preserving privacy.
Source sentence
we propose attentive federated transformers for time series stock forecasting with better performance while preserving the privacy of participating enterprises.
Assumes 6
- Historical daily stock data can be represented with a Time2Vec embedding of the day integer. data
Source sentence
we train our model on historical daily stock data where the input integer (day) is used as the time feature for Time2Vec representation
- Applying a moving average smoothing window of 10 days improves forecasting results. model
Source sentence
We apply the moving average smoothing effect with a window size of 10 days to all the features for a better forecasting result.
- Features are converted to daily volume changes and stock returns to enhance stationarity. data
Source sentence
We enhance the stationarity of our datasets by converting the volume and price features into the daily volume changes and stock returns.
- Data are min‑max normalized and split 80/10/10 for training/validation/test. data
Source sentence
Then, the values are min‑max normalized and split into 80% for training, 10% for validation and 10% for test set.
Show 2 more
- Each enterprise’s dataset represents a private local client with its own forecasting model. execution
Source sentence
Each dataset of an enterprise represents the private database of a local client with its own forecasting model.
- Clients have heterogeneous data sizes, reflecting data heterogeneity in federated learning. data
Source sentence
As we use various sizes of datasets for the data heterogeneity purpose, each dataset con‑
Data
- Yahoo! Finance stock data market-prices public
Source sentence
We retrieve the real‑time historical stock price data of 45 separate global enterprises from the Yahoo! Finance website [...]
Costs
Not applicable
Finds 4
- The proposed FedAtt scheme outperforms both the decentralized SOLO baseline and the FedAvg baseline on most of the tested enterprises. performance
Source sentence
From the table, we can observe that our proposed scheme outperforms the other two baselines in most cases.
- FedAtt yields better forecasting accuracy than FedAvg on the TMUS dataset, which has few data points. predictive-accuracy
Source sentence
Fig. 6 indicates that our proposed FedAtt scheme outperforms the other baselines. It is because the TMUS dataset contains only a small number of data points [...]
- On the IBM dataset, which has many data points, all approaches perform similarly, confirming that a large private dataset can train a high‑performing model alone. predictive-accuracy
Source sentence
... which is a sufficient amount to train a high‑performing model even solely with its private data.
- Attentive aggregation (FedAtt) optimizes the distance between the global model and client models in parameter space, leading to a well‑generalized global model. theoretical
Source sentence
By combing our time series transformer with the FedAtt scheme, it optimizes the distance between the global model and client models in parameter space to learn a well‑generalized global model for different enterprises.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 42 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (43)
- An effective time series analysis for stock trend prediction using ARIMA model for nifty midcap-50 (2013)
- Notice of violation of ieee publication principles: A hybrid arima and neural network model for short-term price forecasting in deregulated market (2009)
- Predicting closed price time series data using ARIMA Model (2018)
- Stock price prediction using the ARIMA model (2014)
- Forecasting of Indian stock market using time-series ARIMA model (2014)
- Using artificial neural network models in stock market index prediction (2011)
- Predicting stock and stock price index movement using trend deterministic data preparation and machine learning techniques (2015)
- An ensemble of LSTM neural networks for high-frequency stock market classification (2019)
and 35 more
- Predictability of machine learning techniques to forecast the trends of market index prices: Hypothesis testing for the Korean stock markets (2017)
- Forecasting stock prices with long-short term memory neural network based on attention mechanism (2020)
- Predicting stock prices using LSTM (2017)
- ModAugNet: A new forecasting framework for stock market index value with an overfitting prevention LSTM module and a prediction LSTM module (2018)
- Ensemble of deep recurrent neural networks for identifying enhancers via dinucleotide physicochemical properties (2019)
- Stock market prediction using LSTM recurrent neural network (2020)
- CNNpred: CNN-based stock market prediction using a diverse set of variables (2019)
- Forecasting stock prices from the limit order book using convolutional neural networks (2017)
- Stock price prediction using LSTM, RNN and CNN-sliding window model (2017)
- Reinforcement learning for optimized trade execution (2006)
- Generative adversarial networks (2020)
- Stock market prediction on high-frequency data using generative adversarial nets. (2018)
- Communication-efficient learning of deep networks from decentralized data (2017)
- Advances and open problems in federated learning (2021)
- Federated learning: Challenges, methods, and future directions (2020)
- Attention is all you need (2017)
- Bert: Pre-training of deep bidirectional transformers for language understanding (2018) — link
- Time2vec: Learning a vector representation of time (2019) — link
- Stock market prediction based on generative adversarial network (2019)
- Deep learning for time series forecasting: a survey (2021)
- Machine learning advances for time series forecasting (2021)
- Time-series forecasting with deep learning: a survey (2021)
- Yahoo! Finance
- An image is worth 16x16 words: Transformers for image recognition at scale (2020) — link
- Swin transformer: Hierarchical vision transformer using shifted windows (2021)
- Transformers in time series: A survey (2022) — link
- Informer: Beyond efficient transformer for long sequence time-series forecasting (2021)
- Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting (2021)
- FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting (2022) — link
- ETSformer: Exponential Smoothing Transformers for Time-series Forecasting (2022) — link
- Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting (2021)
- Learning private neural language modeling with attentive aggregation (2019)
- Enhancing stock movement prediction with adversarial training (2018) — link
- Alphastock: A buying-winners-and-selling-losers investment strategy using interpretable deep reinforcement attention networks (2019)
- CLVSA: a convolutional LSTM based variational sequence-to-sequence model with attention for predicting trends of financial markets (2021) — link
0 of 43 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.