Paper: arXiv 2506.06288
Authors: Xueying Ding, Aakriti Mittal, Achintya Gopal
Abstract
Time-series data is a vital modality within data science communities. This is particularly valuable in financial applications, where it helps in detecting patterns, understanding market behavior, and making informed decisions based on historical data. Recent advances in language modeling have led to the rise of time-series pre-trained models that are trained on vast collections of datasets and applied to diverse tasks across financial domains. However, across financial applications, existing time-series pre-trained models have not shown boosts in performance over simple finance benchmarks in both zero-shot and fine-tuning settings. This phenomenon occurs because of a i) lack of financial data within the pre-training stage, and ii) the negative transfer effect due to inherently different time-series patterns across domains. Furthermore, time-series data is continuous, noisy, and can be collected at varying frequencies and with varying lags across different variables, making this data more challenging to model than languages. To address the above problems, we introduce a Pre-trained MoDEL for FINance TimE-series (Delphyne). Delphyne achieves competitive performance to existing foundation and full-shot models with few fine-tuning steps on publicly available datasets, and also shows superior performances on various financial tasks.
Complexity vs Empirical Score
- Math Complexity: 7.5/10
- Empirical Rigor: 5.5/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper employs advanced mathematics including transformers, probabilistic forecasting with Student-T mixtures, and Bayesian MCMC analysis for negative transfer, while providing empirical results on public financial datasets and ablation studies.
Research Flowchart
flowchart TD A["Research Goal: Create a robust<br>financial time-series model<br>overcoming negative transfer<br>and data challenges"] --> B["Key Methodology: Pre-train on<br>diverse financial datasets<br>using specialized architecture"] B --> C["Data/Inputs: Large-scale<br>public financial datasets<br>with varying frequencies/lags"] C --> D["Computational Process:<br>Delphyne Model Training<br>(MoE architecture,<br>multi-horizon forecasting)"] D --> E["Key Findings: <br>1. Competitive vs. Foundation Models<br>2. Superior on financial tasks<br>3. Few fine-tuning steps needed<br>4. Mitigates negative transfer"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes Delphyne, a pre‑trained model for general and finance‑specific time‑series tasks.
Source sentence
we introduce a Pre-trained MoDEL for FINance TimE-series (Delphyne).
Assumes 6
- Probability forecasting assumes a fixed output distribution. model
Source sentence
For probability forecasting, previous studies usually assume a fixed output distribution (Salinas et al., 2020a).
- A single distribution may be insufficient when data presents varying distributions and supports. model
Source sentence
However, when the data presents varying distributions and supports, relying on a single distribution may be insufficient.
- For simplicity, the Student’s T mixture is chosen as the output distribution. model
Source sentence
For simplicity (Occam’s Razor), we choose the Student’s T mixture as the output distribution, with further discussion in the probability quantification experiment (Sec. 5.4).
- Zero‑shot evaluation uses a fixed context length of 500. execution
Source sentence
For zero-shot, we fix context length equals 500 and report the test performance.
Show 2 more
- Fine‑tuning searches context length in {100, 300, 500, 1000, 2000} and learning rate in {5e‑5, 1e‑5}. execution
Source sentence
For fine-tuning, we search context length in {100, 300, 500, 1000, 2000} and the learning rate in {5e −5, 1e −5}.
- Synthetic pre‑training data are generated as Wavelet functions and GARCH‑style data. data
Source sentence
Pre-training with GARCH and Wavelet Data To simulate datasets encountered in real‑world scenarios, we generate two types of synthetic data: Wavelet functions and GARCH‑style data.
Data
- LOTSA benchmark public
Source sentence
Delphyne is trained on both LOTSA (the largest public times-series dataset (Woo et al., 2024)) and financial data (Sec.4).
- Monash Time Series Forecasting Repository benchmark public
Source sentence
Monash The Monash Time Series Forecasting Repository (Godahewa et al., 2021) is a comprehensive collection of diverse time series datasets.
- Synthetic Wavelet functions synthetic public
Source sentence
functions are composed of a combination of sine and cosine waves.
- Synthetic GARCH‑style data synthetic public
Source sentence
in GARCH models the current time‑steps are based on past squared residuals and past
- Financial returns of stocks across various sectors market-prices public
Source sentence
that Delphyne is trained on both strongly and weakly correlated financial data such as the returns of stocks across various sectors, it employs an any‑variate attention mechanism.
- Various public datasets included in LOTSA (e.g., BuildingsBench, ClimateLearn, CloudOps, G benchmark public
Source sentence
We include both publicly available data and financial data in Del‑phyne’s training, allowing Del‑phyne to gene…
Costs
Not applicable
Finds 5
- Delphyne‑F slightly outperforms the Naive model in zero‑shot performance, while Delphyne‑A and Delphyne‑L achieve similar performance after fine‑tuning, rivaling the best model MOIRAI. performance
Source sentence
In zero-shot performance, Delphyne-F, trained solely on finance data, slightly outperforms the Naive model and the other three benchmarks
- Negative transfer is observed when pre‑training includes data from disparate domains, reducing zero‑shot forecast quality. negative
Source sentence
negative transfer can occur when cross‑domain data is added during pre‑training... The appearances of data from too different distributions can lead to less effective zero‑shot forecast results.
- A mixture of Student’s T distributions performs similarly to the more complex MOIRAI mixture and better than a single distribution for stock return NLL. predictive-accuracy
Source sentence
Fig. 4 shows that the Student’s T mixture performs similarly to MOIRAI’s Mixture on measuring stock returns. On the other hand, a single distribution underperforms both.
- Architectural modifications, including any‑variate attention, improve handling of multivariate financial data without forced mixing. performance
Source sentence
that Delphyne is trained on both strongly and weakly correlated financial data such as the returns of stocks across various sectors, it employs an any‑variate attention mechanism. This mechanism enables the integration of cross‑channel information without forcibly mixing variate data.
- Delphyne consistently outperforms or matches state‑of‑the‑art baselines on both general and finance time‑series tasks. performance
Source sentence
We conduct experiments for both general and finance time‑series tasks, demonstrating that Delphyne consistently outperforms or matches state‑of‑the‑art baselines.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 41 related papers). Coverage is partial, so this is neither replication nor refutation.
1 paper in the archive
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.