Paper: arXiv 2506.06288

Authors: Xueying Ding, Aakriti Mittal, Achintya Gopal

Abstract

Time-series data is a vital modality within data science communities. This is particularly valuable in financial applications, where it helps in detecting patterns, understanding market behavior, and making informed decisions based on historical data. Recent advances in language modeling have led to the rise of time-series pre-trained models that are trained on vast collections of datasets and applied to diverse tasks across financial domains. However, across financial applications, existing time-series pre-trained models have not shown boosts in performance over simple finance benchmarks in both zero-shot and fine-tuning settings. This phenomenon occurs because of a i) lack of financial data within the pre-training stage, and ii) the negative transfer effect due to inherently different time-series patterns across domains. Furthermore, time-series data is continuous, noisy, and can be collected at varying frequencies and with varying lags across different variables, making this data more challenging to model than languages. To address the above problems, we introduce a Pre-trained MoDEL for FINance TimE-series (Delphyne). Delphyne achieves competitive performance to existing foundation and full-shot models with few fine-tuning steps on publicly available datasets, and also shows superior performances on various financial tasks.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 5.5/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced mathematics including transformers, probabilistic forecasting with Student-T mixtures, and Bayesian MCMC analysis for negative transfer, while providing empirical results on public financial datasets and ablation studies.

Research Flowchart

  flowchart TD
  A["Research Goal: Create a robust<br>financial time-series model<br>overcoming negative transfer<br>and data challenges"] --> B["Key Methodology: Pre-train on<br>diverse financial datasets<br>using specialized architecture"]

  B --> C["Data/Inputs: Large-scale<br>public financial datasets<br>with varying frequencies/lags"]

  C --> D["Computational Process:<br>Delphyne Model Training<br>(MoE architecture,<br>multi-horizon forecasting)"]

  D --> E["Key Findings: <br>1. Competitive vs. Foundation Models<br>2. Superior on financial tasks<br>3. Few fine-tuning steps needed<br>4. Mitigates negative transfer"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes Delphyne, a pre‑trained model for general and finance‑specific time‑series tasks.

Source sentence
we introduce a Pre-trained MoDEL for FINance TimE-series (Delphyne).

Assumes 6

  • Probability forecasting assumes a fixed output distribution. model
    Source sentence
    For probability forecasting, previous studies usually assume a fixed output distribution (Salinas et al., 2020a).
  • A single distribution may be insufficient when data presents varying distributions and supports. model
    Source sentence
    However, when the data presents varying distributions and supports, relying on a single distribution may be insufficient.
  • For simplicity, the Student’s T mixture is chosen as the output distribution. model
    Source sentence
    For simplicity (Occam’s Razor), we choose the Student’s T mixture as the output distribution, with further discussion in the probability quantification experiment (Sec. 5.4).
  • Zero‑shot evaluation uses a fixed context length of 500. execution
    Source sentence
    For zero-shot, we fix context length equals 500 and report the test performance.
Show 2 more
  • Fine‑tuning searches context length in {100, 300, 500, 1000, 2000} and learning rate in {5e‑5, 1e‑5}. execution
    Source sentence
    For fine-tuning, we search context length in {100, 300, 500, 1000, 2000} and the learning rate in {5e −5, 1e −5}.
  • Synthetic pre‑training data are generated as Wavelet functions and GARCH‑style data. data
    Source sentence
    Pre-training with GARCH and Wavelet Data To simulate datasets encountered in real‑world scenarios, we generate two types of synthetic data: Wavelet functions and GARCH‑style data.

Data

  • LOTSA benchmark public
    Source sentence
    Delphyne is trained on both LOTSA (the largest public times-series dataset (Woo et al., 2024)) and financial data (Sec.4).
  • Monash Time Series Forecasting Repository benchmark public
    Source sentence
    Monash The Monash Time Series Forecasting Repository (Godahewa et al., 2021) is a comprehensive collection of diverse time series datasets.
  • Synthetic Wavelet functions synthetic public
    Source sentence
    functions are composed of a combination of sine and cosine waves.
  • Synthetic GARCH‑style data synthetic public
    Source sentence
    in GARCH models the current time‑steps are based on past squared residuals and past
  • Financial returns of stocks across various sectors market-prices public
    Source sentence
    that Delphyne is trained on both strongly and weakly correlated financial data such as the returns of stocks across various sectors, it employs an any‑variate attention mechanism.
  • Various public datasets included in LOTSA (e.g., BuildingsBench, ClimateLearn, CloudOps, G benchmark public
    Source sentence
    We include both publicly available data and financial data in Del‑phyne’s training, allowing Del‑phyne to gene…

Costs

Not applicable The paper reports no trading or execution result.

Finds 5

  • Delphyne‑F slightly outperforms the Naive model in zero‑shot performance, while Delphyne‑A and Delphyne‑L achieve similar performance after fine‑tuning, rivaling the best model MOIRAI. performance
    Source sentence
    In zero-shot performance, Delphyne-F, trained solely on finance data, slightly outperforms the Naive model and the other three benchmarks
  • Negative transfer is observed when pre‑training includes data from disparate domains, reducing zero‑shot forecast quality. negative
    Source sentence
    negative transfer can occur when cross‑domain data is added during pre‑training... The appearances of data from too different distributions can lead to less effective zero‑shot forecast results.
  • A mixture of Student’s T distributions performs similarly to the more complex MOIRAI mixture and better than a single distribution for stock return NLL. predictive-accuracy
    Source sentence
    Fig. 4 shows that the Student’s T mixture performs similarly to MOIRAI’s Mixture on measuring stock returns. On the other hand, a single distribution underperforms both.
  • Architectural modifications, including any‑variate attention, improve handling of multivariate financial data without forced mixing. performance
    Source sentence
    that Delphyne is trained on both strongly and weakly correlated financial data such as the returns of stocks across various sectors, it employs an any‑variate attention mechanism. This mechanism enables the integration of cross‑channel information without forcibly mixing variate data.
  • Delphyne consistently outperforms or matches state‑of‑the‑art baselines on both general and finance time‑series tasks. performance
    Source sentence
    We conduct experiments for both general and finance time‑series tasks, demonstrating that Delphyne consistently outperforms or matches state‑of‑the‑art baselines.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 41 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 1 paper in the archive (1 by independent authors)

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.