DELPHYNE: A Pre-Trained Model for General and Financial Time Series

ArXiv ID: 2506.06288 “View on arXiv”

Authors: Xueying Ding, Aakriti Mittal, Achintya Gopal

Abstract

Time-series data is a vital modality within data science communities. This is particularly valuable in financial applications, where it helps in detecting patterns, understanding market behavior, and making informed decisions based on historical data. Recent advances in language modeling have led to the rise of time-series pre-trained models that are trained on vast collections of datasets and applied to diverse tasks across financial domains. However, across financial applications, existing time-series pre-trained models have not shown boosts in performance over simple finance benchmarks in both zero-shot and fine-tuning settings. This phenomenon occurs because of a i) lack of financial data within the pre-training stage, and ii) the negative transfer effect due to inherently different time-series patterns across domains. Furthermore, time-series data is continuous, noisy, and can be collected at varying frequencies and with varying lags across different variables, making this data more challenging to model than languages. To address the above problems, we introduce a Pre-trained MoDEL for FINance TimE-series (Delphyne). Delphyne achieves competitive performance to existing foundation and full-shot models with few fine-tuning steps on publicly available datasets, and also shows superior performances on various financial tasks.

Keywords: Time-Series Pre-trained Models, Financial Time Series, Negative Transfer Effect, Fine-tuning, Foundation Models, General Financial Assets

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 5.5/10
  • Quadrant: Holy Grail
  • Why: The paper employs advanced mathematics including transformers, probabilistic forecasting with Student-T mixtures, and Bayesian MCMC analysis for negative transfer, while providing empirical results on public financial datasets and ablation studies.
  flowchart TD
    A["Research Goal: Create a robust<br>financial time-series model<br>overcoming negative transfer<br>and data challenges"] --> B["Key Methodology: Pre-train on<br>diverse financial datasets<br>using specialized architecture"]

    B --> C["Data/Inputs: Large-scale<br>public financial datasets<br>with varying frequencies/lags"]

    C --> D["Computational Process:<br>Delphyne Model Training<br>(MoE architecture,<br>multi-horizon forecasting)"]

    D --> E["Key Findings: <br>1. Competitive vs. Foundation Models<br>2. Superior on financial tasks<br>3. Few fine-tuning steps needed<br>4. Mitigates negative transfer"]