Deep incremental learning models for financial temporal tabular datasets with distribution shifts

Paper: arXiv 2303.07925

Authors: Thomas Wong, Mauricio Barahona

Abstract

We present a robust deep incremental learning framework for regression tasks on financial temporal tabular datasets which is built upon the incremental use of commonly available tabular and time series prediction models to adapt to distributional shifts typical of financial datasets. The framework uses a simple basic building block (decision trees) to build self-similar models of any required complexity to deliver robust performance under adverse situations such as regime changes, fat-tailed distributions, and low signal-to-noise ratios. As a detailed study, we demonstrate our scheme using XGBoost models trained on the Numerai dataset and show that a two layer deep ensemble of XGBoost models over different model snapshots delivers high quality predictions under different market regimes. We also show that the performance of XGBoost models with different number of boosting rounds in three scenarios (small, standard and large) is monotonically increasing with respect to model size and converges towards the generalisation upper bound. We also evaluate the robustness of the model under variability of different hyperparameters, such as model complexity and data sampling settings. Our model has low hardware requirements as no specialised neural architectures are used and each base model can be independently trained in parallel.

Complexity vs Empirical Score

• Math Complexity: 3.5/10
• Empirical Rigor: 6.0/10 (capped from 7.5: transaction costs not modelled; see penalty flags)
• Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper relies heavily on practical tabular models (XGBoost) rather than deep theoretical mathematics, focusing on empirical performance under distribution shifts. It demonstrates solid empirical rigor through parameter sensitivity analysis and snapshot ensembling on the Numerai dataset, though it is mathematically light.

Research Flowchart

  flowchart TD
    A[Research Goal: Robust incremental learning for financial tabular data with distribution shifts] --> B[Input Data: Numerai financial temporal tabular dataset]
    B --> C[Methodology: Multi-layer deep ensemble of XGBoost model snapshots]
    C --> D[Parallel Computation: Independent base-model training across snapshots & scales]
    D --> E[Robustness Testing: Evaluate performance across market regimes & hyperparameter variations]
    E --> F[Key Outcome: Monotonic performance scaling & robust predictions during regime changes]