Blending gradient boosted trees and neural networks for point and probabilistic forecasting of hierarchical time series

ArXiv ID: 2310.13029 “View on arXiv”

Authors: Unknown

Abstract

In this paper we tackle the problem of point and probabilistic forecasting by describing a blending methodology of machine learning models that belong to gradient boosted trees and neural networks families. These principles were successfully applied in the recent M5 Competition on both Accuracy and Uncertainty tracks. The keypoints of our methodology are: a) transform the task to regression on sales for a single day b) information rich feature engineering c) create a diverse set of state-of-the-art machine learning models and d) carefully construct validation sets for model tuning. We argue that the diversity of the machine learning models along with the careful selection of validation examples, where the most important ingredients for the effectiveness of our approach. Although forecasting data had an inherent hierarchy structure (12 levels), none of our proposed solutions exploited that hierarchical scheme. Using the proposed methodology, our team was ranked within the gold medal range in both Accuracy and the Uncertainty track. Inference code along with already trained models are available at https://github.com/IoannisNasios/M5_Uncertainty_3rd_place

Keywords: Forecasting, Gradient Boosted Trees, Neural Networks, Time Series, Probabilistic Forecasting, Retail / Sales

Complexity vs Empirical Score

• Math Complexity: 3.0/10
• Empirical Rigor: 8.5/10
• Quadrant: Street Traders
• Why: The paper relies on standard machine learning concepts and arithmetic statistics (RMSSE, WRMSSE) without advanced derivations, while detailing a backtest-ready methodology with specific data splits, feature engineering, and public code/repo links.

  flowchart TD
    A["Research Goal: Blend GBT & Neural Networks<br>for Point/Probabilistic Forecasting"] --> B["Data Input: Hierarchical Time Series"]
    B --> C["Key Methodology Step 1: Feature Engineering"]
    B --> D["Key Methodology Step 2: Create Diverse ML Models<br>GBT & Neural Networks"]
    C --> E["Computational Process: Regression on Sales<br>for a Single Day"]
    D --> E
    E --> F["Validation: Careful Set Construction<br>for Model Tuning"]
    F --> G["Key Finding: Gold Medal Range<br>(Accuracy & Uncertainty Tracks)<br>Hierarchy Not Exploited"]
    G --> H["Outcome: Open Source Code & Trained Models"]