Paper: arXiv 2310.13029

Authors: Ioannis Nasios, Konstantinos Vogklis

Abstract

In this paper we tackle the problem of point and probabilistic forecasting by describing a blending methodology of machine learning models that belong to gradient boosted trees and neural networks families. These principles were successfully applied in the recent M5 Competition on both Accuracy and Uncertainty tracks. The keypoints of our methodology are: a) transform the task to regression on sales for a single day b) information rich feature engineering c) create a diverse set of state-of-the-art machine learning models and d) carefully construct validation sets for model tuning. We argue that the diversity of the machine learning models along with the careful selection of validation examples, where the most important ingredients for the effectiveness of our approach. Although forecasting data had an inherent hierarchy structure (12 levels), none of our proposed solutions exploited that hierarchical scheme. Using the proposed methodology, our team was ranked within the gold medal range in both Accuracy and the Uncertainty track. Inference code along with already trained models are available at https://github.com/IoannisNasios/M5_Uncertainty_3rd_place

Complexity vs Empirical Score

  • Math Complexity: 3.0/10
  • Empirical Rigor: 8.5/10
  • Quadrant: Street Traders — empirical and applied, lower math complexity

Why this score: The paper relies on standard machine learning concepts and arithmetic statistics (RMSSE, WRMSSE) without advanced derivations, while detailing a backtest-ready methodology with specific data splits, feature engineering, and public code/repo links.

Research Flowchart

  flowchart TD
  A["Research Goal: Blend GBT & Neural Networks<br>for Point/Probabilistic Forecasting"] --> B["Data Input: Hierarchical Time Series"]
  B --> C["Key Methodology Step 1: Feature Engineering"]
  B --> D["Key Methodology Step 2: Create Diverse ML Models<br>GBT & Neural Networks"]
  C --> E["Computational Process: Regression on Sales<br>for a Single Day"]
  D --> E
  E --> F["Validation: Careful Set Construction<br>for Model Tuning"]
  F --> G["Key Finding: Gold Medal Range<br>(Accuracy & Uncertainty Tracks)<br>Hierarchy Not Exploited"]
  G --> H["Outcome: Open Source Code & Trained Models"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a blending methodology that combines gradient boosted trees and neural networks for point and probabilistic forecasting of hierarchical time series.

Source sentence
In this paper we tackle the problem of point and probabilistic forecasting by describing a blending methodology of machine learning models that belong to gradient boosted trees and neural networks families.

Assumes 5

  • The hierarchical structure of the data is not exploited in the forecasting models. model
    Source sentence
    Although forecasting data had an inherent hierarchy structure (12 levels), none of our proposed solutions exploited that hierarchical scheme.
  • Sales counts are intermittent with many zeros, justifying the use of a Tweedie loss function. data
    Source sentence
    Since the target quantity (sales count) is intermittent and has a lot of zeros (≈68% of daily sales are zeros), using the MSE loss function may lead to suboptimal solutions.
  • Weighted RMSSE uses price‑related weights based on the last 28 observations of the training sample. statistical
    Source sentence
    weight is computed based on the last 28 observations of the training sample of the dataset, i.e., the cumulative actual dollar sales that each series displayed in that particular period.
  • Symmetric distributions are assumed for aggregation levels 1‑9 and skewed distributions for levels 10‑12 in probabilistic forecasting. model
    Source sentence
    We assumed symmetric distributions on levels 1-9 and skewed distributions on levels 10-12.
Show 1 more
  • A simple exponential smoothing (α = 0.96) per product and store improves forecast accuracy. model
    Source sentence
    In the final solution we employed a simple exponential smoothing (α = 0.96) per product and store id (Level 12 - 30, 490) that lead to substantial improvement in both validation sets and the final evaluation.

Data

  • M5 competition dataset · Walmart benchmark public
    February 2011 to April 2016 · 42,840 hierarchical series (stores, departments, products, etc.)
    Source sentence
    The dataset provided contains 42, 840 hierarchical sales data from Walmart. It covers stores in three US states

Costs

Not applicable The paper reports no trading or execution result.

Finds 3

  • The blended model achieved a validation WRMSSE of 0.549 and a final competition score of 0.552, placing the team within the gold‑medal range on both Accuracy and Uncertainty tracks. performance
    Source sentence
    The mean validation WRMSSE score using the final blend of Eq. 4 is 0.549, while our final submission score was 0.552 - only 0.003 units apart.
  • Post‑processing exponential smoothing improved validation and private leaderboard scores, and would have raised the Accuracy track rank by three positions if applied. performance
    Source sentence
    We notice that, although this post-processing should be performed for both competition tracks, we were able to only use it for the Uncertainty track ... as it
  • Gradient boosted trees and neural networks outperformed traditional statistical time‑series methods in the M5 competition. predictive-accuracy
    Source sentence
    ... was the superiority of Machine Learning methods, particularly the Gradient Boosted Trees and Neural Networks, against statistical time series methods.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 11 related papers). Coverage is partial, so this is neither replication nor refutation.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.