Paper: arXiv 2310.13029
Authors: Ioannis Nasios, Konstantinos Vogklis
Abstract
In this paper we tackle the problem of point and probabilistic forecasting by describing a blending methodology of machine learning models that belong to gradient boosted trees and neural networks families. These principles were successfully applied in the recent M5 Competition on both Accuracy and Uncertainty tracks. The keypoints of our methodology are: a) transform the task to regression on sales for a single day b) information rich feature engineering c) create a diverse set of state-of-the-art machine learning models and d) carefully construct validation sets for model tuning. We argue that the diversity of the machine learning models along with the careful selection of validation examples, where the most important ingredients for the effectiveness of our approach. Although forecasting data had an inherent hierarchy structure (12 levels), none of our proposed solutions exploited that hierarchical scheme. Using the proposed methodology, our team was ranked within the gold medal range in both Accuracy and the Uncertainty track. Inference code along with already trained models are available at https://github.com/IoannisNasios/M5_Uncertainty_3rd_place
Complexity vs Empirical Score
- Math Complexity: 3.0/10
- Empirical Rigor: 8.5/10
- Quadrant: Street Traders — empirical and applied, lower math complexity
Why this score: The paper relies on standard machine learning concepts and arithmetic statistics (RMSSE, WRMSSE) without advanced derivations, while detailing a backtest-ready methodology with specific data splits, feature engineering, and public code/repo links.
Research Flowchart
flowchart TD A["Research Goal: Blend GBT & Neural Networks<br>for Point/Probabilistic Forecasting"] --> B["Data Input: Hierarchical Time Series"] B --> C["Key Methodology Step 1: Feature Engineering"] B --> D["Key Methodology Step 2: Create Diverse ML Models<br>GBT & Neural Networks"] C --> E["Computational Process: Regression on Sales<br>for a Single Day"] D --> E E --> F["Validation: Careful Set Construction<br>for Model Tuning"] F --> G["Key Finding: Gold Medal Range<br>(Accuracy & Uncertainty Tracks)<br>Hierarchy Not Exploited"] G --> H["Outcome: Open Source Code & Trained Models"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes a blending methodology that combines gradient boosted trees and neural networks for point and probabilistic forecasting of hierarchical time series.
Source sentence
In this paper we tackle the problem of point and probabilistic forecasting by describing a blending methodology of machine learning models that belong to gradient boosted trees and neural networks families.
Assumes 5
- The hierarchical structure of the data is not exploited in the forecasting models. model
Source sentence
Although forecasting data had an inherent hierarchy structure (12 levels), none of our proposed solutions exploited that hierarchical scheme.
- Sales counts are intermittent with many zeros, justifying the use of a Tweedie loss function. data
Source sentence
Since the target quantity (sales count) is intermittent and has a lot of zeros (≈68% of daily sales are zeros), using the MSE loss function may lead to suboptimal solutions.
- Weighted RMSSE uses price‑related weights based on the last 28 observations of the training sample. statistical
Source sentence
weight is computed based on the last 28 observations of the training sample of the dataset, i.e., the cumulative actual dollar sales that each series displayed in that particular period.
- Symmetric distributions are assumed for aggregation levels 1‑9 and skewed distributions for levels 10‑12 in probabilistic forecasting. model
Source sentence
We assumed symmetric distributions on levels 1-9 and skewed distributions on levels 10-12.
Show 1 more
- A simple exponential smoothing (α = 0.96) per product and store improves forecast accuracy. model
Source sentence
In the final solution we employed a simple exponential smoothing (α = 0.96) per product and store id (Level 12 - 30, 490) that lead to substantial improvement in both validation sets and the final evaluation.
Data
- M5 competition dataset benchmark public
Source sentence
The dataset provided contains 42, 840 hierarchical sales data from Walmart. It covers stores in three US states
Costs
Not applicable
Finds 3
- The blended model achieved a validation WRMSSE of 0.549 and a final competition score of 0.552, placing the team within the gold‑medal range on both Accuracy and Uncertainty tracks. performance
Source sentence
The mean validation WRMSSE score using the final blend of Eq. 4 is 0.549, while our final submission score was 0.552 - only 0.003 units apart.
- Post‑processing exponential smoothing improved validation and private leaderboard scores, and would have raised the Accuracy track rank by three positions if applied. performance
Source sentence
We notice that, although this post-processing should be performed for both competition tracks, we were able to only use it for the Uncertainty track ... as it
- Gradient boosted trees and neural networks outperformed traditional statistical time‑series methods in the M5 competition. predictive-accuracy
Source sentence
... was the superiority of Machine Learning methods, particularly the Gradient Boosted Trees and Neural Networks, against statistical time series methods.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 11 related papers). Coverage is partial, so this is neither replication nor refutation.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.