Statistical and economic evaluation of forecasts in electricity markets: beyond RMSE and MAE

ArXiv ID: 2511.13616 “View on arXiv”

Authors: Katarzyna Maciejowska, Arkadiusz Lipiecki, Bartosz Uniejewski

Abstract

In recent years, a rapid development of forecasting methods has led to an increase in the accuracy of predictions. In the literature, forecasts are typically evaluated using metrics such as Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE). While appropriate for statistical assessment, these measures do not adequately reflect the economic value of forecasts. This study addresses the decision-making problem faced by a battery energy storage system, which must determine optimal charging and discharging times based on day-ahead electricity price forecasts. To explore the relationship between forecast accuracy and economic value, we generate a pool of 192 forecasts. These are evaluated using seven statistical metrics that go beyond RMSE and MAE, capturing various characteristics of the predictions and associated errors. We calculate the dynamic correlation between the statistical measures and gained profits to reveal that both RMSE and MAE are only weakly correlated with revenue. In contrast, measures that assess the alignment between predicted and actual daily price curves have a stronger relationship with profitability and are thus more effective for selecting optimal forecasts.

Keywords: Battery energy storage system, Day-ahead electricity price forecasting, Economic value of forecasts, Dynamic correlation, Statistical metrics, Commodities (Energy)

Complexity vs Empirical Score

• Math Complexity: 5.5/10
• Empirical Rigor: 8.0/10
• Quadrant: Holy Grail
• Why: The paper employs advanced statistical and econometric methods, including complex forecasting models (ARX, NARX, LEAR) and a custom battery trading simulation, requiring significant data processing and implementation. However, it lacks explicit code or algorithmic details in the excerpt, focusing on empirical results and methodology rather than mathematical derivations.

  flowchart TD
    A["Research Goal: Assess the economic value<br>of electricity price forecasts<br>for battery storage operation"] --> B{"Methodology"}
    
    B --> C["Input: 192 diverse day-ahead<br>electricity price forecasts"]
    B --> D["Statistical Evaluation:<br>7 metrics beyond RMSE/MAE<br>e.g., MAPE, Correlation"]
    
    C --> E["Simulation: Battery Storage<br>Optimization based on forecasts"]
    D --> F["Comparison: Dynamic correlation<br>between statistical metrics<br>and actual profits"]
    
    E --> G["Outcome: Calculated revenue<br>for each forecast"]
    G --> F
    
    F --> H{"Key Findings"}
    H --> I["RMSE & MAE show<br>weak correlation with revenue"]
    H --> J["Measures of curve alignment<br>strongly predict profitability"]
    H --> K["Best practice: Select forecasts<br>based on economic utility, not<br>just statistical error"]