Two-stage random search–Bayesian optimization for CNN-based short-term load forecasting

Abstract

This study proposes a two-stage hyperparameter optimization pipeline for convolutional neural network (CNN)–based short-term electricity load forecasting. In the first stage, random search is used to broadly explore candidate configurations, including the number of filters in each convolutional layer, batch size, training epochs, and the loss function. In the second stage, Bayesian optimization based on the tree-structured Parzen estimator (TPE), implemented in Optuna, refines promising regions of the hyperparameter space to obtain a better-performing model. The optimized CNN is evaluated using half-hourly (30-minute) electricity demand data from New South Wales (NSW), Victoria (VIC), and Queensland (QLD), and is benchmarked against a baseline CNN, a multilayer perceptron (MLP), an extended short-term memory network, and single-stage optimization variants. Across the three regions, the proposed approach achieves mean absolute percentage error (MAPE) values between 1.05% and 1.14%, representing an improvement of approximately 58% over the baseline CNN. Statistical robustness is examined using paired Wilcoxon signed-rank tests with Holm–Bonferroni correction on per-timestamp errors. Overall, the results indicate that combining random search with Bayesian optimization improves CNN forecasting accuracy across the three studied regions and provides a transparent tuning framework for future replication.