Convolution neural network hyperparameter optimization using modified particle swarm optimization

Abstract

Based on the literature review, a convolutional neural network (CNN) is one of the deep learning techniques most often used for classification problems, especially image classification. Various approaches have been proposed to improve accuracy performance. In CNN architecture, parameter determination is very influential on accuracy performance. Particle swarm optimization (PSO) is a type of metaheuristic algorithm widely used for hyperparameter optimization. PSO convergence is faster than genetic algorithm (GA) and attracts many researchers for further studies such as genetic algorithms and ant colony. In PSO, determining the value of the weight parameter is very influential on accuracy. Therefore, this paper proposes CNN hyperparameter optimization using modified PSO with linearly decreasing randomized weight. The experiments use the modified National Institute of Standards and Technology (MNIST) dataset. The accuracy of the proposed method is superior, and the execution time is slower to random search. In epoch 1, epoch 3, and epoch 5, the proposed method is superior to baseline CNN, linearly decreasing weight PSO (LDWPSO), and RL-based optimization algorithm (ROA). Meanwhile, the accuracy performance of the proposed method is superior to previous studies, namely LeNet-1, LeNet-2, LeNet-3, PCANet-2, RANDNet-2, CAE1, CAE-2, and bee colony. Otherwise, lost to PSO-CNN, distributed PSO (DPSO), recurrent CNN, and CNN-PSO. However, the four methods have a complex architecture and wasteful execution time.