Classifier comparison benchmark for machine learning weather prediction enhancement

Abstract

Artificial intelligence (AI) and data mining can improve next-generation weather forecasting for urban planning, agriculture, and disaster management. This study investigates how machine learning (ML) classifiers can reduce forecast errors and support decision-making in sectors that require accurate predictions, including agriculture and transportation. We evaluate four classifiers—K-nearest neighbor (KNN), random forest (RF), Naive Bayes (NB), and multilayer perceptron (MLP)—using Waikato environment for knowledge analysis (WEKA) and Orange3 to compare their performance in identifying rain. A 10-fold cross-validation approach is applied to reduce overfitting, and model effectiveness is measured using key performance indicators including accuracy, precision, sensitivity (recall), and F-measure. Results show that classifier performance varies across tools, indicating that the analytical framework can influence outcomes. Among all models, the RF classifier performs best, achieving 99.92% accuracy in WEKA and 99.9% in Orange3. The MLP also shows strong performance with 99.20% accuracy in WEKA and 98.7% in Orange3. KNN and NB exhibit comparable performance, but lower precision and F-measure in WEKA. Overall, the findings suggest that RF is the most effective approach for rain prediction using data mining tools, with practical relevance for agriculture, transportation, and power systems.