Deep residual bidirectional long short-term memory fusion: achieving superior accuracy in facial emotion recognition

Abstract

Facial emotion recognition (FER) is a crucial task in human communication. Various face emotion recognition models were introduced but often struggle with generalization across different datasets and handling subtle variations in expressions. This study aims to develop the deep residual bidirectional long short-term memory (Bi-LSTM) fusion method to improve FER accuracy. This method combines the strengths of convolutional neural networks (CNN) for spatial feature extraction and Bi-LSTM for capturing temporal dynamics, using residual layers to address the vanishing gradient problem. Testing was performed on three face emotion datasets, and a comparison was made with seventeen models. The results show perfect accuracy on the extended Cohn-Kanade (CK+) and the real-world affective faces database (RAF-DB) datasets and almost perfect accuracy on the face expression recognition plus (FERPlus) dataset. However, the receiver operating characteristic (ROC) curve for the CK+ dataset shows some inconsistencies, indicating potential overfitting. In contrast, the ROC curves for the RAF-DB and FERPlus datasets are consistent with the high accuracy achieved. The proposed method has proven highly efficient and reliable in classifying various facial expressions, making it a robust solution for FER applications.