An approach to detect depression through sentiment analysis of speech and facial expressions

Uma Yadav, Priya Dasarwar (Maidamwar), Shweta V. Bondre

Abstract


Depression is one of the most common mental illnesses among all people, and the early detection of such disease is very hard due to subjective way of detection in classical methods. Audiovisual clues can be used as a source for the detection of depressive symptomatology due to the fact that they are not subjectively estimated. This paper presents the multimodal system for the automatic detection of depressive symptomatology based on audiovisual data analysis. The suggested method includes the following stages: preprocessing, audiovisual feature extraction, dimensionality reduction using factor analysis, multimodal fusion and classification using convolutional neural network (CNN) and deep belief network (DBN). Two different types of multimodal fusion techniques, namely feature level blend and verdict level blend. The accuracy of the suggested methods was measured in order to investigate the efficiency of the combination of audiovisual data. The results show that feature level blend achieves accuracies of 86.4% and 85.1% with CNN and DBN, respectively, and verdict level blend achieves higher accuracies of 88.2% and 87.6%.

Keywords


Audio features; Deep learning; Depression detection; Facial features; Video extraction

Full Text:

PDF


DOI: https://doi.org/10.11591/eei.v15i6.11413

Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Bulletin of EEI Statistics

Bulletin of Electrical Engineering and Informatics (BEEI)
ISSN: 2089-3191 , e-ISSN: 2302-9285
This journal is published by the Institute of Advanced Engineering and Science (IAES) .