An approach to detect depression through sentiment analysis of speech and facial expressions
Abstract
Depression is one of the most common mental illnesses among all people, and the early detection of such disease is very hard due to subjective way of detection in classical methods. Audiovisual clues can be used as a source for the detection of depressive symptomatology due to the fact that they are not subjectively estimated. This paper presents the multimodal system for the automatic detection of depressive symptomatology based on audiovisual data analysis. The suggested method includes the following stages: preprocessing, audiovisual feature extraction, dimensionality reduction using factor analysis, multimodal fusion and classification using convolutional neural network (CNN) and deep belief network (DBN). Two different types of multimodal fusion techniques, namely feature level blend and verdict level blend. The accuracy of the suggested methods was measured in order to investigate the efficiency of the combination of audiovisual data. The results show that feature level blend achieves accuracies of 86.4% and 85.1% with CNN and DBN, respectively, and verdict level blend achieves higher accuracies of 88.2% and 87.6%.
Keywords
Audio features; Deep learning; Depression detection; Facial features; Video extraction
Full Text:
PDFDOI: https://doi.org/10.11591/eei.v15i6.11413
Refbacks
- There are currently no refbacks.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Bulletin of Electrical Engineering and Informatics (BEEI)
ISSN: 2089-3191
,
e-ISSN: 2302-9285
This journal is published by the
Institute of Advanced Engineering and Science (IAES)
.