Video summarization based on multi level deep features using convolutional neural network

Abstract

A video summarization (VIDE SUM) system can be used to condense long hours of footage into short relevant summaries. Summaries generated automatically may fail to capture this subjective relevance, resulting in unsatisfactory summaries. To overcome this a novel, VIDE SUM has been proposed for multi-level deep features-based VIDE SUM. It comprises five main contributions, including deep learning (DL), spearman coefficient (SC), entropy, HAECS, and complexity feature based on multi-quality compression (CMQC). A convolutional neural network (CNN) is used to extract the crucial spatial and semantic information from the frames. The deep four features are extracted to support the effective keyframe selection. The CMQC another innovative feature, is designed to determine the degree of difficulty between two consecutive video frames. In fusing the selected keyframes and relevant features, the resultant video summary preserves the fundamental components of the original video. The proposed VIDE SUM method generates the maximum F1-score of 0.8907, whereas the next-best method, namely VS-GSDA, yields 0.8518, which reveals the power of the proposed method. The evaluation results prove the superiority of the proposed method.