Multilevel profiling of situation and dialogue-based deep networks for movie genre classification using movie trailers

Automated movie genre classification has emerged as an active and essential\narea of research and exploration. Short duration movie trailers provide useful\ninsights about the movie as video content consists of the cognitive and the\naffective level features. Previous approaches were focused upon either\ncognitive or affective content analysis. In this paper, we propose a novel\nmulti-modality: situation, dialogue, and metadata-based movie genre\nclassification framework that takes both cognition and affect-based features\ninto consideration. A pre-features fusion-based framework that takes into\naccount: situation-based features from a regular snapshot of a trailer that\nincludes nouns and verbs providing the useful affect-based mapping with the\ncorresponding genres, dialogue (speech) based feature from audio, metadata\nwhich together provides the relevant information for cognitive and affect based\nvideo analysis. We also develop the English movie trailer dataset (EMTD), which\ncontains 2000 Hollywood movie trailers belonging to five popular genres:\nAction, Romance, Comedy, Horror, and Science Fiction, and perform\ncross-validation on the standard LMTD-9 dataset for validating the proposed\nframework. The results demonstrate that the proposed methodology for movie\ngenre classification has performed excellently as depicted by the F1 scores,\nprecision, recall, and area under the precision-recall curves.\n