Exploiting Multi-Modal Features From Pre-trained Networks for Alzheimer's Dementia Recognition

Collecting and accessing a large amount of medical data is very\ntime-consuming and laborious, not only because it is difficult to find specific\npatients but also because it is required to resolve the confidentiality of a\npatient's medical records. On the other hand, there are deep learning models,\ntrained on easily collectible, large scale datasets such as Youtube or\nWikipedia, offering useful representations. It could therefore be very\nadvantageous to utilize the features from these pre-trained networks for\nhandling a small amount of data at hand. In this work, we exploit various\nmulti-modal features extracted from pre-trained networks to recognize\nAlzheimer's Dementia using a neural network, with a small dataset provided by\nthe ADReSS Challenge at INTERSPEECH 2020. The challenge regards to discern\npatients suspicious of Alzheimer's Dementia by providing acoustic and textual\ndata. With the multi-modal features, we modify a Convolutional Recurrent Neural\nNetwork based structure to perform classification and regression tasks\nsimultaneously and is capable of computing conversations with variable lengths.\nOur test results surpass baseline's accuracy by 18.75%, and our validation\nresult for the regression task shows the possibility of classifying 4 classes\nof cognitive impairment with an accuracy of 78.70%.\n