Classifying Textual Data with Pre-trained Vision Models through Transfer Learning and Data Transformations

Knowledge is acquired by humans through experience, and no boundary is set\nbetween the kinds of knowledge or skill levels we can achieve on different\ntasks at the same time. When it comes to Neural Networks, that is not the case.\nThe breakthroughs in the field are extremely task and domain-specific. Vision\nand language are dealt with in separate manners, using separate methods and\ndifferent datasets. Current text classification methods, mostly rely on\nobtaining contextual embeddings for input text samples, then training a\nclassifier on the embedded dataset. Transfer learning in Language-related tasks\nin general, is heavily used in obtaining the contextual text embeddings for the\ninput samples. In this work, we propose to use the knowledge acquired by\nbenchmark Vision Models which are trained on ImageNet to help a much smaller\narchitecture learn to classify text. A data transformation technique is used to\ncreate a new image dataset, where each image represents a sentence embedding\nfrom the last six layers of BERT, projected on a 2D plane using a t-SNE based\nmethod. We trained five models containing early layers sliced from vision\nmodels which are pretrained on ImageNet, on the created image dataset for the\nIMDB dataset embedded with the last six layers of BERT. Despite the challenges\nposed by the very different datasets, experimental results achieved by this\napproach which links large pretrained models on both language and vision, are\nvery promising, without employing compute resources. Specifically, Sentiment\nAnalysis is achieved by five different models on the same image dataset\nobtained after BERT embeddings are transformed into gray scale images.\n Index Terms: BERT, Convolutional Neural Networks, Domain Adaptation, image\nclassification, Natural Language Processing, t-SNE, text classification,\nTransfer Learning\n