A Comprehensive Survey on Word Representation Models: From Classical to State-Of-The-Art Word Representation Language Models

Word representation has always been an important research area in the history\nof natural language processing (NLP). Understanding such complex text data is\nimperative, given that it is rich in information and can be used widely across\nvarious applications. In this survey, we explore different word representation\nmodels and its power of expression, from the classical to modern-day\nstate-of-the-art word representation language models (LMS). We describe a\nvariety of text representation methods, and model designs have blossomed in the\ncontext of NLP, including SOTA LMs. These models can transform large volumes of\ntext into effective vector representations capturing the same semantic\ninformation. Further, such representations can be utilized by various machine\nlearning (ML) algorithms for a variety of NLP related tasks. In the end, this\nsurvey briefly discusses the commonly used ML and DL based classifiers,\nevaluation metrics and the applications of these word embeddings in different\nNLP tasks.\n