An efficient automated data analytics approach to large scale computational comparative linguistics

This research project aimed to overcome the challenge of analysing human\nlanguage relationships, facilitate the grouping of languages and formation of\ngenealogical relationship between them by developing automated comparison\ntechniques. Techniques were based on the phonetic representation of certain key\nwords and concept. Example word sets included numbers 1-10 (curated), large\ndatabase of numbers 1-10 and sheep counting numbers 1-10 (other sources),\ncolours (curated), basic words (curated).\n To enable comparison within the sets the measure of Edit distance was\ncalculated based on Levenshtein distance metric. This metric between two\nstrings is the minimum number of single-character edits, operations including:\ninsertions, deletions or substitutions. To explore which words exhibit more or\nless variation, which words are more preserved and examine how languages could\nbe grouped based on linguistic distances within sets, several data analytics\ntechniques were involved. Those included density evaluation, hierarchical\nclustering, silhouette, mean, standard deviation and Bhattacharya coefficient\ncalculations. These techniques lead to the development of a workflow which was\nlater implemented by combining Unix shell scripts, a developed R package and\nSWI Prolog. This proved to be computationally efficient and permitted the fast\nexploration of large language sets and their analysis.\n