Using the Tsetlin Machine to Learn Human-Interpretable Rules for High-Accuracy Text Categorization with Medical Applications

Medical applications challenge today's text categorization techniques by\ndemanding both high accuracy and ease-of-interpretation. Although deep learning\nhas provided a leap ahead in accuracy, this leap comes at the sacrifice of\ninterpretability. To address this accuracy-interpretability challenge, we here\nintroduce, for the first time, a text categorization approach that leverages\nthe recently introduced Tsetlin Machine. In all brevity, we represent the terms\nof a text as propositional variables. From these, we capture categories using\nsimple propositional formulae, such as: if "rash" and "reaction" and\n"penicillin" then Allergy. The Tsetlin Machine learns these formulae from a\nlabelled text, utilizing conjunctive clauses to represent the particular facets\nof each category. Indeed, even the absence of terms (negated features) can be\nused for categorization purposes. Our empirical comparison with Na\\"ive Bayes,\ndecision trees, linear support vector machines (SVMs), random forest, long\nshort-term memory (LSTM) neural networks, and other techniques, is quite\nconclusive. The Tsetlin Machine either performs on par with or outperforms all\nof the evaluated methods on both the 20 Newsgroups and IMDb datasets, as well\nas on a non-public clinical dataset. On average, the Tsetlin Machine delivers\nthe best recall and precision scores across the datasets. Finally, our GPU\nimplementation of the Tsetlin Machine executes 5 to 15 times faster than the\nCPU implementation, depending on the dataset. We thus believe that our novel\napproach can have a significant impact on a wide range of text analysis\napplications, forming a promising starting point for deeper natural language\nunderstanding with the Tsetlin Machine.\n