Empirical Measurement of Performance Maintenance of Gradient Boosted Decision Tree Models for Malware Detection

Abstract

Important for effective, real-world machine learning (ML) or artificial intelligence (AI)-based malware detection systems is that models demonstrate both high discriminative performance at time of training and also demonstrate a high level of performance maintenance over time subsequent to training. That is, it is desirable that the models have a slow rate of performance decline over time as they encounter previously unseen malware threats. The study of malware detection model empirical performance maintenance on real-world data sets has not been widely addressed despite significant work on ML-based malware detection in general. In this work, we evaluate performance maintenance characteristics of models using a large, one million instance malware-goodware dataset spanning executables collected over one year in duration. Based on the outperformance of gradient boosted decision tree-based models, we investigate this category of model further and demonstrate models with performance and performance maintenance superior to that demonstrated in the previous ML-based malware detection literature. Given the large size of the dataset of real-world executables utilized, the insights into model performance maintenance may have valuable implications for real-world ML-based malware detection systems.