Capturing the symptoms of malicious code in electronic documents by file's entropy signal combined with Machine learning

Abstract Email cyber-attacks based on malicious documents have become popular techniques in today’s sophisticated attacks. Persistent efforts have been made to detect such attacks, but there are still some common defects in the existing methods, including the inability to capture unknown attacks, high overhead of resource and time, and only can be used to detect specific formats of documents. This study proposes a new method named Entropy Signal Reflects the Malicious Document (ESRMD), which can identify malicious documents based on the entropy distribution of the file. ESRMD is a machine learning classifier, which differ from the traditional approaches in that ESRMD extracts both global and structural entropy features from the entropy sequence, enduring it the ability to deal with various formats documents and fight against the parser-confusion and obfuscated attacks. To assess the validity of the proposed model, we conducted extensive experiments on a collected dataset which contains 10,381 samples, including malware (51.47%) and benign (48.53%) samples. Through extensive experiments, ESRMD showed its superiority comparing with some leading anti-virus engines and prevalent tools, achieving good performance on the true positive rate and ROC with the value of 96.00% and 99.2% respectively.