Library

Advancing Vulnerability Classification with BERT: A Multi-Objective Learning Model

The rapid increase in cybersecurity vulnerabilities necessitates automated tools for analyzing and classifying vulnerability reports. This paper presents a novel Vulnerability Report Classifier that leverages the BERT (Bidirectional Encoder Representations from Transformers) model to perform multi-label classification of Common Vulnerabilities and Exposures (CVE) reports from the National Vulnerability Database (NVD). The classifier predicts both the severity (Low, Medium, High, Critical) and vulnerability types (e.g., Buffer Overflow, XSS) from textual descriptions. We introduce a custom training pipeline using a combined loss function-Cross-Entropy for severity and Binary Cross-Entropy with Logits for types-integrated into a Hugging Face Trainer subclass. Experiments on recent NVD data demonstrate promising results, with decreasing evaluation loss across epochs. The system is deployed via a REST API and a Streamlit UI, enabling real-time vulnerability analysis. This work contributes a scalable, open-source solution for cybersecurity practitioners to automate vulnerability triage.

cs.CRcs.AIComputer Science
Source

Paper

References (12)

01“Pre-processing techniques for vulnerability report analysis,”IEEE Transactions on Big Data
02“Improving vulnerability type classification with bert and multi-task learning,”IEEE Transactions on Information Forensics and Security
03“Bert for vulnerability detection in source code and reports,”Proc. IEEE International Conference on Cybersecurity
04“Vulnerability report classification using pre-trained language models,”IEEE Transactions on Dependable and Secure Computing
05“A bert-based approach for severity prediction of cybersecurity vulnerabilities,”IEEE Access
06“Deep learning for multi-label vulnerability classification,”Proc. IEEE International Conference on Machine Learning
07“Enhancing cybersecurity through automated vulnerability triage with nlp,”IEEE Security & Privacy
08“Improving multi-label classification with attention mechanisms in transformers,”IEEE Transactions on Artificial Intelligence
09“Automated triage of vulnerability reports using nlp and deep learning,”IEEE Transactions on Software Engineering
10“Multi-label classification of software vulnerabilities using deep neural networks,”Proc. IEEE International Conference on Software Engineering
11“Multi-modal vulnerability analysis using text and metadata,”IEEE Transactions on Information Security
12“Bert-based models for cybersecurity threat detection,”IEEE Transactions on Network and Service Management

Similar papers

V2W-BERT: A Framework for Effective Hierarchical Multiclass Classification of Software VulnerabilitiesSiddhartha Shankar Das, Edoardo Serra, M. Halappanavar et al.2021 · International Conference on Data Science and Advanced Analytics · 74 citations In Library
CVSS-BERT: Explainable Natural Language Processing to Determine the Severity of a Computer Security Vulnerability from its DescriptionMustafizur R. Shahid, Hervé Debar2021 · 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA) In Library
CVSS-BERT: Explainable Natural Language Processing to Determine the Severity of a Computer Security Vulnerability from its DescriptionMustafizur R. Shahid, Hervé Debar2021 · International Conference on Machine Learning and Applications · 83 citations In Library
Apply transfer learning to cybersecurity: Predicting exploitability of vulnerabilities by descriptionJiao Yin, Mingjian Tang, Jinli Cao et al.2020 · Knowledge-Based Systems · 102 citations In Library
A multi-task deep learning based vulnerability severity prediction methodChun Shan, Ziyi Zhang, Siyi Zhou2023 · 2023 IEEE 12th International Conference on Cloud Networking (CloudNet) · 3 citations In Library
Low Level Source Code Vulnerability Detection Using Advanced BERT Language ModelMansour Alqarni, Akramul Azim2022 · Applied Informatics · 24 citations In Library
VrT: A CWE-Based Vulnerability Report Tagger : Machine Learning Driven Cybersecurity Tool for Vulnerability ClassificationFahad A. Alshaya, Sultan S. Alqahtani, Yasser A. Alsamel2023 · International Workshop on Support Vector Machines · 6 citations In Library
A Self-Knowledge Distillation Approach to the Robustly Optimized BERT Approach for Common Vulnerabilities and ExposuresGurinder Pal Singh, Rohit Bajaj, Manish Kumar Hooda2025 · Proceedings of the 3rd International Conference on Futuristic Technology · 0 citations In Library
Automation of Vulnerability Classification from its Description using Machine LearningMasaki Aota, Hideaki Kanehara, Masaki Kubo et al.2020 · International Symposium on Computers and Communications · 53 citations In Library
Toward Extracting Learning Pattern: A Comparative Study of GPT-4o-mini and BERT Models in Predicting CVSS Base VectorsSho Isogai, Shinpei Ogata, Yutaro Kashiwa et al.2024 · 2024 IEEE 35th International Symposium on Software Reliability Engineering Workshops (ISSREW) · 5 citations In Library
© 2026 NYSGPT2525 LLC