Integrating NLP of dark web forum data with behavioural telemetry from the DAPT2020 dataset, an AI-driven cyber threat intelligence, which is multimodal, is generated and among possible IOCs types, such as IP addresses, CVE numbers, and Malware names, obtained from unstructured text using a finetuned BERT-based Named Entity Recognition (NER) model. The returned IOCs are then mapped to the synthetic network traffic, intrusion alerts, and host-level logs provided by DAPT2020, encompassing multi-stage APT activities. This combination of forensic and behavioural intelligence makes it easier to identify new threats by providing context and evidence of exploitation. The evaluation showed that the NER model achieved an F1-score of 91.3% in the IOC extraction using annotated cybersecurity data. When network telemetry was incorporated, overall detection accuracy was improved by 17.6%, and false positives were reduced by 24.2% compared to a text-only baseline. The use of time series visualisations and IOC chain graphs is also aimed at amplifying the analyst's ability to understand the evolution of the threats and that of the attackers. Through the use of annotated datasets, open-label datasets, and manual annotations to train machine learning models, the platform addresses issues such as data labelling, timestamp alignment, and the ethical use of information gathered from the dark web. The proposed integrated approach presented here can help scale up cyber situational awareness, enable proactive threat hunting, and improve cyber operational decision-making.
Paper
Full text
Fusing Dark Web NLP and DAPT2020 Telemetry for Multimodal Cyber Threat Intelligence
Semantic Scholar · 2025
Abstract
Integrating NLP of dark web forum data with behavioural telemetry from the DAPT2020 dataset, an AI-driven cyber threat intelligence, which is multimodal, is generated and among possible IOCs types, such as IP addresses, CVE numbers, and Malware names, obtained from unstructured text using a finetuned BERT-based Named Entity Recognition (NER) model. The returned IOCs are then mapped to the synthetic network traffic, intrusion alerts, and host-level logs provided by DAPT2020, encompassing multi-stage APT activities. This combination of forensic and behavioural intelligence makes it easier to identify new threats by providing context and evidence of exploitation. The evaluation showed that the NER model achieved an F1-score of 91.3% in the IOC extraction using annotated cybersecurity data. When network telemetry was incorporated, overall detection accuracy was improved by 17.6%, and false positives were reduced by 24.2% compared to a text-only baseline. The use of time series visualisations and IOC chain graphs is also aimed at amplifying the analyst's ability to understand the evolution of the threats and that of the attackers. Through the use of annotated datasets, open-label datasets, and manual annotations to train machine learning models, the platform addresses issues such as data labelling, timestamp alignment, and the ethical use of information gathered from the dark web. The proposed integrated approach presented here can help scale up cyber situational awareness, enable proactive threat hunting, and improve cyber operational decision-making.