| In the field of cyber attack defense,attack defense methods based on network security threat intelligence have gradually become popular.Threat intelligence is primarily used for information exchange and intelligence sharing,as well as for monitoring and alerting to cybersecurity incidents.At present,because open source threat intelligence is mainly released in the form of network security reports,most of these reports exist in the form of unstructured or semi-structured data,and have the characteristics of multi-source heterogeneity and large data volume,resulting in high manual extraction costs and low efficiency.Aiming at the needs of cyber threat intelligence mining in Internet big data,this paper automatically mines threat information from unstructured and semi-structured network security reports through entity recognition algorithm and parallel multi-label classification algorithm,and transforms it into standard threat intelligence in accordance with STIX2.1 format.The main research content and research work of this paper are as follows:(1)To address the problem of how to efficiently mine threat intelligence from open-source network security reports,a TI-NER-based Intelligence Mining(TI-NER-IM)method is proposed.Firstly,collected cybersecurity reports for nearly 10 years and labeled four categories of threat intelligence entities,and constructed a threat intelligence entity identification dataset;Then,in view of the lack of performance of traditional entity recognition models in the field of threat intelligence mining,a Threat Intelligence Entity Identification based on Self-attention Mechanism and Character Embedding(TIEI-SMCE)model is proposed,which fuses character embedding information.The potential dependency weights between words,contexts,and other characteristics are then captured through the self-attention mechanism to accurately identify threat intelligence entities;Secondly,a Threat Intelligence Named Entity Recognition(TI-NER)algorithm based on TIEI-SMCE model is proposed;Finally,a TI-NER-based Intelligence Mining(TI-NER-IM)method is designed and proposed.TI-NER-IM method enables automated mining of threat intelligence from unstructured and semi-structured security reports and converts it into standard threat intelligence in the STIX2.1 format.The experimental results show that compared with the BERT-Bi LSTM-CRF model,the F1 value is increased by 1.43 percentage points,TI-NER-IM method could mine threat intelligence automatically and efficiently.(2)In order to efficiently extract tactical labels from massive open-source intelligence,,Multi-Label Classify Based on Parallel Deep Forest(MLCPDF)was proposed.Firstly,the open-source cybersecurity report is collected and converted into a unified text format,and then five types of tactical labels are labeled to construct a multi-label dataset of threat intelligence tactic classification;Then,aimed at the disadvantages of low execution efficiency and lack of scalability under the serial mode of deep forest algorithms,Multi-Label Classify Based on Parallel Deep Forest(MLCPDF)algorithm was proposed,and the speedup ratio of the algorithm is improved with the help of broadcast variable and LZW compression algorithm;Finally,algorithm fuses mutual information entropy of label as the input feature,captures more potential label associated information,enhances the accuracy of classification.Finally,MLCPDF based Threat Intelligence Mining(MLCPDF-TIM)method was proposed.The experimental results show that MLCPDF algorithm has good node scalability and execution efficiency,and the MLCPDF-TIM method can effectively classify network security reports and mine threat intelligence tactical labels.Based on this,a complete threat intelligence compliant with STIX2.1 format is constructed. |
