Research On Anomaly Detection Based On Fly Locality-sensitive Hashing

With the development of communication technology,the Internet of things system is widely used in more and more fields.How to ensure the safe operation of the Internet of things system has become a hot research issue.The key to ensure the security of the Internet of things system is to detect the real-time abnormal data flow in the logs of the Internet of things system and locate the abnormal behaviors accurately.At present,many researchers have carried out in-depth research in related fields,such as using local sensitive hashing to re-establish the index structure of high-dimensional data,realizing the accurate positioning of data,using classification algorithm to determine large-scale real-time data flow exceptions and so on.However,there are still some defects in the existing algorithms,such as the use of local sensitive hash algorithm to duplicate mapping data will cause data distortion and affect data search and location.When using classification algorithm to determine large-scale real-time data,it is often difficult to respond in time,and the robustness of the model is low.In view of the security risks existing in the existing Internet of things security technology,this paper proposes two novel algorithms: one is a near neighbor search algorithm based on the Fly Locality-Sensitive Hashing for the retrieval of large-scale ultrahigh dimensional data,which realizes the accurate location and search of large-scale highdimensional data in the Internet of things;the other is to improve the search efficiency for large-scale real-time data flow in the Internet of things An anomaly detection algorithm based on Fly Locality-Sensitive Hashing is proposed.The algorithm realizes the accurate separation of abnormal data and normal data in a short time.In addition,this paper proves the maximum regression distance preserving difference theorem,and further improves the dimension expansion theory.Therefore,the main contributions of this paper are summarized as follows:(1)To solve the problem of super high dimensional large-scale data retrieval in the Internet of things system,this paper proposes a novel near neighbor search algorithm,which effectively combines the biological perception process and FJLT transformation matrix,and can complete the accurate positioning and retrieval of data in a short time.(2)Aiming at the problem of anomaly detection of large-scale real-time data flow in the Internet of things,this paper proposes a new anomaly detection algorithm based on the Fly Locality-Sensitive Hashing and Bloom Filter.The algorithm uses the local sensitive hash of Drosophila to map the data and the ability of Bloom filter to process the data quickly.At the same time,it combines the probability calculation of false-positive of Bloom filter and the theory of dimension expansion process Data separation property,training algorithm model and threshold detection data.(3)Further improve the theory of dimension expansion.In this paper,we find that the loss of similarity in the process of data dimension reduction is greater than that in the process of dimension expansion;the dimension of extended dataset can not only preserve the similarity between similar data,but also more easily split the abnormal data.