The Design And Implementation Of Early Warning Analysis System Based On Electronic Fence Data

With the rapid development of information technology,big data technology has been more and more applied,especially in the field of public security.In order to effectively monitor key targets,mine effective intelligence,control the regional situation,and improve the efficiency of adjudication,public security departments have deployed a large number of electronic fence based information collection equipment.However,with the increase of acquisition equipment and the growth of data scale,the intelligence analysis is not sufficient and the control and early warning is not timely.Therefore,it is of great practical significance to study the application of big data in the early warning analysis of electronic fence.Starting from the big data application requirements of public security departments,aiming at the problems of intelligence analysis and real-time early warning,this paper studies and realizes the electronic fence early warning analysis system based on big data.The system adopts the lambda big data architecture of stream batch hybrid,which is divided into four functional modules: data transmission,data storage,query analysis and real-time early warning.The data transmission module is mainly used to transmit the real-time monitoring data of the electronic fence equipment.The module is based on Kafka.It is designed around the theme division,multi consumer groups,message offset submission and synchronous replica collection,and can provide high throughput and high reliability transmission services for the whole system.The data storage module is mainly based on mongodb database,which is the basis of massive electronic fence data.The module adopts the design of separating hot and cold data,stores the recent data independently,and stores the historical data in different databases and tables.On this basis,it provides a balanced and efficient read-write service for the upper application by building a partitioned cluster,reasonably designing the index,and configuring read-write concerns.Query analysis module is an important application module of the system,which realizes many analysis applications including activity rule analysis,one machine multi card analysis,target collision analysis,partner accompany analysis,network account association analysis.In order to ensure the performance of query analysis,the module adopts cache acceleration,offline precomputing,memory computing and other optimization technologies.In addition to using the basic mongodb pipeline query framework,it also introduces the SQL query engine Presto based on parallel computing.The real-time early warning module mainly provides users with real-time statistical data and alarm information of specific target or specific area.The module builds a lightweight real-time flow processing framework based on Kafka stream,realizes realtime statistical applications including traffic flow and people flow through the topology processing model,and realizes many real-time early warning applications by using the hash data model and overdue callback in redis.The system has been designed and developed,and is in the trial operation stage.After trial,the system can meet the needs of early warning analysis of electronic fence data,and has good scalability.At the same time,the theory,architecture,application and technology involved in the paper have certain reference value for practitioners in the field of public security or big data technology practitioners.