Research On Schemes Of Massive Data Stream Processing In Manufacturing Internet Of Things

The technology of manufacturing internet of things is an emerging new manufacturing type and service mode in modern manufacturing industry. It can greatly improve the service level of manufacturing information in China, improve the market competitiveness and the influence of manufacturing enterprises in our country. Under the environment of manufacturing internet of things, large number of sensing devices are deployed in large-scale manufacturing filed to monitor the production situation and they produce massive manufacturing data stream. However, since the massive manufacturing data stream has the following characteristics:huge data capacity, multi-source data content, complex data structures, large disorder of data order, strong uncertainty of data values, a variety of data expression types, small value of data information, high response needs, etc., that makes the existing data process methods for massive data difficult to completely support the processing of massive data above and that makes the existing manufacturing enterprises difficult to directly get their required information from the massive information stream, thus seriously affecting enterprise production management and control level.Firstly, the dissertation carries out the in-depth research on the key technologies on massive data stream based on the analysis of the data characteristics existed in the current massive data stream, the current processing challenges and the existing research results and proposes and designs a series of efficient methods for detecting and processing massive data stream in manufacturing internet of things. The main works carried out and main results gotten in this dissertation are summed up as follows:1) Aiming to solve the difficulty in dealing with massive data stream in time due to its large amount of data over massive multi-source data stream in manufacturing internet of things, we propose and design a complex event detection method based on hash table structure toward massive multi-source data stream in manufacturing internet of things in this dissertation, which can solve the problem of the difficulty in dealing with in time due to lots of backtrackings and repeat search operations existed in the current complex event detection methods during the detecting process for massive data stream. Our suggested method provids a new method for high-efficient detection massive multi-source data stream in manufacturing internet of things. The simulation results show that our proposed scheme in this dissertation can greatly reduce detection time, lower memory consumption and improve event throughput compared with the existing methods.2) Aiming to solve the difficulty in detecting massive data stream in time due to its high disorder over massive data stream in manufacturing internet of things, we put forward and design a complex event detection method based on improved NFA and hash table structure toward massive data stream in manufacturing internet of things, which can solve the problem of the detecting in time due to the difficulty in determining the inner relationship and building the inner structure in current complex event detection methods in detecting massive disordered data stream. Our suggested method provides a new way for high-efficient detecting massive disordered data stream in manufacturing internet of things. The simulation results show that our proposed scheme in this dissertation can be more efficiently to complete a complex event detection compared with the current methods over massive disordered data stream processing of manufacturing internet of things.3) Aiming to solve the difficulty in detecting in time due to its strong uncertain in massive data stream in manufacturing internet of things, we propose and design a complex event detection method based on directed acyclic graph toward massive uncertain data stream in manufacturing internet of things, which can solve the difficult detection problem in time due to the large combination number in the current complex event detection methods during detecting process massive uncertain data. Our suggested method provides a new idea for how to efficiently detect massive uncertain data stream in manufacturing internet of things. The simulation results show that our proposed scheme in this dissertation is very effective in reducing detection time, lowering memory consumption and improving event throughput compared with the existing methods in detecting massive uncertain data stream in manufacturing internet of things.4) Aiming to solve the problem of low decoding performance due to the need to look up variable length tables frequently in decoding massive video stream with the large capacity and small value density, we firstly propose and design table look-up algorithms based on hash table query and index search in this dissertation. Through the introduction of hash table query technology and index search technology into the looking-up table of CAVLC decoding, we use hash table query and index search technologies to reduce the searching and matching operations of code_length, value of code_suffix and code_word, and then solve the problems of low decoding performance over processing massive video stream by reducing high memory access, long search time and big storage spaces in current table look-up methods, thus improving the whole decoding performance of massive video stream. The simulation results show that our proposed table look-up schemes based on hash table query and table look-up scheme based on index search can save 25%、35% table memory access, reduce about 20%、40% table look-up time and save 1056、923byte table storage spaces compared with TLSS method respectively.5) Since the improved table look-up algorithm based on hash table query and table look-up algorithm based on hash-index search still use table look-up way, they still affect the improvement of whole decoding performance due to the need for some memory access, search time and storage spaces. In order to further improve the whole decoding performance, a table look-up algorithm based on program code execution is firstly proposed and designed in this dissertation. This method fully realizes no-table looking-up way for CAVLC decoding by using program code execution to instead of hash table query and index search method, thus improving whole CAVLC decoding performance by reducing the memory access and search time and storage spaces. Our scheme provides a new method for high-efficient table look-up in CAVLC decoding. The simulation results show that our proposed scheme can save 100% table memory access, reduce about 45% table look-up time and save 2320 byte table storage spaces compared with TLSS method in CAVLC decoding and make it more high-efficient to complete the fast decoding function in massive video data stream of in manufacturing internet of things.