Research And Implementation Of Data Access Layered Cluster System Based On Netty In Internet Of Things

Internet of Things data access system is the bridge to realize the interactive connection between terminal sensing devices and application services,and is the key hub to receive terminal data first in the Internet of Things application system.With the rapid development of the Internet of Things,the number of terminal devices is increasing explosively.In the face of massive terminal data uploading,the concurrent receiving capacity and peak load capacity of the Internet of Things data access system will be the key point to guarantee the quality service experience of the Internet of Things application.To solve the problem of massive data access,this thesis conducts in-depth research from the aspects of Internet of Things system architecture and data processing links,and designs and implements a hierarchical data access cluster system based on micro-service architecture.Firstly,aiming at the problems of traditional data access server business module coupling,difficulty in flexible expansion and poor concurrent data access ability,a layered data access cluster architecture design scheme based on Netty is proposed.According to the idea of microservice architecture,the system modules are split in fine granularity,the complex time-consuming business modules are moved to the parsing layer,and the access layer is focused on data receiving and forwarding.Combined with Zookeeper,a node metadata publishing and subscription model is constructed to realize dynamic connection coordination and management among nodes in the cluster.Meanwhile,a lockless inter-thread task scheduling strategy based on Disruptor framework is proposed to further improve Netty thread resource utilization.Experimental results show that this design can effectively improve the response rate and throughput of the server to cope with high concurrent data access.Secondly,in order to further improve the ability of data access system to deal with burst traffic,on the basis of the original hierarchical data access cluster system,a differentiated link processing scheme based on the customized message protocol is designed,in which Kafka message queue is introduced in the non-core link to achieve asynchronous communication and concurrent buffering between nodes.At the same time,in order to improve the ability of data links to adapt to various service requirements,a dynamic link scheduling strategy was proposed based on the lightweight Remote Procedure Call(RPC)framework to meet the requirements of link division under different service conditions.It is tested that the differential link processing scheme can effectively reduce the load pressure of the system during peak flow period.