| With the development of Internet,Internet advertising is becoming more and more popular,and well-known companies in the industry have opened their advertising platforms.Advertising fee deduction system is an important module of advertising platform.This thesis designs and implements a micro service based advertising fee deduction system according to the deduction demand of advertising trading platform in the real Internet scene.In this thesis,Java is used to complete the development of advertising fee deduction system.The whole part of advertising fee deduction is based on the microservice framework sofa.Sofa is ant financial's open-source RPC framework,which provides excellent communication protocol and communication framework,as well as basic capabilities such as load balancing,traffic forwarding,link tracking,link data transmission and fault elimination.Advertising fee deduction module and advertising data platform are the main two parts.In order to deal with the short-term data peak,the deduction module optimizes the deduction logic of the deduction system by using Kafka message queue.In order to avoid repeated deduction,a global unique ID generation strategy based on the database auto increment primary key is proposed.The global unique ID will be added to each exposure log to ensure that the advertising expenses of the merchants are not deducted by mistake.In order to enhance the timeliness of the fee deduction system,an algorithm is proposed to allocate the fee deduction thread based on the user ID module,so that the system can still maintain a high timeliness when it processes huge data every day;an advertising data platform is established to synchronously store the exposure information of the advertisement into HDFS through the timing task every day,and spark is used to aggregate the data of multiple dimensions,so that the advertisement can be exposed The light situation is more clear to the user.Spark processes advertising data in a distributed environment,making full use of the characteristics of the distributed system,such as high efficiency,fault tolerance,etc.,to provide highly available and scalable services.After testing,the system can accurately complete the function of advertising fee deduction,daily processing of massive data,design performance higher than the actual performance,can calmly cope with traffic growth.In the development process,link tracking is used for troubleshooting,which can quickly solve the problem in extreme cases.The deduction system is efficient,reliable and excellent to fulfill the deduction demand,which has a certain practical value. |