| With the rapid development of the Internet and sensors,the amount of data increases dramatically.With the explosive growth of data,people enter the era of big data.In the era of big data,people need to have a variety of means and tools for mass data,in order to find the great value of it.However,with the increasing amount of data needed to be stored and processed in big data system,some performance problems of the system become more and more serious,which lead to the decrease of the processing speed and make the system not return the results in a timely manner.At most serious situation,the system even cannot provide services.In order to improve the system performance,the system needs to be optimized according to the architecture level of the big data system.The performance optimization of big data system is a complex work,which covers many aspects of the system.Therefore,in the process of system optimization,all factors that affect the performance of the system should be considered.Based on research of big data system architecture and related technology,from the actual situation of the development and application of the system,the performance optimization of big data system is presented in the following four aspects:data collection optimization,data processing optimization,database optimization,web server optimization.These four aspects relate to the big data system architecture in the multiple levels.At the end of this paper,we introduce a big data system,which mainly aims at analyzing and processing all kinds of TV series data,movie data and TV program data.This paper describes the functions of various modules,database design,as well as the performance programs in the system.To solve each performance problem,this paper illustrates specific optimization process and technical details,and also provides a reference method for the optimization of big data system,and finally uses the test tool to test the performance of the optimized system. |
PDF Full Text Request