The Research And Implementation On Xiang Jiao Cloud Of Distributed Log Data System

For complicated multi-system applications and multi-user platform,the log data from the application systems witnesses a daily increase.However,the log data of different applications is stored in different systems separately,on the more complex log collection managing method,cumbersome,not simply,conveniently massive log data storage and analysis.In this paper,the log management Xiang Jiao Cloud platform for the study,how simple and easy the log management and analysis in complicated multi-system applications and multi-user platform.This major research work are as follows:1?Focusing on platform for complicated multi-system applications and multi-user systems consisting of a proposed log collection and management solutions,the distributive technology of log data management based on Hadoop.Based on the Hadoop distributed log system in the Hadoop platform,the use of educational electronic identity design based on REST interface security authentication method to collect the log and the timing task push log two ways to collect the log,and then after data cleansing conversion or real-time task analysis and other processing,stored in the database to facilitate the application of the system query and management.2?Each module based on Hadoop distributed logging system was realized.Based on the Hadoop distributed log system from the Hadoop platform to build,server software access log module,business log module,data authentication module four aspects of each module in detail to achieve,achieve server software access logs and logs from collection business analysis to storage and other functions.3?Based on Hadoop distributed logging system tested by experiment and Xiang Jiao Cloud platform.Were written test from functional testing,business logs high concurrency,high traffic logs concurrent queries test,high availability testing,system scalability test five experimental and testing.The test results show that the research and implementation on Xiang Jiao Cloud of distributed log data system can be simply and easily to complex multi-application system log storage and analysis that can supports functions including user-defined lag data from the third-party application and data authentication of client-side operation.Meanwhile,this system is of high usability and expandability,for available nodes can be added anytime to avoid single-node malfunction in the distributive log data system of Xiang Education Cloud,without influencing the whole service.