Design And Implementation Of Large Scale Data Analysis System Based On Hana Sap Memory Computing

In the era of big data, information becomes the most important competitive force for companies. To manage, analyze and dig values from information by utilizing big data technology has become the main focus of information officers and CIOs.As one of the biggest global company, Sinopec‘s business scope covers the ecological chain of the entire petrochemical industry. The production and marketing channels of many Sinopec’s product lines are deployed nationwide and worldwide. According to plans and requirements set by Sinopec’s information department, the project means to speed up the construction of enterprise data warehouse(EDW). Sinopec has built up an EDW+BW system to support enterprise’s demand for daily analysis and reports, to help managing level to get real time information about the company’s functioning. Using this system, each business departments can generate daily, monthly report, analyze and monitor the business development.As business scale keeps extending recent years, Sinopec’s information technology and efforts have been strengthened accordingly. The data volume is undergoing an exploding growth with the construction of EPR and other information system. The data volume of BW system and financial reporting system grew rapidly. Up to early 2015, EDW data volume has already exceeded 30 T and growth become even faster each year. As demands become more complex, business departments regard the current EDW+BW model is insufficient to meet their requirement in aspects of system function, computing power and response speed. Under this background, Sinopec begins to use SAP HANA system for sales management.Computer architecture has changed rapidly recently. With the development of parallel processing produced by the fast communication between different processor cores, multiple cores processor has become a standard. SPA HANA is the latest technique proposed by SAP. SAP HANA which has taken advantage of innovative and high-end memory techniques to store data is suitable for processing large table database or relationship database, having an unprecedented performance. In the function library of HANA, APRIORI algorithm and K-means algorithm would analyze the data under the HANA model and make sales strategy.The article serves as a brief introduction of Sinopec enterprise data warehouse, methods and working process used in SAP Hana large scale data analysis will be its main focus. Hana memory computing technology, K-means, Apriori are used during the process, HANA SQLScript is adopted as its programming language.