Analysis Of E-Commerce Information Based On Crawler And Data Mining

With the rapid development of the Internet industry,the leap-like evolution of computer-related technologies and the continuous expansion of industry scale innovation.The impact of the e-commerce platform on people's lives is self-evident.On the “double eleventh” day of 2017,Taobao's transaction volume on one-day platforms exceeded RMB 168.2 billion.People's daily lives can not be separated from e-commerce platform.In the e-commerce platform's online transactions,consumers can obtain various types of information by browsing the web pages of goods to determine whether to purchase,and these information can bring about better opportunities for marketing.Therefore,This paper will combine with The Crawler Technology and the corresponding Data Mining Methods to collect and inquire into the e-commerce webpage information,expect to build a systematic method from data extraction to data mining,help the e-commerce and consumers to explore demand better,and provide a reference for enriching marketing planning.From view of e-commerce consumer groups,the proportion of users in the 80 s and 90 s is nearly 80%,which is the core user of e-commerce spending.Consumers in this age group must undergo interviews and employment.Therefore,choosing suits as a research object is very representative.This paper through build a web automation crawler framework obtained more than 8,000 e-commerce webpage information on suits from Taobao.And the data mining method which is suitable selected by checking the data information.The final decision was to combination of descriptive analysis,regression analysis,and text clustering analysis to explore data values,thus providing high-quality data mining technology support for the rapidly evolving e-commerce industry.The main work of this article is divided into six parts:The first part: Introduction,mainly elaborates the research background,research content and structure arrangement.The second part: Introduces the method of data extraction and storage.It briefly describes the tools and its advantages and disadvantages,and lays a foundation for thedata crawling.The third part :The processing method of webpage information,which respectively describes the multivariate regression analysis model about the sales volume of goods and the “unsupervised learning” text clustering method used for webpage information.The fourth part: The development of K-Means algorithm,from the data preprocessing,VSM algorithm,IDFTF-algorithm and the evaluation of K-Means algorithm and so on,recorded the algorithm implementation process in detail.The fifth part: The empirical part,from the environmental configuration,data acquisition and storage,descriptive analysis,regression analysis,text clustering to build a set of systematic data exploration methods to maximize the exploration of data values to achieve research purposes.The sixth part:Summary and Outlook.It provides reference suggestions and support for data mining and market development of e-commerce website information.