Design And Implementation Of Vertical Search Engine And Data Analysis System Oriented To Network Politics Inquiry

With the popularization of the Internet and the promotion of information technologies such as Big Data and Artificial Intelligence,the mode of government governance has undergone digital transformation step by step from offline to online.Among them,the Network Political Inquiry platform is an important way to shorten the distance between the government and the people and to simplify the cost of communication between the people and the government.The Network Political Inquiry has gradually developed into a new form of democratic politics by virtue of the advantages of real-time interaction,equality of participation and anonymity.In recent years,through the establishment and operation of major political inquiry portal websites,a large number of first-hand data that can reflect social conditions and public opinions and hot social issues have been accumulated.However,due to problems such as scattered data sources,difficulty in obtaining data statistics,and single function of political inquiry platform,users can only rely on general search engines to retrieve data.This retrieval method usually contains a large number of meaningless links,repeated data and expired resources,which makes it difficult for users to obtain more professional,in-depth and accurate information in a short time,especially for experts and scholars in specific fields,who need certain technical support in order to obtain valuable information in batches.Based on this,this paper studies the technologies related to Big Data,web crawler,distributed system,vertical search engine and data analysis,and designs and implements a vertical search engine and data analysis system for the field of Network Political Inquiry.In this paper,based on the full text retrieval tool Lucene,weight setting and synonym extension query optimization are carried out according to the characteristics of network political query data,so as to improve the precision and recall rate of the search engine.At the same time combining the Hadoop ecosystem under large data related technology,the distributed storage of data and service and deployment,build a web crawler system based on distributed computing platform and network oriented ask vertical search engine in the field of politics,and offers the data statistical analysis visualization of front page,in the realization of system functional requirements on the basis of fully considering the distributed cluster of high performance,high availability,extensible and fault tolerance.The system designed in this paper will carry out real-time crawling,accurate management and full-text retrieval of the political inquiry related data on the network,provide convenience for users to retrieve political inquiry information and experts and scholars to query public opinion data,improve the efficiency of information retrieval and data acquisition,reduce the technical difficulty,and have great significance for promoting the research in the related fields of government affairs and the digital governance of the government.By comparing with the search results of ordinary search engines,the accuracy and recall of this system are higher than those of general search engines.It has certain practicability and can provide political data retrieval services for the general public,related experts and scholars and government personnel.