| In recent years,the semantic Web developed rapidly,the RDF data has been supported and researched widespread.As the scale of RDF data is bigger and bigger,the RDF data query researches in traditional centralized environment are increasingly unable to adapt to the requirements of the data query field,especially for the top-k RDF data query.With the gradually development of the distributed field,the distributed system which has massive storage ability and parallel computing ability gradually become a hotspot to solve this kind of problem.And Spark distributed system is an outstanding one.Based on the Spark distributed computing system and the HBase distributed storage system,this paper designed a method for the large-scale RDF data storage and researched the top-k query algorithm.This paper analyzes the advantages and disadvantages of traditional RDF data storage model,designed and implemented a RDF storage index structure which based on the HBase storage characteristics to apply to top-k query.Based on the storage mode and draw lessons from the characteristics of the traditional top-k query processing technology,this paper proposed a top–k query plan STA query algorithm which committed to reducing the connection operation of RDF data during the running process of algorithm.On this basis of the STA query algorithm and according to the characteristics of Spark distributed system data processing,this paper improved the STA algorithm and puts forward a new top-k query algorithm SSJA query algorithm,which efforts to reduce the sorting related operations for the intermediate data.In addition,this paper also constructed the HBase distributed storage environment and the Spark distributed computing environment,and implemented the two algorithms then tested them.The experimental results show that the SSJA algorithm is better than the STA algorithm in performance and applicability. |
