| As the Human Genome Project was completed,life science research has stepped into the post-gene era of information sharing and analysing.In the process of exploring and analysing the biological genetic data,in which DNA sequence is one of the foucus for the study.k-mer and it’s frequencies in DNA sequence have important application including genome assembly,error correction and multiple sequence alignment.Due to the development of high-throughput sequencing technology and the rapid development of DNA sequence research,a large amount of data was produced in the researh institutes,which brings the urgent demand for data storage,efficent computing and analysising.New patterns of data storage and computing must be studied to solve the problem of the sharp increase of DNA sequence data.This paper started with the theoretical research,elaborating on the concept,characteristics,and application range of DNA sequence and k-mer.Then the research status of k-mer frequency counting algorithm of DNA sequence was analyzed,and foucus on the advantages and disadvantages of BTKC algorithm.At the same time,the characteristics and application of MapReduce programming model were studied,and the characteristics and design principles of Hadoop distributudied processing platform were also studied.From the analyzing and comparing,MapReduce programming model and Hadoop processing framework are convinced to be superior than the traditional computing framework on mass sequence data’s storage and computing.Then,this paper studied DNA sequence k-mer frequency counting algorithm-BTKC(backtrace k-mer counting algorithm),and analysis the principles of BTKC algorithm and its’ advantange and disadvantange.In order to solve the problems of the algorithm,an improved algorithm was proposed.The improved algorithm specifies the available memory and the Hash table loading factor by the user,and then the sequence data partitions are loaded into the Hash table for iterative calculation.At the same time,this paper designs a sort strategy to sort out the results of the Hash table.Experiments show that the algorithm can handle arbitrary size of data sets in a limited memory,and the results of Hash sequencing can be sorted to output in the case of large data sets,by which improves the efficiency of the improved algorithm.Finally,for the problems of storage and computing method for large-scale DNA sequence data,the applicability of Hadoop framework and MapReduce pragraming model was analyzed based on theories and fulfillments.This paper then studied the parallelization of the improved algorithm,and proposed a k-mer frequency counting algorithm of DNA sequence based on MapReduce,which was tested on Hadoop cluster for the effective of the algorithm. |
