Accuracy Assessment Of Large Scale Land Cover Datasets

According to the need of the branch project of the National Key Basic Research Program on Global Change(Grant No.2011CB952001),titled "Spatial and Temporal Characteristics of the Large-Scale Land Use and Cover Change of China over the Past 30 Years",in this study,accuracy assessment of five kinds of global land cover datasets in China and adjacent region were performed to guide the data users to select and make better use of these datasets.Quality of five commonly used global land cover datasets—IGBP DISCover,UMD,GLC2000,MOD 12Q1-2001,GlobCover2009—were evaluated in this study.Firstly,characteristics of these five kinds of datasets were analyzed to explore their potential influence to the accuracy of classification result.Secondly,a comparison analysis was did to reveal the difference among these five datasets from two sides:area agreement and spatial agreement.Thirdly,four accuracy measures-overall accuracy,producer's accuracy,user's accuracy,and Kappa coefficient were calculated combined with two periods validation samples collected via visual interpretation of the high resolution image on Google Earth.Finally,and a geostatistical model was used to evaluate the spatial variation on accuracy of these five datasets.The main contents and results of the study are presented as follows:1.The results of characteristic analysis show that satellite data,classification system and classification method used in each land cover dataset can affect the classification result significantly.(1)The outdate urban data came from Digital Chart of World lead to the underestimation of urban and built-up in IGBP DISCover and UMD.(2)For UMD,training sample site of wooded grassland—one kind of typical tropical landscape—was not only distributed in tropical regions,but also in temperate and boreal zones,which result in a serious overestimation of its area.(3)For GLC2000,The urban and built-up in China was obtained via visual interpretation of SPOT Vegetation image,some little cities were lost affected by some subjective factors.(4)Supervised decision tree,the classification method used in UMD and MOD 12Q1-2001,can easily make domain class be exaggerated and small class be lost by assignment all pixels in the terminal note of the decision tree to the domain class.(5)IGBP classification system and FAO LCCS define two different upper limit about vegetated cover for barren,thus area and spatial pattern of barren in datasets with IGBP classification system is obvious different from datasets with FAO LCCS.2.The aim of comparison analysis is to reveal the difference in area and spatial pattern of the same class among five datasets.The results of area comparison show that the difference of per-class area in five datasets is significant.(1)MOD12Q1-2001 has the largest urban and built-up area,which is approximately 5.0,5.0,5.0 and 6.4 times of urban and built-up area in IGBP DISCover,UMD,GLC2000 and GlobCover2009 respectively.(2)GlobCover2009 has the least grasslands area,which is just about 12.1%,10.4%,14.7%10.4%of the grassland area in in IGBP DISCover,UMD,GLC2000 and GlobCover2009 respectively.(3)GLC2000 has the largest forest area,with the aggregated classification system,which is about 3 times of forest area in GlobCover2009.The results of spatial agreement analysis show that spatial pattern of some class,especially for five kinds of forest,shrublands,croplands,grassland and barren,among five datasets is obviously different and ranges from region to region.Furthermore,spatial pattern of barren and shrublands in five datasets has certain relationship with their original classification system,Land cover datasets with same original classification system exhibit better agreement than those with different original classification schemes.In the study area,five datasets show high spatial agreement only at some regions,such as Tarim basin,Tibetan plateau,North China Plain.3.Accuracy assessment of five datasets calculated four accuracy measures:overall accuracy,producer's accuracy,user's accuracy,and Kappa coefficient.(1)Among the five datasets,GLC2000 has the highest overall accuracy and Kappa coefficient.Except for mixed forest,shrublands,cropland/natural vegetation mosaics,all remain kinds of classes in GLC2000 have a relatively higher producer's accuracy compared with other four datasets.(2)UMD has a lowest overall accuracy during five datasets,however,its producer's accuracy of evergreen needleleaf forest,evergreen broadleaf forest,deciduous needleleaf forest,and deciduous broadleaf forest is the highest among the five datasets.(3)When 16 kinds of classes in IGBP classification system were aggregated into 10 kinds of class,the overall accuracy of five kinds datasets rises significantly,and user's accuracy of forest in five datasets is all above 70%,but producer's accuracy of forest in is still low.4.A simple geostatistical model were used to evaluate the spatial variation on accuracy of these five datasets,at each location,connected domain boundary or a fixed size moving window was used as geographical constraint rule to define a subset of validation samples used in local accuracy assessment.The results show that accuracy of five datasets ranges from region to region.(1)The accuracy of all five datasets is low in Southeast Coastal Area in China,the eastern Tibetan,Yungui Plateau,Sichuan Basin.(2)The accuracy of all five datasets is high in Tarim basin,Tibetan plateau,North China Plain.(3)The accuracy of five datasets in North Tibetan Plateau,Northeast part of Inner Mongolia Plateau,Great Xingan Mountain,Xiaoxing'an Mountain has significant difference.(4)Accuracy of each dataset has closed relationship with the degree of its landscape complexity.Generally,regions with high homogeneity have higher accuracy than those with complex landscape.Characteristics analysis about the satellite data,classification system and classification method used in five datasets and its potential influence to the accuracy of classification result can provide a guide to new dataset production.Information about area agreement,spatial agreement,four kinds of accuracy measures and spatial variation on accuracy of these five datasets can guide the data users to select and make better use of these five kinds of large scale land cover datasets.