Analysis Of The Quality Of Results Reporting Of L2 Questionnaire Surveys In And Outside China

In recent years, questionnaire surveys, as an important means of quantitative research, have been increasingly drawing the attention of researchers in L2 research. Especially in the field of foreign language teaching and acquisition, more and more papers are using questionnaires to collect data needed (秦晓晴,2009:2-3). The main reason for the phenomenon lies in the fact that the questionnaire can be easily designed, using the questionnaire can collect data in a wide range, and, more importantly, the data collected by questionnaires can be analyzed quantitatively with a higher efficiency (Dornyei & Taguchi,2010:1). The standard design of questionnaires, the proper selection of samples and the appropriate data analysis approaches are crucial to ensuring the study quality, but how to cope with and report the study results of the questionnaire also occupies a significant position in a good study. Reporting the study results reasonably and clearly can help the researcher to describe and elaborate the study results, the reader to better understand the research findings and other researchers to do replicate research or meta-analysis research. Lacking in specific results reporting standards at present, L2 studies have to report the study results based on the standards from other fields such as psychology, education and linguistics (Larson-Hall & Plonsky,2015), which is a regret to some extent in the field of L2.In an attempt to solve the problem above, the thesis first reviews the standards for reporting the research results at home and abroad so as to provide the results reporting standards of L2 questionnaire surveys. Then, the study makes content analysis of 504 articles collected and selected from 14 key domestic and overseas journals between 2000 and 2015 in which the questionnaire is used as the main data collection tool. Next, based on the standards of descriptive statistics like samples, means and standard deviations, inferential statistics such as effect sizes, confidence intervals, exact p values and instrument reliability, and visual presentation of data and data sharing proposed by the author, the study explores the characteristics of results reporting and analyzes the similarities and differences between them. The study obtains the following results:1) In the reporting of descriptive statistics, there is almost no change in the trend of sample sizes over the years in and outside China and the using of means and standard deviations has decreased over the years in and outside China; most researchers at home and abroad presented sample sizes, and a majority of the researchers reported means and standard deviations, while the number of foreign researchers was larger than that of domestic researchers.2) In the reporting of inferential statistics, in terms of confidence intervals, slight increases have taken place in China in the reporting of confidence intervals as a whole, while there is no improvement in the aspect outside China over the years; with regard to effect sizes, there is a slight increase taking place in China and a relatively dramatic increase has been found outside China over the years; in terms of exact p values, there is a relatively dramatic decrease taking place in China while a dramatic increase takes place outside China in the reporting of exact p values; in terms of instrument reliability, a dramatic increase has taken place in the reporting of instrument reliability over the years in China while there is a slight increase taking place in this aspect as a whole outside China. In the detailed reporting, most researchers didn’t report confidence intervals and effect sizes, while foreign researchers were more than the counterpart in China in the reporting of effect sizes; many researchers reported exact p values and reliability coefficients, while more domestic researchers reported the exact p value than foreign researchers and more foreign researchers reported reliability coefficients than domestic researchers.3) In terms of visual displays, there is a great increase in the using of both tables and figures in China while there is a relatively dramatic decrease in the above aspect outside China over the years. And less than 40% of the sample papers presented both tables and figures in and outside China, while the number of foreign researchers in the sample was larger than that of domestic researchers. The majority of the researchers usually reported the data with tables, and fewer researchers presented it with figures.4) With regard to the raw data, no change takes place in providing the raw data in and outside China over the years, and most of research in the sample at home and abroad didn’t provide access to obtain the raw data.This thesis has great significance. In theory, the study establishes an evaluation standard for results reporting in L2 questionnaire surveys, filling in the gap existing in the fields of results reporting, which provides a theoretical basis to some extent in evaluating the study quality. In practice, for one thing, it can provide a fact evidence for other researchers to evaluate the overall study quality of China’s L2 empirical research objectively, which is beneficial to standardizing the domestic L2 research and improving the quality of L2 research in China so as to make it acceptable to the international L2 studies. For another, it can find out the problems and weaknesses in domestic L2 research, enhancing the related teaching and guidance in paper writing.