Exploring the Decision Making Process in Statistical Data Analysis: A Qualitative Study of Quantitative Researchers

Quantitative data analysis is a cognitively demanding process. Inferences from quantitative analyses are often used to inform matters of public policy and to learn about social phenomena. However, as statistical analysis is typically conducted behind closed office doors, little is known about how analysts decide on the final statistical model that important policy decisions rely upon for determining the effectiveness of programs and policies. As social programming becomes increasingly reliant on quantitative data analysis, it becomes imperative to examine the quality of information stemming from these sources of evidence. This project presents the results of a qualitative research project that explores the cognitive processes of quantitative data analysts.;Seven quantitative analysts participated in the study. Participants were interviewed and observed during the course of analyzing a quantitative data set to uncover underlying cognitive processes used during data analysis. A typology of the decisions encountered by quantitative analysts in the sample is presented. A framework for decision making in quantitative analyses is presented that is borrowed from the field of cognitive psychology. Metacognition in quantitative data analyses is also explored.;Findings suggest that despite prescriptive procedures that are intended to facilitate objective analysis, quantitative analysts are frequently subject o external influences that may affect the results of quantitative analyses. Social factors and context influence how quantitative researchers make decisions during data analysis which call into question the objective nature of quantitative analyses. Quantitative analysts use story telling as a method of making sense of the findings from quantitative analyses. Implications for practice and for the teaching of quantitative methods are discussed.