| The principal objective of this study is to investigate the potential value of control charts, as decision aids, to one important aspect of operational auditors' production efficiency evaluations: assessments of production process stability based on analyses of output/input ratios and related measures of performance. Two subject groups evaluated the stability of hypothetical production processes based on analyses of efficiency measurement data presented in tabular and graphical formats. The graphs presented to the second subject group were supplemented with center lines and control limits and brief descriptions of what these lines represented. In addition to comparing the decision accuracy of the two subject groups, the researcher compared the decision accuracy of each group to (1) a benchmark equal to random accuracy, and (2) the decision accuracy of statistical process control (SPC) models. Decision accuracy was measured relative to "true" process states as modeled using computer simulation. The average decision confidence levels of the two subject groups were also compared.;The researcher concludes that inexperienced auditors (as represented by experienced auditing students in this experiment), with no SPC training, can not consistently make correct stability assessments at rates higher than expected by chance when their assessments are based solely on visual analyses of efficiency measurement data presented in tabular and graphical formats. Furthermore, estimates of process average and spread, without interpretation of how this incremental information is used in making stability assessments, do not contribute significantly to the accuracy of, or the confidence in, such decisions. Except for the most obvious cases, p chart and c chart decision models will outperform inexperienced auditors, with no SPC training, at assessing performance stability, when the auditors' decisions are based strictly on visual analyses of efficiency measurement data supplemented with lines representing estimates of process average and spread. However, the relative power of X charts to correctly signal stability and instability is less than satisfactory. |
