Performance Assessment And Standards Making Of Hedges Q Homogeneity Test

In psychological research fields, as a reliable and scientific methodology to summarize primary research quantitatively, meta-analysis method were used widely. In all works of meta-analysis, detecting heterogeneity of effect sizes of primary studies was a crucial part of meta-analysis, and it was also the first one of the three main goals of meta-analysis that should be resolved. Thus, investigating the performance of homogeneity test of effect sizes is very important to use meta-analysis method correctly.The purpose of this article was to explore the performance of Hedges Q tests under different situations that were approximated to realistic meta-analysis situations. On the basis of creating the most realistic meta-analysis situations, a series of studies had been carried out to explore the performance of Hedges Q test from between-primary study level and within-primary study level by means of Monte Carlo simulation method.Study 1 examined population effect size distribution effect on the power performance of Hedges Q test. The results of this study indicated that population effect size distribution had impact on the power performance of Hedges Q test, but the impact was not large on the whole.Study 2 focused primary study data distribution effect on the control of type I error rate of Hedges Q test in within-primary study level. The results of this study showed that primary study data distribution played an important role in the control performance of type I error rate of Hedges gtest. In particular, these results included that:(1) with normal distribution of primary study data, population effect size θ had no effect on the control performance of type I error of Hedges Q test. But, with skewed distribution of primary study data,θ played an important role in the control performance of type I error of Hedges Q test; (2) With skewed distribution of primary study data, if the variances of experimental and control groups were homogeneous, then the estimated type I error rate of Hedges Q test at the right side of θ were higher than at the left side of θ on the whole. By contrast, then the estimated type I error rate of Hedges Q test at the right side of θ were lower than at the left side of θ; (3) The more greater degree of the distribution of primary study data deviating from normal distribution, More greater negative effect was leaded to the control of type 1 error rate of Hedges Q test; (4) Increasing the average primary study sample sizes (N) was an effective strategy to improve the control performance of type I error rate of Hedges Q test when the primary study data were distributed normally.Furthermore, study 3 explored the effect of the distribution of primary study data on the statistical power performance of Hedges Q test based on study 1. The results of this study indicated that the statistical power of Hedges Q test could be influenced by the distribution of the primary study data greatly. In particular, these results included:(1) Larger values of variance of population effect size (σθ2) were associated with higher probabilities of detecting heterogeneity; (2) In general, Hedges Q test systematically show power values increasing as average primary study sample sizes (N) increased, but this trend would fluctuate to some extent in some extreme situations; (3) In most simulated situations, the power value of Hedges Q test increased as the number of primary studies increased. But, this trend was not free from situation variables and primary study data distribution; (4) Except for some extreme simulating situations, the power of Hedges Q test were very sensitive to the variance ratio of experiment/control groups (σE2/σC2). Its values would fall rapidly as as σE2/σC2 increased; (5) the average population effect size (μθ) had no impact on the power of Hedges Q test when the primary study data was distributed normally. However, μθ had great effect on the power of Hedges Q test when the distribution of the primary study data was skewed distribution. But, this kind of effect of μθ could fall down to a negligible level if N was large enough; (6) If the distribution of primary study data was skewed distribution, then μθ had great impact on the power of Hedges Q test. (7) In most of meta-analysis situations, the powers of Hedges Q test couldn’t reach the reasonable minimum level of 0.8.Based on the previous studies, study 4 made a standards system for assessment of performance of Hedges Q test.