| Statement of the problem. This study examined the frequency of selected published educational research studies in which the group served as the statistical unit of analysis, and similarly reviewed use of the groups-nested-within-treatment design. The primary goal was to provide documentation to support speculation that the group was used much less often than many methodologists would consider appropriate, given the nature and setting of most educational research.;Methods and procedures. Content analysis was used to review 1466 articles from four journals (American Educational Research Journal, Educational Research Quarterly, Journal of Educational Psychology, and Journal of Educational Research) from seven years of publication spanning 1971-1989. Collected data included randomization method, sample size, treatment level, use of intact groups, unit of analysis, cell sample size, statistical method, and use of nested design. Data from 925 research studies were summarized across journals and years. Tests of differences between proportions were used to test differences for various categories.;Results. Treatment was administered at the group level in 55% of the studies; however, the group served as the unit of analysis in only 4% of the studies. The Journal of Educational Psychology had significantly fewer studies in which the group was used as the unit of analysis. No significant trend was found for a change in use of the group across the reviewed time span. There were only two studies which included use of a groups-nested-within-treatment design. The latter studies were analyzed in detail.;Conclusions. This study strongly supports speculation that the individual is being used much more frequently than the group as unit of analysis. The frequent use of intact groups and treatment at group level substantiates the premise that the individual is often used inappropriately. Furthermore, the particularly infrequent use of the groups-nested-within-treatment design suggests that a priori pooling is taking place without benefit of preliminary tests to inspect for independence of measures. |
