Web-based experimental research in psychology and law: Methodological variables that may affect dropout rates, sample characteristics, and verdicts

The development of the Internet and the Web has allowed all researchers to employ a new method of collecting data, but questions remain about methodological threats to the validity of research findings. There are many variables associated with what is termed "Web-based research," each of which might influence the results of Web-based studies. Four studies examined the effects of different sample types, methods of soliciting participants, financial incentives, manners of collecting personal or demographic information, designs of tables on Web pages, and methods of obtaining informed consent on dropout rates, sample characteristics, and the ultimate dependent variable in these studies---verdicts. Each variable showed some effect on dropout rates but two patterns emerged. Some variables had consistent effects, as certain conditions led to increased dropout at several points in the study. Other variables had effects on dropout at one point in the study but then effects in the opposite direction appeared at subsequent stages of the study. Sample type and table design had consistent effects on sample characteristics, reflected in mean differences and differences in variance on attitude measures. Sample type was related to verdict in one study, in which there were also several interactions. Other methodological variables generally did not affect results.