Response sets, item format, and thinking style: Implications for questionnaire design

A major threat to survey reliability is response set, a pattern of response to items which is thought to be unrelated to item content. This study examined several possible contributors to response set--item format, item controversy, and thinking style. Degree of certainty, gender, and major area of study were also explored.;The most significant contributors to response set were item format and controversy; minor contributors were gender and major area of study. Thinking style was found insignificant. Degree of certainty was moderately associated with extreme responding style across all formats and content areas. Large residuals from the ANOVAs indicated other important contributors are yet to be found.;Other findings included the unique contribution of fit statistics to response set information. Also, response set emerged differently on the semantic differential than on the other two formats, and the semantic differential was found most reliable of the three formats.;This study concluded that: (1) response set cannot be considered independent of content; (2) variety in format is important in reducing response set; (3) the degree of certainty variable is essential in survey design; and (4) the semantic differential offers reliable application for surveying attitudes.;A three-way repeated measures ANOVA comprised the main analysis. Volunteer undergraduate and graduate university students (N = 569) from several major areas of study were targeted for a wide distribution of thinking styles. Subjects were administered a survey packet including the Gregorc Style Delineator (Gregorc, 1984), 12 short attitude surveys in semantic differential, rating scale, and magnitude estimation formats, and a demographics page. Attitude topics included two controversial topics and two non-controversial topics. Hui and Triandis' response set model (1985) was applied, consisting of acquiescence/directional bias, response range, and extreme responding style. In addition, person fit statistics, derived from the Rasch model, were analyzed for response pattern.;Recommendations included: (1) further investigation of thinking style using other thinking style and response set models; (2) wider use of the semantic differential format; and (3) ongoing exploration of other possible contributors to response set. Possibilities are motivation, temperament, and/or personality.