| As the biotechnology industry grows rapidly, it requires increasing numbers of biotechnicians with problem-solving skills and technical knowledge, yet a college-level, work-related and completely validated assessment measuring biotechnology problem solving skills does not exist in test banks or the problem-based learning literature. The purpose of this study was to develop and validate two parallel forms of an instrument that measures the biotechnology problem-solving skills of students enrolled in community college biotechnology programs. The Biotechnology Problem-Solving Skills Assessment is a 17-item, written, short-answer test containing work-related biotechnology problems in five short problem analysis cases and one integrated performance memo. The assessment validation process answered research questions about the reliability of scores on the assessment, its usefulness and authenticity, and the extent to which scores on the assessment support inferences about students' biotechnology problem-solving skills on the job. The assessment evolved through three testing phases: preliminary, pilot, and field testing. In each round of testing the assessment was administered, and students and experts were interviewed. Additionally during the field test with 115 students and 11 experts, three raters scored 10 student assessments, and two expert biotechnicians rated 10 student assessments. The assessment scores were reliable (alpha = 0.81 for form A and 0.69 for form B). The assessment was viewed as authentic and useful for giving students feedback, as an instructional tool, and as a possible interviewing tool. Student scores on the assessment correlated positively with a proxy measure of on the job problem-solving performance, employer ratings of student assessment answers (ρ = 0.746, p = 0.013). Experts validated the biotechnology and problem-solving content on the assessment. Intra- and inter-rater reliabilities were reasonable (intrarater, ρ = 0.94, 0.91, and 0.73; interrater, ρ = 0.67 for form A and ρ = 0.54 for form B). Subtotal and total scores on the two forms of the assessment correlated postitively, significantly and moderately. The assessment distinguished between experts and students (Mann-Whitney U test, p = 0.004 for form A and p < 0.001 for form B). |
