Individual preparation for formal technical reviews of information system diagrams: A cognitive approach

Improving software quality requires effective, yet efficient, methods for software defect reduction/removal. Formal Technical Review (FTR) detects defects early in the development cycle, thus improving quality while reducing cost. Research indicates the practitioner evaluation phase (PE) of FTR is key to FTR effectiveness. This study examines cognitive processes used by twelve experienced IT practitioners to detect diagrammatic model defects during PE.; Prior research and practice focus on FTR of textual specifications of software systems; little research addresses defect detection in diagrammatic specifications. Diagrammatic models are becoming a dominant form of software specification, and code-generating CASE tools, which make text-based FTR impossible, are increasingly being used.; Cognitive theory indicates spatial configuration differences are associated with differences in understanding, and therefore in diagrammatic review effectiveness of defect identification. Complex information processing tasks like diagrammatic review produce cognitive load that negatively affects performance—theory indicates process-coping strategies (e.g., task planning) can be used to compensate. Theoretical implications are investigated via three research issues: (1) What search patterns are utilized in searching for defects across software diagrams and how do these patterns impact effectiveness? (2) What search patterns are utilized in searching for defects within a diagram and how do these patterns impact effectiveness? (3) What amounts of task planning are utilized during defect detection and how do amount variations impact effectiveness?; Subject performed a verbal protocol while inspecting a defect-seeded software specification. Two raters coded the verbal protocol with high interrater reliability. Subject utterances were visually analyzed for defect search patterns; discovered patterns were related to effectiveness. The task-planning utterances/effectiveness relationship was analyzed statistically.; Search pattern variations across diagrammatic data and process models are associated with effectiveness differences. Patterns for searches within the data model are not discerned. Significant task-planning variations are found; variations are positively associated with effectiveness.; The study supports the cognitive theory thesis that how an individual processes information impacts processing success, and extends the theory to include search-type tasks with visual/diagrammatic stimuli. For MIS and software engineering, the study establishes the explanatory power of cognitive theory for the software-testing context. Specific recommendations for improving practice are advanced.