Comparing the accuracy of performing digital and paper checklists using a feedback package during normal workload conditions in simulated flight

This study examined whether pilots completed airplane digital or paper checklists more accurately when they received post-flight graphic and verbal feedback. Participants were 6 college student pilots with instrument rating. The task consisted of flying flight patterns using a Frasca 241 Flight Training Device which emulates a Cirrus SR20 aircraft. The main dependent variable was the number of checklist items completed correctly per flight. An alternating treatment, multiple baseline design across pairs with reversal, was used. During baseline, the average percent of correctly completed items per flight varied considerably across participants, ranging from 13% to 57% for traditional paper checklists and ranging from 11% to 67% for digital checklists. Checklist performance increased to an average of 90% for paper checklist and an average of 89% for digital checklists after participants were given feedback and praise, and continued to improve to an average of nearly 100% for paper checklists and an average of 99% for digital checklists after the feedback and praise were removed. A slight decrement in performance was observed during a post-experiment probe between 60--90 days. Visual inspection and statistical analysis of the data suggest that paper checklist accuracy does not differ significantly from digital checklist accuracy. The results suggest that graphic feedback and praise can be used to increase the extent to which pilots use both digital and paper checklists accurately during normal workload conditions.