| Background: Medical errors caused by breakdowns in teamwork and interprofessional communication contribute to many deaths in the United States each year. Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPSRTM) is an evidence-based teamwork system developed to improve communication and teamwork skills among health care professionals to combat these breakdowns. Preparing pre-licensure health profession trainees with efficient interprofessional communication and teamwork skills was proposed as a possible solution. However, the effectiveness of such trainings was inconclusive. Evaluating the effectiveness of training requires reliable and valid measures of teamwork and interprofessional communication skills. The research team was not successful in finding existing validated tools. The "Performance Assessment for Interprofessional Communication and Teamwork" (PACT) tools were developed for this specific purpose.;Objectives: This dissertation consists of two papers. The objective of the first paper is to describe the processes, challenges and lessons learned during the development stage of the PACT tools. The objectives of the second paper are to 1) describe the pilot study using the PACT tools and 2) summarize the validity evidence for the PACT tools.;Methods: A literature search was performed to identify existing tools that evaluate teamwork, team performance, and team communication. Items from these tools were compiled and reviewed by an expert panel. After several iterations, the PACT tools were created for different uses. Pilot data were collected in 2010 from six interprofessional student teams (49 students total) each practicing communication skills in three acute simulated scenarios. The PACT tools were then modified based on the feedback received.;The validity evidence was gathered for the final PACT tools in 2011. Approximately 500 students from a medicine, nursing, pharmacy, and physician assistant (PA) program were recruited to participate in TeamSTEPPSRTM simulation training between May 31and June 3, 2011. Students participated in interprofessional teams to perform in three acute simulated scenarios each set addressing one of three clinical domains: 1) asthma, congestive heart failure (CHF), and supraventricular tachycardia (SVT), 2) three pediatric scenarios, or 3) three obstetric scenarios. The PACT tools were used to assess teamwork and team communication skills in real-time and retrospectively through analysis of the video recordings of the asthma and SVT scenarios. The validity evidence for PACT tools was presented based on Messick's (1995) framework that includes six distinguishable aspects: 1) content, 2) substantive, 3) structural, 4) generalizability, 5) external, and 6) consequential.;Results: The total numbers of records for PACT-Novice, PACT-Expert and PACT-Video were 934, 85, and 120, respectively. The ICC of PACT-Novice was .85 (n=904, F=6.46, p < .001). The ICC of PACT-Expert was .76 (n=85, F=4.09, p < .001). The ICC of PACT-Video was .90 (n=112, F=9.63, p < .001). One-Way ANOVA and exploratory factor analyses were also performed to provide preliminary validity evidence for the PACT tools.;Conclusions: The PACT tools are reasonably consistent, reliable and exhibit some evidence they can be validly used for debriefing and evaluating team performances with pre-licensure health professional students. Further studies with larger sample sizes, standardized training experiences, and longitudinal follow-ups are needed to build the validity evidence for the PACT tools. |
