Integrating program process and outcome: A program evaluation using the logic model framework

Four county-wide forums in New Jersey, designed to encourage school and community professionals to use quality data in their prevention program planning, were evaluated. The program evaluation was based on the logic model framework and included a process evaluation component and an outcome evaluation component. The process evaluation component examined whether the center staff members felt prepared to facilitate the forums, whether the forums were implemented as planned, whether the objectives of the forums were met, and how participants perceived the forums. The outcome evaluation component examined whether participants applied the information they learned from the forums to their prevention planning work several months after the events ended. Quantitative and qualitative data were collected from participants using various survey instruments, and responses were analyzed using descriptive statistics and thematic analysis. Results indicated that facilitators felt well prepared to lead the activities of the forum, that the forums were implemented as planned, and that the objectives of the forums were met. Furthermore, the majority of participants had positive impressions of the forums, though the overall responses from one county were not as positive as the responses from the other counties. In addition, the outcome evaluation indicated that the majority of participants either had already applied the information they learned from the forums, or had intentions of doing so in the future. A limitation of the evaluation is that there was a low response rate to the follow-up survey that provided the outcome data. As a result, it is not possible to make a broad generalization that the successful operation of the forums encouraged most attendees to apply the information they learned. Collection of more extensive outcome data would provide more information as to whether most attendees applied the forum information to their prevention work. The integration of the process and outcome evaluation data is discussed, and an evaluation of the program evaluation is made. Implications for school psychologists and recommendations for program designers are provided.