The development of an expert system to evaluate the individualized education program components of student records

In 1975, Congress passed the Education of All Handicapped Children Act, mandating that every student receiving special education have an individualized education program (IEP). This document is a report on a research and development project designed to develop an expert system, IEP Advisor, that provides users with evaluative feedback on IEPs contained in student records.;The project included formative and summative evaluations of IEP Advisor. Formative evaluations involved an expert panel who guided the development of the system through three phases: product definition, product design, and prototype development. Results from the formative evaluations provided evidence of the content validity of the system.;During summative evaluations of IEP Advisor, the researcher studied the reliability and validity of system reports generated through the use of IEP Advisor. To estimate the reliability of information in reports, the researcher compared conclusions generated by IEP Advisor when different users examined the same student record. Results indicated moderate and statistically significant levels of interuser agreement.;To estimate the validity of IEP Advisor, the researcher compared reports generated through the use of IEP Advisor with reports generated by local personnel recognized for their skill in IEP development. The users of IEP Advisor were, by their own admission, naive about how to evaluate IEPs. Both groups based their reports on the same set of simulated student records. The reports from both groups were given to an expert panel that was blind to the source of the reports. Expert panel members were asked to rank order the reports from best to worst.;The results indicated wide variation in the opinions of expert panel members on which reports contained the best information. Despite the variation, the analysis of phi coefficients comparing the source of the reports (IEP Advisor user vs. expert) to their ranking (high vs. low) revealed that reports generated by IEP Advisor were consistently ranked as good as or better than reports produced by locally recognized experts.;Recommendations for future research included: simplifying the judgments required by users, updating the knowledge base, dividing the expert system into several smaller systems, and additional studies of system validity.