An evaluation of information technology systems at Pepperdine University

Information technology programs in higher education are passing from tactical to strategic and infrastructure roles. As this happens, they become harder to evaluate in traditional financial terms because of their broad impact on the institution. As collections of tactical systems coalesce into organizational programs, their original goals change, invalidating any formative evaluations that may have existed. The mission of higher education is being increasingly carried out via information technology programs, as evidenced by investment and strategic trends in the literature. It is important to find ways to evaluate these evolving and vital programs.;The purpose of this study was to evaluate a snapshot of the program of interdependent systems in the key areas of communications, academic services and course management at Pepperdine University. To find a concrete basis of evaluation, the study used system logs to describe the current state of the program: its uses, users and features. Logs were collected and analyzed from a 6-week period in the Spring term of 2004. The selections from 242 million lines of logs representing systems in these key areas were analyzed after uploading of relevant data into databases and other analysis software.;The evaluation snapshot of this program showed certain academic programs tended to use certain systems and feature sets to the exclusion of others. Other features and systems were under utilized or overlapped with other systems. Use patterns of systems differed significantly by academic program as did faculty/staff use.;Recommendations from the study include establishment of ongoing evaluation for these systems. The data collected for this study will provide a baseline and the addition of other sources of data will allow the effectiveness of these systems to be tracked over time and the appropriate recommendations for their further evolution to be based on concrete data. A set of technology guidelines and evaluation procedures for evaluating other systems of this type in higher education is provided as one of the outcomes.