| The need to manage the creation and flow of intangible assets like ideas, innovations, best practices, corporate policies, and other aspects of intellectual capital has caused many organizations to invest in a particular type of information system generically referred to as a “knowledge repository” or “knowledge base” (KB). Although organizations continue to make substantial investments in these systems, research indicates that their approach to evaluating the success of those systems is typically one-dimensional and overly dependent upon traditional accounting measures (e.g. cost-benefit, productivity, ROI) as well as intuition. Failing to address the complex organizational contexts in which these systems operate leaves KB managers with an incomplete and unreliable diagnostic of system performance. Using an action inquiry approach, this case study examined the evaluation practices of an academic organization's technical support KB. Over several months, the author worked collaboratively with the KB manager to investigate existing evaluation practices, design and execute improvements based on theoretical and practical influences, and understand management's thinking throughout the process. Results indicate: (1) strong support for an integrated approach to information system evaluation that uses both quantitative and qualitative evaluation methods, (2) support for an evaluation strategy that addresses the socio-organizational objectives of the information system in addition to the technical objectives, (3) discoverable factors that led to the KB manager's willingness to change various aspects of the existing evaluation program and interest in doing so, and (4) possible modifications to existing frameworks that prescribe methods for evaluating similar systems. In addition to the theoretical and practical implications of this study, directions for future research are discussed. |
