Investigations of distribution, specification, and approximation assumptions in IT spending benchmarking

IT managerial control ratios (such as IT Budget/Revenues, IT Budget/Operating Expense, IT Budget/Employees) are used extensively by both practitioners and researchers (Bender 1986; Harris and Katz 1991; Harris and Katz 1991; Weill 1992; Mahmood and Mann 1993; Hitt and Brynjolfsson 1996; Mitra and Chayam 1996; Bharadwaj, Bharadwaj et al. 1999; Lee and Bose 2002). Practitioners use these ratios in IT cost benchmarking---the practice of comparing IT spending against industry averages or companies known for their "best in class" IS organizations (Doll, Deng et al. 2003). Researchers use these ratios as measures of organizational IT spending in studies investigating the link between organizational IT spending and measures of organizational performance, such as profitability and cost efficiency. Thus, IT managerial control ratios are often at the heart of important capital allocation decisions (Kobelsky, Richardson et al. 2008) as well as studies which form important stakeholders' perceptions regarding the business value of IT and value of Information Systems (IS) as a discipline (Kohli and Grover 2008).;Unfortunately, these two important uses of IT managerial control ratios are often plagued by poor understanding of properties of these ratios. Accounting literature on ratio analysis (Lev and Sunder 1979; Whittington 1980; Barnes 1982) has argued that validity of benchmarking and studies using these ratios is contingent on the extent to which these ratios satisfy the following three assumptions: (1) Cross-sectional probability distribution of IT managerial control ratios within an industry is normal; (2) The relationship between numerator and denominator used in IT managerial control ratios is strictly proportional; (3) Means and medians can be used as an adequate approximation of the relationship between numerator and denominator used in IT managerial control ratios. Violation of these three assumptions can introduce significant bias into benchmarking and research, leading to poor budgeting decisions and incorrect inferences from research results. Thus, a theoretical argument substantiated with empirical evidence is made concerning the degree to which these ratios comply with the three assumptions listed above.;Drawing on the strategy literature and basic microeconomics theory, this study makes a number of theoretical and analytical arguments as to why IT managerial control ratios are not likely to adhere to the three assumptions. These theoretical arguments are supported by a rigorous, multi-method empirical investigation using a large cross-sectional sample of North American companies compiled by a leading IT research firm.;This study does not suggest that both researchers and practitioner should abandon using these ratios in research and benchmarking. The reality is that researchers are often constrained by data availability, while practitioners usually do not have time for a more rigorous analysis of IT spending. With these constraints in place, these ratios may still be imperfect, yet useful instruments, since having some data to base an inference on may be better than having no data at all. At the same time, in the light of these findings, both practitioners and researchers are advised to interpret results of benchmarking and academic studies using IT managerial in conjunction with the ratios' properties uncovered by this study. Important capital allocation decisions as well as studies which form important stakeholders' perceptions in relation to the value of IT should not be carried out without understanding the properties of IT managerial control ratios uncovered by this study.