1.4. Summary of Approaches to Economic Evaluation
Having defined and illustrated the approaches to economic evaluation, we provide a summary that allows them to be compared. As shown in Table 1.1, the approaches are similar but have key distinctions. Each has strengths and weaknesses. Although BC analysis is the most comprehensive, it requires the most analytical effort and may not be required to respond to a particular decisionmaker’s needs. After reading the subsequent chapters, you may find it helpful to return to Table 1.1 in order to solidify your understanding.
Most economic evaluations use either CE or BC analysis (CF is often incorporated as a preliminary endeavor, and CU analysis is a derivative of CE analysis). The bulk of our attention is therefore directed to these two methods.
Although they seem similar, CE and BC analyses are different in one very important way. We distinguish between the two in simple terms. The CE analyst would ask, What is the lowest cost alternative to get outcome X? The BC analyst would ask, Should we invest in program Y that produces outcome X? So, one might investigate with CE analysis whether the Wilson Reading System is more cost-effective than Corrective Reading. Instead, one might investigate with BC analysis if investing in reading is more valuable than investing in math (remembering that doing one implies less of the other because of resource constraints). Formally, both questions can be described as efficiency questions: CE refers to productive efficiency, and BC refers to allocative efficiency. However, to avoid confusion we describe results from CE analyses based on whether they are cost-effective relative to alternatives; BC analyses have tended to use the term efficiency, so we use that term here.
Table 1.1
The consistency that carries throughout economic evaluation is the measurement and reporting of the intervention’s cost. This textbook describes and discusses the application of the ingredients method. The ingredients method utilizes the economic principle of opportunity cost to include all resources that were utilized during implementation to generate an outcome. The approach is transparent and easy to interpret by following common procedures in cost accounting. The resulting research on costs provides the audience with information about the resources uses, including their qualities and quantities, as well as the price value of each resource. The resulting estimate of total social cost and average cost per student are consistent for any of the economic evaluations discussed here.
1.5. Economic Evaluations and Policymaking
We suspect that some education professionals and researchers may question the use of economic evaluation methods. With CE, they may argue that an education intervention with complex and multifaceted outcomes cannot be evaluated using a single measure. With BC, researchers may reject the idea of turning education processes and student growth into dollars based upon monetary values that can then be moved around to generate efficiency gains.
We believe the case for economic evaluations is strong. Fundamentally, there is the essential idea of scarcity and opportunity cost. Resources are required to implement preschool programs; those same resources cannot then be used to provide youth support programs. By using economic resources to implement preschool programs, we are therefore implicitly saying that these have a priority over using similar resources for youth support programs. Using economic evaluation gives this decision a scientific basis rather than relying solely on preferences without consideration of other alternatives.
Moreover, the issue is not whether every educational program or policy should be evaluated closely using economic concepts. Rather, the issue is whether too much or too little economic evaluation is being performed (Belfield, 2015). Based on a very general review of education research, we believe that more educational evaluation should be performed. Although the application of these methods is expanding in education, large subfields of education include little or no CE or BC analysis—school choice, teacher value-added modeling, and special education are notable examples. Also, despite an exhaustive focus on test score accountability, few studies look at how to increase test scores for the lowest cost or even how valuable it would be to increase test scores. As already noted, it would have been very helpful to know the costs of class size reduction (or vouchers or small schools) before embarking on research to determine the effects on achievement.
More generally, economic evaluations can function as a framework for many of the factors that might influence a decision about education provision. Reform of the education system or the introduction of new programs raises a whole array of policy issues. Not all of these are about efficiency. For instance, more school choice may help students who seek alternative educational opportunities, but it may hinder those who are left behind in failing schools. The efficiency of school choice should be traded off against the potential increase in inequities for students in low-quality schools. Teacher accountability mechanisms may raise the quality of instruction but may reduce teacher job satisfaction and undermine a culture of professionalism. The gain in instructional quality may be offset by the extra risk to teachers that they will lose their jobs. Investing in science programs may increase the earnings of science graduates at the expense of funding for graduates in the humanities; similarly, investing in preschoolers means fewer resources for disadvantaged youth. All policies entail trade-offs, and most educational changes have political and social repercussions. However, by performing an economic evaluation we can identify some of these issues and distinguish between efficiency and equity. For instance, if we know that preschool is actually an investment that saves money—with fewer students later retained in grade such that the savings outweigh or offset the costs of preschool—this means more resources are available for disadvantaged youth (and perhaps they lead to a reduction in disadvantaged youth). Similarly, if accountability systems better demonstrate high-quality instruction, then parents might be more willing to pay for quality; teacher salaries might increase to compensate for the riskier work environment. Economic evaluation can clarify some issues and frame other key issues, allowing the decisionmaker to make better informed judgments and decisions.
It is important to clarify the purpose of economic evaluation: It is to help decisionmakers make better decisions. If it does not help with decisionmaking and thereby improve the policymaking process, it is not necessary (Posner, 2000). Critically, however, there is a logical step between providing an economic evaluation and making a decision. This step is clearly explained in U.S. government Executive Order 13563 issued in January 2011 (a supplement to Executive Order 12866 issued in September 1993). This order establishes that U.S. government agencies must “propose or adopt a regulation only upon a reasoned determination that its benefits justify its costs (recognizing that some benefits and costs are difficult to quantify).” BC analysis will yield an economic metric that has implications for investment of public funds. But that does not mean that these funds must be invested. As Executive Order 13563 states, there may be some benefits and costs that were not quantified; these may be sufficiently important to override the economic imperative. Moreover, the decisionmaker must still make a “reasoned determination”—that is, explicitly justify the decision. Economic evaluations should help make that decision, but they do not determine it. Just because an intervention does not pass a formal BC test does not mean it should not be supported. If society believes that open-access college is an important commitment that a community should make to its future, then this belief need not be automatically negated by an economic evaluation that finds that the costs exceed the measured benefits. Throughout this book, we encourage the reader to examine how economic evaluations help improve education policy cognizant of the difference between an economic result and a reasoned determination.
1.6.