This volume attempts to offer an up-to-date and broad discussion of economic evaluation of education programs. It is a significantly revised and expanded version of two earlier editions addressing the same topics by Levin (1975, 1983) and Levin and McEwan (2001). In some respects, the basics of the method have not changed since the first edition in 1983. The ingredients method is the same, as are the metrics that are intended to represent efficiency. However, there has been a broad expansion of applications for CE and BC. In addition, new methodological developments—in effectiveness and benefits measurement, approaches to cost data collection and analysis, and sensitivity testing—all represent rich topics for improving the use of the tools. These topics are included in this revision. Also, the evidence base on results has grown, and we offer a general review of this evidence. Most importantly, there is now a much greater recognition that economic evaluations can play an important role in both education research and policy formation. New material in this book reinforces the theme that economic evaluations are valuable methods for social scientists to apply.
The remainder of the book will be devoted to a presentation and discussion of the use of economic evaluations in education as well as a description of the principles and techniques for developing such analyses. The next chapter will discuss the decision context, audience, and particular issues that are pertinent to the choice of analysis, its implementation, and its presentation. Chapters 3 through 6 will address the nature of costs and their identification, measurement, and distribution. The reader should be aware that the discussion in these chapters applies equally well to all modes of cost analysis. That is, the differences among the modes are primarily on the outcomes side rather than the cost side. Chapters 7 and 8 focus on how to measure effects and how to perform CE analysis. In a parallel form, Chapters 9 and 10 focus on how to specify benefits and how to perform BC analysis. An essential element of economic evaluation is sensitivity testing and dealing with uncertainty. This topic is covered in Chapter 11. As a review, Chapter 12 provides a checklist for appraising the quality of economic evaluations. Finally, Chapter 13 considers how to link the evidence from economic evaluations to policy and decisionmaking. Each chapter includes exercises and discussion questions. Appendix A gives sample answers to even-numbered exercises at the end of each chapter.
Discussion Questions
1 Typically, educational evaluations look at the effects of alternative interventions on student outcomes without considering the cost consequences. Under what circumstances would adopting the “most effective” alternative actually increase overall costs to the school district for any specific educational result relative to choosing a “less effective” alternative?
2 There have been many studies of the relation between enrollment levels in schools and school districts and the cost per student. These studies purport to show how cost varies with school size, and they attempt to determine the enrollment ranges in which costs are lowest. Do these studies meet the criteria for CE analysis?
3 Imagine a state introduces a new tax on soft drinks, and the tax revenues are earmarked for spending on education. Which level of education—preschool, elementary school, secondary school, college, or vocational training—should receive priority for public funding from these additional tax revenues?
4 What are the fundamental differences between CE and BC analyses? Provide examples of educational interventions where CE analysis is preferred. Provide examples of educational interventions where BC analysis is preferred.
2 Establishing an Analytic Framework
Objectives
1 Identify the evaluation problem.
2 Determine the audience and perspective for the evaluation.
3 Select the appropriate type of economic analysis and research design in relation to the theory of change.
4 Establish the need for chosen economic evaluation.
Before beginning an economic evaluation, it is important to establish the analytical framework that will be utilized. This framework consists of identifying the nature of the problem and the theory of change, clarifying the specific alternatives that should be considered in the analysis, establishing the identity of the audiences—and the perspective—for the analysis, and selecting the type of cost analysis to use. These tasks are all preparatory to the actual research inquiry, data collection, and analysis, which are addressed in subsequent chapters. At the initial stage, it is important to consider the extent and scope of the inquiry to ensure that the economic evaluation is appropriate and justifiable.
In this chapter, we discuss these issues in turn. Although they might seem outside the scope of a direct economic evaluation, establishing the appropriate structure for the research inquiry is critical. It is going to be very difficult to rederive new estimates of costs and benefits if the research questions change. Importantly, the research findings will depend on how the research is framed. Hence, we describe key issues in establishing the analytic framework for economic evaluation.
2.1. Identifying the Problem
One of the critical, but sometimes neglected, areas of evaluation design is proper identification of the problem. By proper identification, we mean that the nature and scope of the problem should be posed to justify the intervention and to determine which alternatives for reform appropriately address the issue. To accomplish this, aspects of the problem must be documented to understand who is affected and how (for an overview of evaluation methods and practice, see Rossi, Lipsey, & Freeman, 2004).
For example, it is widely accepted as important that children acquire reading skills progressively through school based on age or grade level. If it is found that children are not reading at the appropriate level, it is important to know how many children are behind in reading, at the age or grade level, how severe the gap is between the skills children have and the skills needed to be on target, and where the children are located who need the most assistance. Based on this information, one should ascertain potential reasons to explain the deficiency. The origins may be community or family based and related to insufficient skills upon school entry, reading deficiencies among the population, or lack of books for children outside of school. Another possibility is that the teaching strategies or materials may not be conducive to the variety of skill in the classroom consistent with up-to-date research on pedagogy for teaching reading. A third possibility could be related to attendance. A fourth could be that the children with lagging reading skills are English-language e-learners. Each of these causes would require a different intervention to sufficiently impact the outcome of interest.
We can also review another example where the problem is one at the system level rather than with student skill development. As noted in Chapter 1, the question that is often posed by school districts facing financial exigencies because of declining enrollments is this: Which school or schools do we close? The real problem that must be faced, however, is how to address a shrinkage in resources that does the least damage to the education system. The alternatives to consider include the possibilities of school closure, but they also include the options of reducing personnel, cutting specific offerings, increasing class size, leasing excess space in existing schools, and taking a variety of other potential routes to cutting