Practical tools are the strategies, interpersonal skills, practices, and moves evaluators use in their work as they strive to carry out an evaluation. Arguably, these are one of the hardest tools to see and teach across expert knowledge occupations. This is why, for example, doctors spend time developing bedside manners, psychotherapists spend time unpacking video recordings of themselves with patients, and teachers spend time critically reflecting on their in-the-moment instructional decisions. In all of these instances, you have to be in the right place at the right time to catch a glimpse of a professional using a particular practical tool, and at the same time, you have to have a systematic process in place for unpacking use of these tools. A strength of the chapters included in this volume is that they make visible some of the practical tools evaluation practitioners and educators use. For example, how evaluators go about building relationships and with whom, the importance of trust, and the communication strategies evaluators use that are aligned with CAE principles. Moreover, in Chapters 7 and 9, we see how novice and emergent evaluators engage with learning to identify and use these practical tools, as well as the practical tools evaluation educators use to foster learning.
While each of these tools is important, in practice, they are and must be intertwined. Learning about a theory or learning to justify a technical method is not the same thing as learning to use that theory in practice or to use a particular design in an evaluation, just as learning about the technical aspects of writing and learning about narratology1 is not learning to write. The latter requires that an author actually engage in writing, understanding, for example, how to use technical rules, when it makes sense to break rules, how potential readers understand or interpret what they are reading, how they react to prose and whether that is what you intended, and so forth. Because evaluation is situated in the social, learning how to evaluate requires all three tools. It is how one learns to use our evaluation theories, the benefits and limits of particular evaluation approaches, how to generate evidence that will be perceived as credible to a wide body of stakeholders, which dilemmas to anticipate and how to think through addressing unanticipated issues, and so forth.
1 The Oxford English Dictionary defines narratology as “the branch of knowledge or criticism that deals with the structure and function of narrative and its themes, conventions, and symbols” (https://en.oxforddictionaries.com/definition/narratology).
This volume has given us a window into our complex work, which will be useful for novice and seasoned evaluators learning how to use the conceptual, technical, and practical tools of our profession. It will also be useful for evaluation educators who are working to facilitate the process of learning to practice. Evaluation researchers who are interested in describing and understanding practice will also find much use in this volume.
Bianca Montrosse-Moorhead, Marvin C. Alkin, and Christina A. Christie Volume Editors
References
American Evaluation Association. (2017, August 28). AEA evaluator competencies. Retrieved from https://www.eval.org/p/do/sd/sid=8317&fid=2290&req=direct
Mathison, S. (2005). Preface. In S. Mathison (Ed.), Encyclopedia of evaluation (pp. xxxiii–xxxv). Thousand Oaks, CA: Sage.
Schwandt, T. A. (2015). Evaluation foundations revisited: Cultivating a life of the mind for practice. Stanford, CA: Stanford University Press.
Scriven, M. (1991). Evaluation thesaurus. Newbury Park, CA: Sage.
United Nations Evaluation Group. (2016). UNEG evaluation competency framework. Retrieved from http://www.unevaluation.org/2016-Evaluation-Competency-Framework
Preface
This edited volume arises from the initial work of the Collaborative Opportunities to Value Evaluation (COVE) research group2 on the development and validation of a set of principles to guide practice in collaborative approaches to evaluation (CAE)—evaluations that implicate evaluators working in partnership with program stakeholders to produce evaluative knowledge. Such approaches are on the rise. Several types (e.g., rapid rural appraisal, participatory action research) have been practiced in international development contexts for decades. But many others have been developed more recently (e.g., contribution analysis, developmental evaluation) and provide evaluation practitioners and commissioners with a range of options that depart quite significantly from traditional mainstream approaches to evaluation. At least partly in response to this growing family, the COVE research group committed to developing and validating a set of evidence-based principles to guide CAE practice. These principles were first published in 2016 (Shulha, Whitmore, Cousins, Gilbert, & Al Hudib, 2016) and launched globally in January 2017 in English, French, and Spanish.
2 J. Bradley Cousins, Elizabeth Whitmore, Lyn M. Shulha, Hind Al Hudib, Nathalie Gilbert
The COVE research team considers the CAE principles to be an initial set and advocates for their ongoing review and revision. This volume is a contribution toward that end. The book provides an up-close look at the application and analysis of the CAE principles in a wide range of contexts. Readers interested in learning about how the principles can be applied, the benefits of doing so, and the challenges that are prone to emerge will find great value in this volume. Following a comprehensive treatment of the rationale for and development and validation of the principles, eight empirical field studies are reported. Each field study offers particular insights into collaborative practice but also provides commentary on the utility and value of the principles themselves. The final integration chapter looks across the field studies to summarize what has been learned and to foreshadow potential revisions to the principles in the not-too-distant future.
The volume will be of high interest to a wide swath of evaluation community members. In many applied fields of endeavor, following the wisdom of the late MIT Professor Donald Schön, practitioners seek to develop their reflective practice.3 Not all evaluation practitioners will engage with CAE, but for those for whom