Collaborative Approaches to Evaluation. Группа авторов

had some form of serious engagement with participatory and collaborative approaches to evaluation. Shulha had completed her doctorate in evaluation at the University of Virginia and joined the Faculty of Education at Queen’s University (Kingston). Over her career, her interests centered on classroom assessment as well as program evaluation and collaboration have been a consistent aspect of her work. In her early years at Queen’s, she worked with her colleague Bob Wilson on an Ontario school district-university collaborative partnership (Shulha, 2000; Shulha & Wilson, 1995) and an international development project in the Colombian education sector. Both projects involved forms of monitoring and evaluation that were very much collaborative (Shulha & Wilson, 1995). Whitmore completed her doctoral research on participatory evaluation at Cornell University in the late 1980s (Whitmore, 1988) and eventually joined the faculty at the School of Social Work at Carleton University (Ottawa). Here she continued her research and practice on participatory and collaborative approaches to evaluation, including a project in the agriculture sector along the US/Mexican border that turned out to be a valuable learning experience (Whitmore, 1998b). Whitmore’s approach to participatory evaluation has been very much sociopolitical, centering on social justice and leveraging change through empowerment (1988, 1991, 2001). These themes are reflected in her later work in feminist evaluation (Whitmore, 2014) and social work (Whitmore & McKee, 2001; Whitmore & Wilson, 2005).

      Cousins finished his Ph.D. on evaluation use in 1988 at the Ontario Institute for Studies in Education (Toronto) and subsequently joined their faculty in a satellite office located in Peterborough, Ontario. The mandate of that unit was educational field development, and Cousins’ principal contributions were in the development of curriculum, leadership, and especially educational evaluation. In those early years, he became directly involved in working with small school districts to help them conduct evaluations of their own programs and interventions. These evaluations provided fodder for the ongoing development and improvement of educational programs and initiatives. Soon, he connected with Dr. Lorna Earl, then head of the evaluation unit at a large urban school district in Toronto. Together, they recognized that they were engaged in very similar evaluation practice: working with educators to help them evaluate and improve their own programs. They began to think of this work as participatory evaluation (Cousins & Earl, 1992) and recruited a range of colleagues in Canada and the US to publish empirical case studies focused on such work (Cousins & Earl, 1995). Arising from these efforts were obvious connections between participatory evaluation and enhanced utilization, as well as organizational learning.

      The common thread among the bourgeoning research programs of Shulha, Whitmore, and Cousins was trained researchers and evaluators working in partnership with members of the community of program practice to coproduce evaluative knowledge. Over the years, we have referred to such work by various names including collaboration (Shulha & Wilson, 1995), collaborative evaluation (Cousins, 2001; Cousins, Donohue, & Bloom, 1996), collaborative inquiry (Cousins & Whitmore, 1998), and participatory evaluation (Cousins & Chouinard, 2012; Cousins & Earl, 1992; Whitmore, 1998b). In essence, they all touched on this foundational notion of joint engagement of evaluators and nonevaluators in planning, implementing, and disseminating evaluation.

      Hind Al Hudib and Nathalie Gilbert later joined the project as research assistants while completing their doctoral programs in evaluation at the University of Ottawa. Both had strong interests in CAE. Al Hudib’s research focuses on the relationship between evaluation policy and evaluation capacity building (ECB) where, as is the case with CAE, the learning function is implicated rather directly (Al Hudib, 2018). Gilbert’s interests lie in the health sector with a focus on innovative approaches to patient engagement, a natural fit for CAE (Gilbert, 2018; Gilbert & Cousins, 2017).

      Collaborative Approaches to Evaluation: What’s in a Name?

      Today, and since about 2014, we have steadfastly used collaborative approaches to evaluation (CAE) as an umbrella term for the genre of evaluation approaches that meet the basic requirement of evaluators working hand-in-hand with nonevaluators to produce evaluative knowledge. In our view, the term is preferable as a masthead for this family of approaches because it is inclusive of a very wide range of approaches that share the common basic prerequisite—evaluators working with nonevaluators in evaluative knowledge production. Box 1 provides an incomplete list of family members, many of these having long histories of application in international development contexts (e.g., most significant change technique, participatory action research, rapid rural appraisal), others being more familiar in Western, particularly North American, contexts (e.g., contribution analysis, collaborative evaluation, utilization-focused evaluation, practical participatory evaluation). Some are associated with cross-cultural perspectives on evaluation (e.g., culturally responsive evaluation), including approaches that embrace indigenous perspectives (e.g., indigenous evaluation framework).

      Box 1: An Incomplete List of Members of the CAE Genre

       Collaborative Evaluation (O’Sullivan, 2004, 2012; Rodriquez-Campos, 2005, 2015)

       Contribution Analysis (Mayne, 2001, 2012)

       Culturally Responsive Evaluation (Hood, Hopson, & Kirkhart, 2015; Wehipeihana, McKegg, Thompson, & Pipi, 2016)

       Deliberative Democratic Evaluation (Ryan & DeStefano, 2000)

       Developmental Evaluation (Patton, 1994, 2011)

       Empowerment Evaluation (Fetterman, 1994; Fetterman & Wandersman, 2005)

       Indigenous Evaluation Framework (LaFrance & Nichols, 2008)

       Most Significant Change Technique (Davies & Dart, 2005; Serrat, 2009)

       Rapid Rural Appraisal (Chambers, 1981)

       Participatory Action Research (Fals-Borda & Anisur-Rahman, 1991; Wadsworth, 1998)

       Participatory Evaluation (Practical, Transformative) (Cousins & Whitmore, 1998; King, 2005; Whitmore, 1998a)

       Principle-focused Evaluation (Patton, 2017)

       Stakeholder-based Evaluation (Bryk, 1983)

       Transformative Research and Evaluation (Mertens, 2009)

       Utilization-Focused Evaluation (Patton, 1978, 2008)

Other contributors have chosen to use different terms to describe the genre or subsets of it. For example, King considers “participatory evaluation [to be] an overarching term for any evaluation approach that involves program staff or participants actively participating in decision-making and other activities related to the planning and implementation of evaluation studies” (2005, p. 291, emphasis in the original). As mentioned above, in our own work, we had often used the terms participatory and collaborative approaches (see Cousins & Chouinard, 2012; Cousins & Whitmore, 1998) in a generic and encompassing way. Yet CAE strikes us as being preferable because all approaches involve collaboration; people jointly working together. In another example, Fetterman and colleagues (2018) use the term stakeholder involvement approaches in an overarching manner. Although they never really define the term, their use of it appears to be limited in scope only to collaborative evaluation, participatory evaluation, and empowerment evaluation approaches which, by no coincidence, correspond with the name of the topical interest group (TIG) of the American Evaluation Association (i.e., TIG-CPE).³ The Fetterman et al. (2018) book is devoid of any recognition of the other approaches listed in Box 1. In addition to limited scope, the term stakeholder involvement in evaluation could easily imply program community members acting merely as sources of data for evaluation, as opposed to being the cocreators of the evaluation knowledge. This potential for confusion is unsatisfactory in our view. The standard dictionary definition of collaboration is “the act of working with another or others on a joint project—often followed by on, with, etc.,”⁴ and this is precisely how we define CAE: the joint engagement of evaluators working with nonevaluators in planning, implementing, and disseminating