Despite the existence of some marginal and even negative results, the range of conditions and contexts under which studies have shown that gains can be achieved must indicate that the principles that underlie achievement of substantial improvements in learning are robust. Significant gains can be achieved by many different routes, and initiatives here are not likely to fail through neglect of delicate and subtle features. (Black & Wiliam, 1998a, pp. 61–62)
While we did not conduct a formal meta-analysis, in a subsequent publication (Black & Wiliam, 1998b), we did try to provide some indication for practitioners and policymakers of the likely potential benefits of formative assessment. We suggested that the effective use of formative assessment would increase achievement by between 0.4 and 0.7 standard deviations, which would be equivalent to a 50 to 70 percent increase in the rate of student learning (see Wiliam, 2006, for details).
While we were confident that the research evidence that we had compiled made a compelling case for making classroom formative assessment a priority, we were not sure that these ideas could be implemented in real classrooms, especially where students were regularly subjected to external standardized tests, and where teachers were held accountable for their students’ achievement.
We therefore recruited twenty-four (later expanded to thirty-six) secondary school mathematics and science teachers in six schools in two districts in England to help us explore what classroom formative assessment might look like in classrooms (Black, Harrison, Lee, Marshall, & Wiliam, 2003). The work with teachers had two main components. The first was a series of eight workshops over an eighteen-month period, which introduced teachers to the research base underlying how assessment can support learning, allowed them the opportunity to develop their own plans for implementing formative assessment practices, and, at later meetings, provided them the time to discuss with colleagues the changes they had attempted to make in their practice. Most of the teachers’ plans contained reference to two or three important areas in their teaching in which they were seeking to increase their use of formative assessment, generally followed by details of techniques that they could use to make this happen. The second component was a series of visits to the teachers’ classrooms, so that the researchers could observe teachers implementing some of the ideas they had discussed in the workshops and could discuss how their ideas could be put into practice more effectively.
Because each teacher had made his or her own decisions about what aspect of formative assessment to emphasize and which classes to try it with, it was impossible to use a traditional experimental design to evaluate the effects of our intervention. Therefore, we designed a poly-experiment. For each class with which a teacher was trying out formative assessment techniques, we looked for the most similar comparison class and set up a mini-experiment in which we compared the test scores of the class that was using formative assessment with the test scores of the comparison class. In some cases, this was a parallel class taught by the same teacher; in some cases, it was a similar class to one the teacher had taught in previous years; and in other cases, it was a similar class taught by a different teacher. This experimental design is not as good as a random-allocation trial, because the teachers participating in the experiment might have been better teachers to begin with, and so the results need to be interpreted with some caution. Nevertheless, in this study, using scores on externally scored standardized tests, the students with which the teachers had used formative assessment techniques made twice as much progress over the year (Wiliam, Lee, Harrison, & Blade, 2004).
Definitions of Formative Assessment
As the evidence that formative assessment can have a significant impact on student learning accumulates, many researchers have proposed a variety of definitions of formative assessment. In our original review, Paul Black and I (1998a) define formative assessment “as encompassing all those activities undertaken by teachers, and/or by their students, which provide information to be used as feedback to modify the teaching and learning activities in which they are engaged” (p. 7). Writing around the same time, Bronwen Cowie and Beverley Bell (1999) qualify this slightly by requiring that teachers and students act upon the information from the assessment while learning takes place. They define formative assessment as “the process used by teachers and students to recognise and respond to student learning in order to enhance that learning, during the learning [emphasis added]” (Cowie & Bell, 1999, p. 32, emphasis added). Others also emphasize the need for action during instruction and define formative assessment as “assessment carried out during the instructional process for the purpose of improving teaching or learning” (Shepard et al., 2005, p. 275). Reviewing practice across eight countries, OECD defines formative assessment as “frequent, interactive assessments of students’ progress and understanding to identify learning needs and adjust teaching appropriately” (as cited in Looney, 2005, p. 21).
What is notable about these definitions is that, however implicitly, they regard formative assessment as a process. Others tend to regard formative assessment as a tool. For example, Stuart Kahl (2005), cofounder of Measured Progress, defines formative assessment as “a tool that teachers use to measure student grasp of specific topics and skills they are teaching. It’s a ‘midstream’ tool to identify specific student misconceptions and mistakes while the material is being taught” (p. 11). Indeed, it appears educators more often use formative assessment to refer to a particular kind of assessment instrument than a process to improve instruction.
The difficulty with trying to make the term formative assessment apply to a thing (the assessment itself) is that it just does not work. Consider an advanced placement (AP) calculus teacher who is getting her students ready to take their examination. Like many teachers, she has her students take a practice examination under formal test conditions. Most teachers would then collect the papers, score them, write comments for the students, and return the papers to the students so that they could see where they went wrong. However, this calculus teacher does something slightly different. She collects the papers at the end of the examination, but she does not score them. Instead, during her next period with the class, each group of four students receives its unscored papers and one blank examination paper, and has to compile the best composite examination paper response that it can. Within each group, the students review their responses, comparing their answers to each question and discussing what the best answer would be. Toward the end of the period, the teacher reviews the activity with the whole class, asking each group to share its agreed-on answers with the rest of the class.
The AP calculus assessment that the teacher uses was designed entirely for summative purposes. The College Board designs AP exams to confer college-level credit so that students passing the exams at a suitable level are exempt from introductory courses in college. However, this teacher uses the assessment instrument forma-tively—what Black and I have called “formative use of summative tests” (Black et al., 2003, p. 53). Describing an assessment as formative is, in fact, what philosopher Gilbert Ryle (1949) calls a category error: the error of ascribing to something a property that it cannot have, like describing a rock as happy. Because the teacher can use the same assessment both formatively and summatively, the terms formative and summative make much more sense as descriptions of the function that assessment data serve, rather than of the assessments themselves (Wiliam & Black, 1996).
Some people (for example, Popham, 2006; Shepard, 2008) call for the term formative assessment not to be used at all, unless instruction is improved. In the United Kingdom, the Assessment Reform Group argues that using assessment to improve learning requires five elements to be in place (as cited in Broadfoot et al., 1999).
1. Providing effective feedback to students
2. Actively involving students in their own learning
3. Adjusting teaching to take into account the assessment results
4.