Of course, these failures could be due to our inability to capture the kinds of subject knowledge that are necessary for good teaching, but they suggest that there is much more to good teaching than just knowing the subject. We know that teachers make a difference, but we know much less about what makes the difference in teachers. However, there is a body of literature that shows a large impact on student achievement across different subjects, across different age groups, and across different countries, and that is the research on formative assessment.
The Origins of Formative Assessment
Polymath and academic philosopher Michael Scriven coined the term formative evaluation in 1967 to describe the role that evaluation could play “in the on-going improvement of the curriculum” (p. 41). He contrasted this with summative evaluation. Summative evaluation’s job is:
To enable administrators to decide whether the entire finished curriculum, refined by use of the evaluation process in its first role, represents a sufficiently significant advance on the available alternatives to justify the expense of adoption by a school system. (Scriven, 1967, pp. 41–42)
Two years later, Benjamin Bloom (1969) applied the same distinction to classroom tests:
Quite in contrast is the use of “formative evaluation” to provide feedback and correctives at each stage in the teaching-learning process. By formative evaluation we mean evaluation by brief tests used by teachers and students as aids in the learning process. While such tests may be graded and used as part of the judging and classificatory function of evaluation, we see much more effective use of formative evaluation if it is separated from the grading process and used primarily as an aid to teaching. (p. 48)
Bloom (1969) went on to say, “Evaluation which is directly related to the teaching-learning process as it unfolds can have highly beneficial effects on the learning of students, the instructional process of teachers, and the use of instructional materials by teachers and learners” (p. 50).
Although educators used the term formative infrequently in the twenty years after Bloom’s (1969) research, a number of research reviews began to highlight the importance of using assessment to inform instruction, the best known of which is cognitively guided instruction (CGI).
In the original CGI project, a group of twenty-one elementary school teachers participated, over a period of four years, in a series of workshops that showed teachers extracts of videotapes selected to illustrate critical aspects of children’s thinking. The researchers then prompted teachers to reflect on what they had seen, by, for example, challenging them to relate the way a child had solved one problem to how he or she had solved or might solve other problems (for a summary of the whole project, see Fennema et al., 1996). Throughout the project, researchers encouraged the teachers to make use of the evidence they had collected about the achievement of their students to adjust their instruction to better meet their students’ learning needs. Students taught by CGI teachers did better in number fact knowledge, understanding, problem solving, and confidence (Carpenter, Fennema, Peterson, Chiang, & Loef, 1989), and four years after the end of the program, the participating teachers were still implementing the principles of the program (Franke, Carpenter, Levi, & Fennema, 2001).
The power of using assessment to adapt instruction was vividly illustrated in a 1991 study of the implementation of the measurement and planning system (MAPS), in which 29 teachers, each with an aide and a site manager, assessed the readiness for learning of 428 kindergarten students (Bergan, Sladeczek, Schwarz, & Smith, 1991). The researchers tested students in mathematics and reading in the fall and again in the spring. Their teachers learned to interpret the test results and to use the classroom activity library—a series of activities typical of early grades instruction but tied specifically to empirically validated developmental progressions—to individualize instruction. The researchers then compared these students’ performances with the performances of 410 other students taught by 27 different teachers. At the end of the year, 27 percent of the students in the control group were referred for placement and 20 percent were actually placed into special education programs for the following year. In the MAPS group, only 6 percent were referred, and fewer than 2 percent were placed in special education programs.
In addition to these specific studies, in the late 1980s, a number of research reviews began to highlight the importance of using assessment to inform instruction. A review by Lynn Fuchs and Douglas Fuchs (1986) synthesized findings from twenty-one different research studies on the use of assessment to inform the instruction of students with special needs. They found that regular assessment (two to five times per week) with follow-up action produced a substantial increase in student learning. A couple of other findings were notable. First, some studies required teachers, before assessing their students, to make up systematic evaluation rules that would tell the teachers when or how they were to make changes to the instructional plans they had made for their students. In other studies, teachers made judgments about what instructional changes might be needed only after seeing their students’ results. Both strategies increased student achievement, but the benefit was twice as great when the teachers used rules, rather than judgments, to determine what to do next. Second, when teachers tracked their students’ progress with graphs of individual students’ achievements as a guide and stimulus to action, the effect was almost three times as great as when they didn’t track progress. These findings suggest that when teachers rely on evidence to make decisions about what to do next, students learn more.
Over the next two years, two further research reviews, one by Gary Natriello (1987) and the other by Terence Crooks (1988), provided clear evidence that while classroom assessment can improve learning, it can also have a substantial negative impact on student achievement. Natriello (1987) concludes that much of the research he reviewed was difficult to interpret because of a failure to make key distinctions in the design of the research (for example, between the quality and quantity of feed-back). As a result, although some studies showed that assessment could be harmful, it was not clear why. He also points out that assessments serve a number of different purposes in schools, and many of the studies showed that assessment that is designed for selecting students (for example, by giving a grade) is not likely to improve student achievement as much as assessment that is specifically designed to support learning. Crooks’s (1988) paper focuses specifically on the impact of assessment practices on students and concludes that although classroom assessments do have the power to influence learning, too often the use of assessments for summative purposes—grading, sorting, and ranking students—gets in the way.
In 1998, Paul Black and I sought to update Natriello’s and Crooks’s reviews. One of the immediate difficulties we encountered was how to define the field of study. Their reviews cited 91 and 241 references respectively, and yet only 9 references were common to both papers. Neither paper cited Fuchs and Fuchs’s (1986) review. While many reviews of research use electronic searches to identify relevant studies, we found that the keywords used by authors were just too inconsistent to be of much help. Where one researcher might use the term formative assessment, another might use formative evaluation, and a third might use responsive teaching. In the end, we decided that there was no alternative to actually going into the library and physically examining each issue, from 1987 to 1997, of seventy-six education and psychology journals that we thought most likely to contain relevant research studies. We read the abstracts, and where those looked relevant, read the studies. Through this process, we found just over 600 studies related to the field of classroom assessment and learning, of which approximately 250 were directly relevant.
We did consider at this point conducting a formal meta-analysis of the studies we had identified, but we quickly realized that with such a diverse range of studies, meta-analysis would simply not be appropriate (see Wiliam, 2016, for an extended analysis of the problems with meta-analysis in education). Instead we conducted what we might call a configurative review (Gough, 2015) because our main purpose was to make sense of