Study provides control group review of standards-based grading craze
Kramer and the paper’s authors went to great lengths to provide the mostsophisticated (and expensive) analysis of one promising approach to formative assessment. They compared results from two groups of schools.
Given my complaining about the dearth of control group research in the spreading, and I think troublesome, standardsbased grading movement, I must give special attention to a new study that reveals some good results from this reform.
Prepare yourself for one of those predictably dense titles. The 43-page paper is: “The Impacts of a Standards-based Grading System Emphasizing Formative Assessment, Feedback, and Re-Assessment: a Mixed Methods, Cluster Randomized Control Trial in Ninth Grade Mathematics Classrooms,” by Steven L. Kramer, Michael A. Posner, Alexander S. Browman, Nancy R. Lawrence, Jennifer Roem and Kathleen Krier.
The standards-based grading movement is a well-intended but confusing mix of classroom practices. More school districts are adopting at least parts of it, causing bitter battles among educators. Combatants cannot even agree which methods are part of the movement and which aren’t.
I have quoted many good teachers who loathe two parts of the reform: removing any penalties for homework assignments submitted late or not at all, and inflating grades so nobody gets less than 50 percent.
Lead author Steven L. Kramer, a senior research analyst at the 21st Century Partnership for STEM Education, and his colleagues don’t deal directly with those two issues in this paper. The study focused on an approach called formative assessment — what effect grading by learning outcomes and reassessing student progress can have.
The research paper, supported by a grant from the National Science Foundation, said: “In contrast to the traditional system, where assessments are used solely to provide a final, summative judgment of student performance, formative assessment uses assessments intermittently to provide teachers and/or students with feedback about each student’s progress. This enables teachers and students to adapt their teaching and learning strategies to help each student progress based on their individual needs.”
Some of the most influential books on standard-based grading provide little or no control group research. They instead quote teachers who say they liked the results of grading reforms they used in their classes and think other teachers should try them.
Kramer and the paper’s authors went to great lengths to provide the most sophisticated (and expensive) analysis of one promising approach to formative assessment. They compared results from two groups of schools. One cohort of 11 public schools, two Catholic schools and one charter school used an approach called Proficiency-based Assessment and Reassessment of Learning Outcomes (PARLO) for two years. A second cohort of 13 public schools, one Catholic school and one charter school used whatever grading system they liked for two years.
The researchers collected detailed information on 2,736 students. In the PARLO group were 1,649 students taught algebra and geometry by 38 teachers. In that control group were 1,087 students taught those subjects by 27 teachers.
The PARLO teachers graded each student on each desired learning outcome. The grades in descending order were: highperformance, proficient or notyet reaching those levels. Students who wished to do so were allowed to reassess for full credit, after completing additional work addressing their weaknesses. The additional activities included error logs where students explained what they did wrong and tried again, remediation plans for activities that would buttress their understanding, and flashback days in which they worked individually or together to revisit the learning outcomes their teachers were striving for.
After controlling for differences in background and previous achievement levels, the researchers reported that the PARLO students scored 0.33 of a standard deviation higher than students in the control group on end-of-course tests designed and administered by the research team. Kramer told me that meant the group finished about a third of a year of learning ahead of the control group.
That’s pretty good, as such studies go. But the necessarily narrow approach of the study — social scientists distrust broad conclusions — doesn’t resolve the argument over whether prohibiting grades below 50 percent and letting kids blow off homework are good for learning.
Kramer said the study has been submitted for publication to the Journal for Research in Educational Evaluation. He said none of the PARLO teachers inflated grades to achieve a 50 percent minimum, but since they graded students on their final understanding that meant the students weren’t penalized for late or missing homework. “I do agree that how best to handle homework under a standards-based grading system is an open question,” he said.
He liked a suggestion by researcher Robert Marzano that teachers give each student a separate grade reflecting their study habits, such as turning in homework and showing up on time. Would that produce more learning? “I believe it is a researchable question,” Kramer said, “if only one could get the funding.”
School districts who want to try the PARLO method will learn much from this study. What concerns many teachers is that districts are adopting other standards-based grading methods that are not so well researched.
As has happened in the past, these reforms are considered worth trying mostly because they sound good. At such a difficult time in American education, school districts should be more careful about using grading methods that themselves have not been properly assessed.