Is it possible that news reporters are becoming more sophisticated in their approach to testing, or is it just Emma Brown of the Washington Post? An article of hers, “D.C. officials’ choice allowed math tests to show gain,” starts with this news:
The four-point gains D.C. public school students achieved citywide on the most recent annual math and reading tests were acclaimed as historic, as more evidence that the city’s approach to improving schools is working.
But the math gains officials reported were the result of a quiet decision to score the tests in a way that yielded higher scores even though D.C. students got far fewer math questions correct than in the year before.
The decision was made after D.C. teachers recommended a new grading scale — which would have held students to higher standards on tougher math tests — and after officials reviewed projections that the new scale would result in a significant decline in math proficiency rates.
Instead of stopping there, or assuming that the officials’ motivation in changing the scoring mechanism (leaving aside their decision to keep that change quiet) was completely sinister, Brown reports that the officials wanted “to continue comparing student performance consistently from year to year” in order to “see an apples-to-apples comparison of student growth.” She then consults an expert:
Greg Cizek, a testing expert at the University of North Carolina at Chapel Hill, said applying new standards and equating test results to align with old standards are both reasonable and defensible approaches.
“But from a purist perspective, I think you pick your approach first and then live with the results,” Cizek said. “It’s not as common to pick an approach, not like it, and then go with a different approach.”
There’s a lot to chew on in this article, but here’s a point that bears repeating:
“Proficiency is not an immutable thing. Proficiency is a judgment call, and there’s a lot of things that go into making that judgment call,” said Charlene Rivera, an education research professor at George Washington University.
I’ve been choosing selections from Assessing Mathematical Proficiency (Alan Schoenfeld, ed.) for my students to read, so naturally my immediate response to this quote is that once you define proficiency, don’t assume that it’s easy to measure. Still, given the limitations of the form, Brown has done well here. Let’s hope a few people read her whole article and don’t get completely stuck on the secret decision angle.