by Danny Rosenthal
Teachers at a school in D.C. probably cheated on standardized tests. Does that mean that then-Chancellor Michelle Rhee’s* education program is fatally flawed and that Rhee has lost her credibility as an advocate for reform? Of course not. The exposé published by USA Today confirms the need for strong measures to catch cheating teachers. But we shouldn’t abandon efforts to measure student progress or make education decisions based on data.
But first things first. The heated debate over this issue has obscured the underlying facts. Just as Rhee’s supporters shouldn’t reflexively dismiss the story, critics shouldn’t assume that the article provides support for their view that Rhee is misguided, negligent or dishonest.
The documents relied on by USA Today are helpfully collected here, and they are worth a look. They indicate that cheating may have occurred in D.C. but that the school system took allegations seriously. Many have seized on USA Today’s statement that Rhee “balked” at a request from the DC State Superintendent to investigate cheating. A different story emerges from the actual response of the district’s data chief, available on pages 17 to 19, and subsequent reports by the testing company (pages 20 to 22) and an independent investigator hired by the district (pages 106 to 108).
In retrospect, the district probably should have investigated cheating more vigorously. And Rhee’s brusque response to the newspaper last week was also a mistake, which she corrected in a later statement. But there’s been no indication that Rhee or anyone in the district office acted dishonestly. The district leadership concluded that a more extensive investigation based on the available data would have been unjustifiably expensive, intrusive, or unfair to teachers and administrators.
Finally, the evidence for widespread cheating is far less overwhelming than some have suggested. USA Today mentions but does not emphasize that two outside analyses recommended against concluding that cheating had occurred. The article focuses on one school, Noyes Education Campus, and presents two main types of evidence for cheating at that school. First, there were high rates of “Right-to-Wrong Erasures,” in which incorrect answers were changed to correct ones. This evidence is highly suggestive but not conclusive. Second, the article highlights suspicions from a Noyes parent and former teacher — both occurring before Rhee’s tenure.
Outside of Noyes, USA Today’s evidence is much less clear. For example, the article notes that more than half of D.C. schools had an above-average erasure rate at least once in the last three years, a claim that Diane Ravitch repeated in slamming Rhee on The Daily Beast. But think about that for a second: If erasure rate were entirely random, and not a product of cheating, we would expect about half of schools to have an erasure rate above the district average each year. And over three years, we would expect this to happen to almost every school at least once. The article also notes that classrooms in many schools were “flagged” for investigation, but it’s not clear what this means.
With that series of disclaimers out of the way, let’s now suppose that there was cheating at Noyes and maybe even other schools in D.C. What should we make of that?
Cheating at a few schools simply doesn’t demonstrate that testing does not yield useful data or that test results should not be tied to rewards or penalties. Critics have implied that there is no way to administer tests in such a way that cheating will be minimal enough to make results reliable. But that’s a colossal a leap from the evidence in this story. A few cases do not prove a rule. In fact, cheating is already rare and could be reduced even further through better procedures.
Of course, the story also doesn’t prove that testing and accountability is the right approach to reform. We’ll have to save that debate for another day. In the meantime, the story shows that cheating can be a challenge for standardized testing. Indeed, it can be a challenge in almost any situation where incentives are linked to performance (think Barry Bonds). But that doesn’t mean we should abandon the project of measuring progress and acting based on data. It means that we should take every possible step to ensure that our data is fair and accurate.
* The author interned for the D.C. Public Schools while Michelle Rhee was chancellor.