This morning I tweeted: “Test-based accountability doesn’t identify effective or ineffective schools, but who is ahead or behind at a moment in time. Judgments at that moment are judgments prior to evidence, which by definition makes them invalid. Time for a better way.”
Think about that in terms of graduation rates so you can really see what I’m talking about: who is better at graduating students, the school at 99% or the school at 65%?
Most people pick the 99%. After all, they appear to have done their job of graduating students, while it appears the 65% school did not. But both of those assertions are statements made prior to evidence, and are therefore invalid. And watch how wrong those invalid inferences risk being.
Imagine the the 99% school. Where is it, what does the neighborhood look like, how often are the streets violent, and how many students regularly go to bed hungry? The odds are that this school is in a middle or upper-middle class neighborhood with fairly stable households, safe streets, and parents who know how to value and prioritize their children’s education. Given that, what would be the odds that these kids would graduate regardless of where they went to school? The answer is, very high. So, if the students are going to graduate in spite of the school they attend, it can rightly be said that the school is not a cause for that to happen and for the school to take credit for it would be a little silly.
Now imagine the 65% school. Odds are this school is in a more challenging environment. There will likely be more violence on the street, fewer parents with college degrees, and a lot more students who regularly go to bed or school hungry. Let’s consider a similar question regarding these students: what are the odds they would graduate without the efforts of that school? The answer there is a little more tenuous–the fact is that in many instances it would not be great. It will likely be the case that a considerable number of students within that 65% would not have graduated under different circumstances. The school in that case can be considered a clear cause in making that happen.
So in that hypothetical situation, which school is actually better at adding value to their students’ lives when it comes to graduation? It is just the opposite of what the rushed judgment concluded. The 65% school, at least in that regard, is adding the greater value to their students’ lives.
The point of this example is simply to point out the danger of jumping to judgment prior to having evidence. The moment you have a comparison of something that is all you have, and you don’t yet have sufficient evidence to make a judgment, or know if one is even warranted.
When you have a test score from a state test you can see at that moment who is ahead and who is behind and that’s it. Making a judgment at that moment is not just silly, it’s wrong, invalid, and possibly downright hurtful.
