Why I Oppose Test Data for Evaluations

So, it looks like the education bill in Iowa is hung up on the question of whether teachers will be evaluated on standardized test data, or not. For what it’s worth, I’m against the idea for two big reasons. First, test data are not very accurate, it turns out. Second, for those who are interested in school change as I am, test data focuses our attention on the wrong things; elaborate evaluation systems wouldn’t lead us toward the schools we want. Like a bull in the ring, we are chasing the red robe of test data while the real work of school reform, building cultures of learning and excellence in schools, aren’t being adequately addressed.

Tarde de Toros 1. Tiento
Photo Credit: Jesus Solana via Compfight

Just so you know, I’m not against evaluation. I don’t know any teachers who are against being evaluated. We actually enjoy when someone cares enough about what we do to talk to us about our teaching. In fact, we’d like more of that talk (And better talk, too, not just the Roman emperor, thumbs up-thumbs down kind that happens in evaluations.) The vast majority of teachers relish the chance to talk about our teaching whenever we can.

At the core of using standardized assessment data to evaluate teachers is a belief that poor teaching is a prime cause of our current “crisis” in education. Tough evaluations are necessary to make teaching better. Champions of the use of test data seem to believe that either (or both) of two things will happen when we focus on standardized test data: 1) We’ll weed out the bad teachers, bringing up the level of teaching overall; 2) Teachers will “up their game” to avoid being sacked.

Unfortunately, the focus on evaluation as a driver of change distracts us from what we really need to be doing.

First, as many people have noted (here, here, and here), “value added” test measures are not simple numbers that tell an unambiguous story. These numbers are highly variable, changing dramatically from year to year. Also, as Florida is finding, it is difficult to wrap every teacher’s evaluation into the test data. Turns out not every teacher teaches in a tested area. For example, some Florida teachers have argued that they are being evaluated on measures they have little control over. How can a music teacher be evaluated by tests that yield reading and math scores? A history teacher be evaluated on everything but history? Even if practices like this are constitutional (and the May 2nd Florida court ruling suggests it might be) systems like these don’t inspire confidence or our best effort. They become something to endure. And anything that doesn’t inspire a best effort is a distraction, not a driver of real change.

Second, it’s an element of faith on the reformer side that the profession is rife with bad teachers. Getting rid of them is the highest priority for these folks. Yet, what makes us think that there are so many bad teachers out there that weeding them out would significantly up the game? Where studies have been done, the rates of poorly performing teachers is really low. There’s not a lot to show for all that ‘weeding’ effort in past years.

Third, if you work with kids daily you know that standardized tests do not measure the important stuff. How can they? These are kids we are talking about. Grit is as important as knowledge when taking a math test. Empathy never gets tested, yet empathy will be what you and I both want most in a human as we encounter difficult economic and social issues in our increasingly flat world. The level of learning that most tests assess is low indeed. Their questions focus our attention on “skills” rather than thinking, and never, ever on asking good questions, and formulating good ideas. And, the higher the stakes, the more pressure there will be to narrow the curriculum to raise the test scores, regardless of the learning that will be sacrificed. Tests won’t drive us toward better learning.

What will be the effect of all this effort? Sure some teachers will be “weeded out,” but some of the best teachers will eventually leave as they become fed up with the capriciousness of it all and the focus on tested material that will ensue. Who will take their place in a system as arbitrary as the one that is being constructed? Reformers often act as if teaching was a job that people were clamoring to enter. But is that true? It’s not as if there are a boat load of people wanting to take the job. Inject the system with even more arbitrary data, and the number of interested people goes down further, especially for the kind of folks we want to attract into the profession.

Finally, and most importantly, all this emphasis on value added measures distracts us from the real, hard work that needs to be done to make our schools better: we need to build teachers’ capacity to be better teachers. Why focus so much effort on weeding when what will drive school reform is the healthy fertilizer of engagement, teamwork, and motivation?

How do we fertilize, rather than weed? Take the resources being invested in value added measures and use them to create workplaces (and classrooms!) that are about learning, growing, sharing ideas, innovation that is grounded solidly in learning theory, reflection, and critique. In short, we need a revolution in the cultures of schools. A revolution takes resources, vision, and leadership. It’s a roll-up-your-sleeves-and-get-busy kind of work that engages people and systems effectively. It’s not the kind of leadership that you do entirely from behind a spreadsheet. Other places have made these kinds of cultural changes (here and here.) We can, too.

As a teacher, as a human, I know that real accountability comes from people who care about what they do. It originates from the desire to do well coupled with having the tools to do the job effectively, then multiplied by the good will of many people working toward a common goal. Good (and poor) work is super easy to see, you don’t need a lot of spreadsheets to find it. If we work hard to foster good conversations, learning, and reflection, accountability will take care of itself because everyone will be engaged in raising the bar and everyone will have a stake in accomplishing something great…together.

That’s the brave new world I want to live in. Do you?

2 thoughts on “Why I Oppose Test Data for Evaluations

  1. Agree, agree, agree. 50% of my evaluation next year will be based on my students’ scores.

    We’ll just have to see how that works out…

    • That’s just sad…and crazy. I’ve posted to the blog of our Department of Ed. Director (who I share some similar long-term goals with) about this issue, but it looks like he’s supporting the data as information idea.

      This evaluation issue gets turned so quickly into “what are you trying to hide,” while it should be “how do we best build a learning organization?” To me, Michael Fullan’s ideas (and those of Pasi Sahlberg from Finland) are so much more compelling (and realistic) avenues for real change than all that time and effort placed on tweaking evaluation algorithms to ferret out a few “bad teachers” who aren’t hard to spot and aren’t the real problem…

      Grrr.

Leave a Reply

Your email address will not be published. Required fields are marked *