Leaning PISA Scores
It’s that time of year again. Students and teachers are anxiously preparing for winter break, a much needed respite from the day in, day out, school routine. Meanwhile, governments, and media are pouring over the latest results from the Programme for International Student Assessment (PISA) exam.
Do a quick search on the PISA exam and you’ll find all kinds of articles wondering “Are American Students Falling Behind the World?” or “Can’t We Do Better?” These aren’t terrible questions necessarily, but they’re misguided.
“Can’t we do better” is not an interesting question to me. Yes. We could do better on the PISA exam, and other international exams. More compelling is figuring out whether we should; whether this is the proper target for our energy.
Assessment informs instruction and you get what you measure. This makes sense. If you measure how far you can run without stopping, that’s what your subject will improve on. Your speed may not improve, but your ability to run long distances without stopping should. If you measure students test scores on an exam, that’s what will improve. Should we improve our scores on an exam? Well, that depends on what the exam measures. What’s most important is to ensure that our measure aligns with our intended outcomes.
Let’s stick with running for a bit. Imagine that Jane is trying to get faster. She wants to run a mile in under six minutes. As a coach, it stands to reason then that I should measure how fast she can complete one mile of running and include interventions that will help her run a faster mile time. Measuring how quickly she completes a 5K run, or measuring how far she can run without stopping do not provide the data that lets me know if she’s progressing toward her goal. Just because I’m capable of collecting these other kinds of data, doesn’t mean I should collect them. In fact if I measure data that does not inform the goal (such as distance without stopping) I may try to implement interventions that hinder the initial goal. Strength training is essential to improving times over shorter distances, less important to increasing a runner’s endurance.
The same holds true in education. I need to set goals for what I want to improve in education and measure accordingly. At every level of education, from the individual classroom, all the way up to Arne Duncan’s office, the first question has to be: “what do we want to improve?” (The follow up question is “how do you know that’s the right choice,” but let’s not get ahead of ourselves.)
Do we really want to improve PISA scores? Does PISA measure the data that we want? Is that the best goal we can come up with? I really hope there’s something more interesting that’s getting discussed at the Department of Education. You want to improve PISA scores? Ok here’s a freebie: figure out what the test measures. Teach it. Go home satisfied.
My challenge to education reform is that we can do better. Measuring changes in test scores is certainly easy, but they tell us little. Learning is an extremely complex system the is influenced by countless variables including, but by no means limited to: parents’ education level, income, age, physical health, mental health, peer group, and class size. Let’s work on developing more complex measures of student learning. I’m much more interested in whether students can create a reasoned argument, if they are curious, if they have the social-emotional ability to sustain effort, if they have a growth mindset, and if they’re interested in developing their community. Ability to take a standardized test does not inspire me.
I’m missing the argument for why PISA is a good assessment. Everywhere you look there’s a push for better scores, but no reflection on the value of those improved scores. We’re making an assumption that improving these scores will tell us that education is solved, and that’s simply untrue. Shanghai tops the chart regularly, and their students work in an extremely competitive environment centered entirely around improving scores on exams. Singapore also scores well on PISA and their government spends large amounts of money on consultants to help shift focus off of standardized exams in order to build in more time for discussion and student-centered education. There is evidence that some of Japan’s intense bullying in schools is a reaction to extreme academic pressure to succeed. Is this what we want to create?
PISA scores are easy. We can count and stack rank anything we want. I am suspect of an easy solution to a complex problem. Improving our scores on international exams won’t fix our education system. In fact, when controlling for child poverty the United States scores extremely well already. Ability to follow instructions, and do what you’re told is not invaluable, however it is not enough. We need citizens capable of critical thought and if we want our students to improve their critical thinking, we should model it for them by thinking critically about how we measure academic achievement and how we define successful education.