In just a few short weeks, students in California will be taking high-stakes tests. But the tests won't just cover math, reading and science. Students will also be responding to survey statements like "I usually finish what I start," or "I can do anything if I try."
A group of big-city districts there is among the first to try to measure students' self-control, empathy and other social and emotional skills — and to hold schools accountable for the answers.
The new federal education law requires states to include at least one non-academic outcome in their accountability formulas, so these kinds of tests are likely to become more common nationally. The Nation's Report Card will be asking questions like this next year; so will the international PISA test.
But researchers I've talked to describe a moment of painful irony, where ideas they've advocated for years are catching on and being taken mainstream — before they're ready.
"The enthusiasm is getting ahead of the science," Angela Duckworth, the psychologist and MacArthur Fellow most associated with the concept of grit, told me in May. She has since resigned from the advisory board of the group that's working to test these qualities in California.
The science shows they're important things to teach and learn. Students who believe they can do better with more effort, who try harder, who can delay instant gratification and control their impulses, who take feedback well and know how to work on teams, are likely to become happier, healthier, more successful adults.
What's missing right now, though, is a consensus on how best to cultivate those qualities.
(The problem is inherent in the slippery language we use to describe them. If social-emotional qualities are "traits," then they might be fixed, even genetic. If they're "skills" or "habits" then, it stands to reason, they can be coached or taught.)
And researchers definitely disagree on the best way to measure them. Asking students to take surveys, or complete certain tasks, are powerful tools in an experimental, research context. But they're not as robust when applied on a wide scale to measure the performance of hundreds of thousands of students, or as a driver of high-stakes decisions like whether a student should move on to the next grade.
"We feel like people will read about something designed for research, and immediately assume that you can start grading 4th-grade teachers on [their students' answers]," Duckworth said. As examples, Duckworth cited the famous "marshmallow test" and her own "grit scale."
The marshmallow test is Stanford psychologist Walter Mischel's famous experiment, where 4-year-olds were told they could have one piece of candy right now, or they could have two pieces in 15 minutes. The ones who managed to wait turned out to be much more successful adults.
The grit scale is a 12-item checklist that Duckworth developed that features self-ratings on items like like "I usually finish what I start."
The grit scale is powerfully predictive in experiments. But if schools and even teachers are graded on their students' replies, it would be extremely easy to nudge students to give the right answer on a survey.
Same problem with the marshmallow task: you can coach children on successful strategies for waiting (like singing a song to distract yourself.) But will that really improve their long-term outcomes?
Chris Gabrieli, leader of the group behind these changes in California, told me that measurements, with consequences attached, are the only way to move the needle on nonacademic skills.
"Until the education system begins to value these attributes, they won't focus on them," he says. "That's why we started with measurements."
He points out that in the case of California, schools where students score low on these attributes won't be punished. Instead, they'll be paired with higher-performing schools for mentoring and support.
The researchers who are raising the alarm have one negative historical analogy in mind: Self esteem.
In the 1970s and 1980s, insights into the importance of high self-worth led to a widespread school culture that handed out praise willy-nilly and gave awards for participation.
This misapplication of research did little to raise achievement. Experts like Jean Twenge have argued that it also led to skyrocketing narcissism among younger Americans.
"When people start thinking, 'I'll make the kids feel good and they'll learn,' that's how the self-esteem movement gains its traction," Stanford's Carol Dweck, most associated with the concept of growth mindset, told me. "People think any intuition, whether tested or untested, is valid."
RENEE MONTAGNE, HOST:
Students in California will be sitting down soon to take high-stakes state tests. And they'll be tested on more than their knowledge of math or reading. Many large districts will be trying for the first time to test students' social and emotional skills. The new federal education law demands that states hold schools accountable for at least one nonacademic outcome, so these kinds of tests are going to become more common nationally. Still, some experts are sounding the alarm on such tests. And for more we turn to Anya Kamenetz of the NPR Ed team who's been following this story for a long time. Good morning.
ANYA KAMENETZ, BYLINE: Good morning, Renee.
MONTAGNE: And so precisely what are social and emotional skills that would be tested in this context?
KAMENETZ: Well, there's no complete consensus or official list, but they can include anything from empathy to perseverance, believing if you try harder that you can succeed, which is known as growth mindset. What we do know, though, is that they are very, very important indicators. Researchers say that measures of, for example, self-control at a young age can reliably predict about half of your life outcomes, not just your education level but your income or even your lifespan and your health.
MONTAGNE: But one can understand the concern - and even the alarm - over tests of what are often thought of as personality traits.
KAMENETZ: Right, Renee, and that's really part of the problem. I mean, are they traits? Are they skills or habits? If they're traits, it's pretty hard to cultivate them. And the experts I have talked to like Angela Duckworth, who's one of the researchers most associated with social and emotional skills, are pretty adamant that we don't yet know enough about these qualities to be testing or judging individual students. And in fact it could be really unfair to do that.
MONTAGNE: Well, how so?
KAMENETZ: Well, say a test shows that a student appears to lag behind her peers in impulse control. But that's something that could be very much affected by trauma that that child's experiencing out of school. Let's say she's homeless. So is she going to get extra help or is she going to be branded as a low achiever? And another problem that a lot of researchers are pointing out - in California, for example, what they're using are these self-report surveys. And so they ask students to answer questions like, agree or disagree, I usually finish what I start. And while that kind of survey is useful in a research context, it could be really easy to game, you know, to prompt students to give the, quote, unquote, "right answer" if you're actually attaching consequences to that survey.
MONTAGNE: You mentioned, Anya, the word cultivate. What that, though, suggests is the opposite of this, that if schools could figure out what the needs are of their students and the strengths and weaknesses, they could actually cultivate these habits or qualities.
KAMENETZ: Well, that's exactly what schools are trying to do not only in California but all around the country. And they're developing lesson plans. They're developing curricula. They're trying to improve in these measures. And they are asking these questions to find out how they are doing. But the issue is really that, you know, to impose consequences on schools for doing a bad job with this before we really know if the tests are reliable or indeed if it's possible, that would be very premature. And that's really what the argument is about right now.
MONTAGNE: Meaning that potentially this project of testing these emotional and social skills is doomed even before it begins?
KAMENETZ: Well, the hope is that over time we'll develop better, more robust measures that incorporate different kinds of data. So, for example, attendance and behavior statistics, even participation in afterschool activities, these are all kinds of more objective third-party ways that you can take the emotional temperature of kind of a school as a whole and figure out, is this a good place to develop these social and emotional qualities that we know are so important?
MONTAGNE: Anya Kamenetz is with the NPR Ed team. Thanks very much.
KAMENETZ: Thanks, Renee. Transcript provided by NPR, Copyright NPR.