Wednesday, November 5, 2014

Brookings: "Poor Kids Suck"


When it comes to slick-looking research of questionable results in fields outside their area of expertise, you can always count on the folks at Brookings. They have a new report out entitled The Character Factor: Measures and Impact of Drive and Prudence, and it has some important things to tell us about the kinds of odd thoughts occupying reformster minds these days.

The whole report is thirty-five pages long, but don't worry-- I've read it so that you don't have to. Fasten your seatbelts, boys and girls (particularly those of you who can be scientifically proven to be character-deficient)-- this will be a long and bumpy ride.

Character Is Important

Yes, some of this report is clearly based on work previously published in The Journal Of Blindingly Obvious Conclusions. And we announce that in the first sentence:

A growing body of empirical research demonstrates that people who possess certain character strengths do better in life in terms of work, earnings, education and so on, even when taking into account their academic abilities. Smarts matter, but so does character. 

In all fairness, the next sentence begins with "This is hardly a revelation." That sentence goes on to quietly define what "character" means-- "work hard, defer gratification, and get along with others." But we push right past that to get to Three Reasons This Field of Study Is Now a Thing.

1) There's concrete evidence to back it up, a la Duckworth et. al.
2) That evidence suggests that character is as important as smartness for life success
3) Given that importance, policymakers ought to be paying more attention to "cultivation and distribution of these skills."

Now, at first I thought point 3 meant that policymakers need to develop better character themselves, and I was ready to get on board-- but no. Instead, Brookings wants character building to be something that policymakers inflict on other people (and they have a whole other article about it). I am less excited about that.

Also, "non-cognitive skills" is nobody's idea of what to call this stuff.

Narrowing Our Focus, Muddying the Water

Let's further define our terms, and distinguish between moral character (qualities needed to be ethical) and performance character (qualities needed to " realize one's potential for excellence").  Some scholars apparently argue that the distinction is not clear cut and/or unhelpful. It appears to me that performance character could be defined as "the kind of character one could have and still be a sociopath," which, in terms of anything called "character," seems problematic.

For this report, Brookings is going to go with performance character. Specifically, they're going to stick with Duckworth's work, defining performance character as a composite of the tendency to stick with long term goals and self control. They reference her revered grit scale and other products of Grittological Studies .

At any rate, for the purpose of this report, we are going to pretend that sticktoitivity and self-control are the key to understanding character. Or, alternately, we could say that we are going to study these two small qualities and do our damndest to pretend that they have broader implications. And to complete this process of obfuscatorial magnification, we're going to give these two qualities new names-- "drive" and "prudence."

We'll define "drive" as the ability to apply oneself to a task and stick to it. We'll define "prudence" as the ability to defer gratification and look to the future. And we will establish the importance of our definitions by, I kid you not, putting them in table form.

Bizarre Side Trip #1

Brookings uses a footnote to cover why they call these things "character strengths" instead of traits. It is totally NOT because that attaches a positive value judgment to them, but because it shows they are deeper than skills and more malleable than traits. Not quite simply born with them, but deeper than simple learned behavior. Remember that for later.

The footnote also has this rather sad observation: "It is hard to learn kindness, but somewhat easier to learn self-control." No particular research base is offered for that extraordinary observation, but it is sheer poetry in terms of efficiently describing the sad inner lives of some folks. Dickens could not have better described the broken soul of Ebenezer Scrooge. But here, as throughout pretty much the whole report, we're going to take the personal experience of one select sampling and assume it to be true for all human beings.

How Much Does Drive Matter?

Here Brookings will throw a bunch of research projects at the wall to see what sticks. They include, for instance, the classic grittological studies that showed that people who tend to complete long projects will tend to complete long projects (because every Department of Grittology needs a Professor of Tautological Studies). "Drive appears to be related to college completion," they observe, and back it up by saying it does better at predicting college completion that SAT or ACT scores, which is a mighty low bar to clear. We're a little fuzzy on how we determined drive ratings for the individuals in these studies; if they have anything to do with high school GPA, then of course they're good predictors. It's like saying that knowing how far your eyeballs are above the ground is a good predictor of your height.

They do have some interesting data from the ASVAB test, which includes some sections that test a student's resistance to mind-numbingly dull tasks (really). And they cite themselves in another paper to prove that non-cognitive skills (sorry-- they backslid, not I) correlate to economic mobility. If I personally had a higher drive rating, I would go read that paper too and report back, but alas, I am not that drivey.

And What About Prudence?

Can I just say how much I love that we're talking about prudence, because it's such a lovely word, steeped in the aroma of maiden aunts and pilgrims. Prudence. Just breathe it in for a moment.

K. For this, we're going to trot out the old four year olds vs. marshmallows research. There has been some great research in the last forty years to parse out what this hoary old study might actually mean and might actually miss. I like this one in particular from Rochester, because it finds a huge difference factor in the environment. Some researchers behaved like unreliable nits, while others proved true to their words, and the result was a gigantic difference in the children's wait time. This is huge because it tells us something extremely important--

It's much easier to defer gratification till later if you can believe that you'll actually get it later. If you believe that deferring gratification means giving it up entirely-- you are less likely to defer. Brookings does not include the new research in their report.

Brookings concludes this section with

Drive and prudence contribute to higher earnings, more education, better health outcomes
and less criminal behavior.And as long as we're just making stuff up:

We can also easily imagine that they are important for marriage, parenting, and community involvement.

Plus, we can imagine that they give you better hair, firmer muscle tone, and fresher smelling breath. Plus, you probably won't get cancer. But as unsupported as these suppositions are, they are still a critical part of the foundation for what comes next.

Yes, Rich People Really Are Better

Brookings now bravely turns to the question of how class is related to these character strengths. And I can't accuse them of burying the lede:

If character strengths significantly impact life outcomes, disparities in their development may matter for social mobility and equality. As well as gaps in income, wealth, educational quality, housing, and family stability, are there also gaps in the development of these important character strengths?

This is followed by some charts that suggest that poor kids do worse on "school-readiness measures of learning-related behavior." Another chart shows a correlation between income and the strengts of persistence and self-control through the school years.

About Those Numbers

Brookings moves straight from the charts to a whole section addressing the fact that there aren't any "widely accepted tests for character strengths." So here's some of the measures and data that they massaged, including some cool stuff from KIPP, "a highly successful national network of charter schools" which-- surprise-- currently employs one of the authors of this paper. Anyway, KIPP has those cool character report cards, so you know they must have a handle on this whole character thing. Well, performance character. Moral character is outside our scope here.

Anyway, they used surveys, behaviors and tests. They also figured out how to crunch large data sets with a nifty punnett square that crosses direct-indirect with broad-narrow, to get four sorts of character markers. Indirect and broad, which is something like "risky sexual behavior" is a one start marker, while direct and narrow, like the grit scale, is the tops.

Using that rating system, they ploughed through acres of US Data Sets, rating each one based on how well it would indicate character strengths (or the lack thereof), and created a few pages worth of charts. I am impressed by the amont of drive and prudence it must have taken to do all this. Bottom line-- most of these from the Fragile Families Survey to National Longitudinal Survey of Children and Adults, don't provide the kind of awesome data that standardized subject tests provide for cognitive skills (choke). So they would like more direct acquisition of data please. We need more standardized character tests in schools.

So, Let's Just Go There

So after sorting through all those data sets, they selected some faves. Their first choice was perhaps unfortunate-- from the Behavior Problems Index, they plucked the hyperactive scale. Now, they would like us to know that this does not certainly does not "necessarily indicate that a child is medically hyperactive (that is, has a diagnosis of attention-deficit hyperactivity disorder). In this sense, the terminology here is unhelpful."

Well, yes. Suggesting that a behavior problem (particularly one tied to a medical problem) is a sign of a character deficit would be unhelpful. Is there any way we could make this even more unhelpful?

Sure there is. Let's link scoring low on the hyperactive scale and therefor demonstrating a lack of character-- let's link that to socio-economic class! Yes, this character deficit ties most closely to being born into the bottom quintile-- also teen mom, especially if she's a high school dropout. (The good news, I suppose, is that the researchers see no link in their meta-analyses to race as a factor.)


They also worked backwards, starting with good outcomes and looking to see how the data feeding into those incomes looked. The same picture emerged-- good things (including not getting pregnant and finishing school) were less likely to happen to the poor kids.

Micro-Macro

The study notes that the BPI hyperactivity rating connected to five specifics

• Has difficulty concentrating/paying attention
• Is easily confused, seems in a fog
• Is impulsive or acts without thinking
• Has trouble getting mind off certain thoughts
• Is restless, overly active, cannot sit still

These five very specific traits connected to the BPI hyperactivity score (a small slice of the larger BPI) which we used as a marker of the two qualities that we picked as representative of the one kind of character that we're studying as the stand-in for the full range of non-cognitive skills. So basically we're doing that thing where we look at an elephants eyelash and use it to make pronouncements about the status of all endangered animal species on the African continent.


Oops

Brookings, who don't always seem to get all of the reformster memos, go a page too far now by suggesting (with charts!) that their prudence and drive measures (which would be a half-decent band name) are as good a predictor of success as cognitive/academic measures. Which means that we can totally scrap the PARCC and the SBA tests and just check to see if the kid is able to sit still and wait fifteen minutes for a marshmallow. I will now predict that this is NOT the headline that will be used if leading reformster publications decide to run this story.

What Does It All Mean?

Brookings is not going to put their other foot in it, so it is not clear whether they want to say that lack of character strengths causes poverty or if poverty causes a character strength deficit. They are clear once again at the conclusion that character is a necessary element of success.

Character matters. Children who learn and can exhibit character strengths attain more years of education, earn more, and likely outperform other individuals in other areas of life. Of course, many other factors matter a great deal, too – most obviously cognitive skills, but also a host of cultural, social and education attributes.

Also, capabilities don't automatically equal motivation to act. And there's other stuff that could be important, too. Including, I kid you not, self-esteem. But we need more data for research. Also, we can build character, so we need more programs to do that, too.

Did I Miss Something?

Well, somebody did. Best case scenario-- we've re-demonstrated that people who come from a high socio-economic background tend to be successful in school, and those who don't, don't. Stapel on some tautologies as a side show and call it an insight.

Or maybe this is a report that buttresses old farts everywhere by suggesting that if your kid can't learn to sit still, he probably lacks character and is likely to fail at life.

And remember up above when we decided to call these "character strengths." That meant these behaviors are deeper than simple learned behaviors, but not quite genetically hardwired. So we're stopping just short of saying that poor kids are born with a lack of character.

But at worst-- at worst-- this is codified cultural colonialism. This is defining "success" as "making it in our dominant culture, which we will define as normal for all humans." And then declaring that if you want to make it as (our version of) a normal human, you must learn to adopt our values. This is going to Africa and saying, "Well, of course these people will never amount to anything-- they don't wear trousers."

Whether character strengths can be developed through explicit public policy is quite another, and here the answer appears to be: we don’t know. Policymakers often fall into the trap of what philosopher Jon Elster describes as ‘willing what cannot be willed.’ But as we learn more about the importance of character strengths, and disparities in their development, the need to move forward – if only through more research and evaluations of existing character-development programs – becomes more urgent, not least in terms of boosting social mobility. For greater mobility, we need not only to increase opportunities, but also to insure that people are able to seize them.

The authors miss a third, important need-- the need to increase opportunities which can be grasped by the people who we'd like to see grasp them. You don't really increase cutting opportunities for left handed children by setting out a larger supply of right-handed scissors. Nor do you help them out by trying to beat them into being right-handed. The best solution is to meet them where they are-- buy some left-handed scissors.

There are so many things wrong with this report-- sooooooo many things-- and I'm about stumped for wrapping it all up in a neat conclusion. It is such a thin tissue of supposition, weak arguments, cultural biases, part-for-the-whole fallacies and poorly reasoned conclusions that I get rather lost in it myself. I can only hope that as of this post, I'm the only person who's really paid this much attention to it.

9 comments:

  1. Sounds like they hired Hernstein and Murray to write it.

    ReplyDelete
  2. Brookings falls into the trap that catches so many in the dominant culture - mistaking difference for deficit. I think it shows a lack of drive and prudence on their part to publish such obvious drivel.

    ReplyDelete
  3. Enlightening and entertaining, as usual, Peter.

    ReplyDelete
  4. Oh goody! Another crap study for my critical thinking class to dissect. They've been analyzing an absurd study whose various graphs and mystic-statistic formulae apparently demonstrate that men under stress start fancying larger girls. But it's time for a change, and this looks like a good one.

    ReplyDelete
  5. If you thought that was fun, this might just make your head explode: http://edexcellence.net/articles/you%E2%80%99re-invited-to-education-for-upward-mobility

    Go ahead, click on the agenda - and try to get through it without the urge to yack up your last 3 meals. I dare you. *shudder*

    ReplyDelete
    Replies
    1. Whoa - this is scary! Wasn't Halloween last week?

      Even scarier was this tidbit in Politico yesterday:

      http://www.politico.com/morningeducation/1114/morningeducation15931.html

      MOVERS AND SHAKERS
      — Fordham Institute’s New Media Manager Joe Portnoy — the man behind the “What does the Gadfly say?” [http://bit.ly/1uef8v3] and “Schoolhouse of Cards” [http://bit.ly/1flrpVy] videos — is headed to the Education Department’s Office of Outreach and Communications. Morning Education was reassured that Fordham’s video production will continue. Portnoy also said he’ll bring the comedy to his new post.

      Must be that the Common Core needs messaging.

      Delete
  6. Peter,
    I'm sure some folks wondered what I was laughing at while reading this post. Thank you for covering this stuff and at least tempering the awfulness with humor. But it's hard to accept that people take this seriously and use it to "inform" policy ideas.
    I wonder if anybody in ed-reformster land has ever considered the possibility that kids living in poverty are under more stress?? That seems to be one of the Journal of Blindingly Obvious studies that never gets read.

    ReplyDelete
  7. Nice work. In 2006, Paul Tough (in a 6,000 word homage to KIPP) noted this:
    "Levin’s contention is that Americans of a certain background learn these methods [sitting up, listening, asking questions, nodding, and tracking the teacher] for taking in information early on and employ them instinctively. KIPP students, he says, need to be taught the methods explicitly."
    Fixing the children of the poor to make them suitable for "some Americans" has become the new civil rights issue for those corporate ed TFA cult members who are otherwise preoccupied with their kale smoothies or finding the perfect latte.

    ReplyDelete
  8. You are railing against your own bias. Who says the character strength of sticktuitiveness is a universal good, such that the lack of it means someone "sucks" (your translation)? I have always felt sorry for us WASPs -- the only thing we are good at is showing up for work on time and obeying others. A person can be funny, kind-hearted, artistic, popular or have a dozen other highly regarded character strengths without being conscientious. The latter just seems to translate well into higher income, no surprise on average.

    The Brookings paper ends by saying the income favorable strengths they discussed can be cultivated and that, to help folks climb out of poverty we should attempt to do so. Sounds like something from the Department of Blindlingly Obvious.

    Now what can we do about us tedious, obedient WASPs?

    ReplyDelete