Is IQ actually AQ? (Mistaking Achievement for "Intelligence")

Have we been mistaking achievement for "intelligence"?

I've just gotten my hands on a copy of Andrew Elliott and Carol Dweck's mammoth Handbook of Competence and Motivation. Following the lead chapter from the editors is an utterly fascinating contribution from Yale psychologist Robert Sternberg, who, in just a few pages, seems to completely shatter the popular myth of I.Q. and intelligence testing. Being an academic text, the writing is a little dry; still, it may be the most important thing I've yet read on the subject.

According to I.Q. advocates and to popular understanding, intelligence tests are able to discern each individual's raw, natural intelligence -- which academic psychologists refer to as g for "general intelligence." So-called g is supposedly an innate, unchanging cluster of intellectual abilities that each of us simply possesses -- it is the hand we're dealt. This pure intelligence is not what we've learned, but simply how well our brains work. Furthermore, it seems to correlate so well with later job performance and life success, people have come to believe that each person has a specific amount of inherited intelligence that truly drives his/her level of success.

To reinforce the idea of pure intelligence tests, the testing community has gone to great efforts to distinguish between these so-called "ability tests," which reveal our innate intelligence, and "achievement tests," which examine the knowledge and skills we've been able to develop.

But what if those distinctions simply don't exist? What if every intelligence test measures a certain combination of skills and knowledge, revealing only what we've learned up to that point in our lives? And what if this correlation is more of a mirage than a true indication of cause-and-effect?

These are Sternberg's staggering -- and yet rational -- claims. "There is no qualitative distinction between various kinds of assessments," writes Sternberg. "The main thing that distinguishes ability tests from achievement tests is not the tests themselves, but rather how psychologists, educators, and others interpret the scores on these tests."

"Conventional tests of intelligence and related abilities," he says, "measure achievement."

Furthermore: "These skills develop as results of gene-environment covariation and interaction. If we wish to call them intelligence, that is certainly fine, so long as we recognize that what we are calling intelligence is a form of development competencies that can lead to expertise."  [We will discuss gene-environment covariation in another post].

In other words, it's not at all fine, because that is not at all how we use the word "intelligence." Intelligence is defined in the dictionary and in popular understanding as "the ability to acquire and apply knowledge" -- our natural ability. Distinct from knowledge and learned skills, it is what is built-in to our brains.

Sternberg argues that no current tests actually measure such built-in intelligence, and that intelligence testers are instead relying on a dangerous circular logic:

"Some intelligence theorists point to the stability of the alleged general (g) factor of human intelligence as evidence for the existence of some kind of stable and overriding structure of human intelligence. But . . . [w]ith different forms of schooling, g could be made either stronger or weaker. In effect, Western forms and related forms of schooling may, in part, create the g phenomenon by providing a kind of schooling that teaches in conjuction the various kinds of skills measure by tests of intellectual abilities."

In other words: we are teaching certain skills in our schools -- skills which do correlate reasonably well with Western job performance -- and then measuring how well kids learn these skill. Then we pretend that the results reveal a person's raw intelligence, when all they actually reveal is how well a child learned those skills. All we're really learning from intelligence tests is that some kids do better than others in school. We are not, as intelligence testers claim, uncovering the innate cause of these differences.

Is Sternberg saying there's no such thing as innate intelligence?
No. But he is saying that such intelligence is "not directly measurable," that it is not one general ability which can be scored, and that it is not inherently limiting. The evidence shows that skills and abilities are inextricably interwoven, and that all skills are modifiable. "The main constraint in achieving expertise is not some fixed prior level of capacity, but purposeful engagement involving direct instruction, active participation, role modeling, and reward."

What about the famous correlation between intelligence test scores on the one hand and job performance/life success on the other?
It's a mirage. The correlation does exist, says Sternberg, but not because one causes the other; rather, it's because they both measure the same abilities. Or as Sternberg puts it:
     "Such correlations represent no intrinsic relation between intelligence and other kinds of performance, but rather overlap in the kinds of competencies needed to perform well under difference kinds of circumstances. The greater the overlap in skills, in general, the higher the correlations."

Sternberg then points to a series of studies demonstrating that practical expertise does not correlate well with analytical ("intelligence") tests but do correlate very nicely with job performance and life success.
-- The Yup'ik Eskimo children of Alaska have "extremely impressive competencies and even expertise for surviving in a difficult environment, but because these skills are not ones valued by teachers" they tend to do very poorly in school. (Grigorenko et al, 2004).
-- In Brazil, street children who are extremely successful in running street businesses, and highly expert in math skills necessary for those affairs, do very poorly in abstract, pencil-and-paper math propblems. (Nunes, 1993 and 1994).
-- In Berkeley, California, there is "no correlation" between housewives' impressive abilities in comparison shopping math and scores on pencil-and-paper math tests. (Lave, 1989).

The essential point being that whatever our innate abilities -- which clearly exist but are still far from being understood and specified -- they do not limit us in a way that I.Q. scores imply. Ultimately, life success is a function not of inherent abilities, but of highly developed skills.

Comments

[Dave McDougall]

"Intelligence" tests record abstract abilities; real-world success is tied to practical abilities. It's easier to go from abstract problems to practical aplication than vice versa. Success corellates with practical ability but abstract ability translates to the practical with reasonable consistency. Practical ability itself can't be measured in isolation (because it is not abstract it's also not abstractable).

The ability to consciously abstract to principles and then apply those principles to new, specific examples is what we call intelligence. Abstraction helps solve problems that occur outside of already-learned paradigms, and thus implies some greater mental flexibility.

One might argue that at least some the abilities of abstraction are heritable.
How does this study refute that idea?

[Jane Shevtsov]

Dave,

In high school, I was lousy in math, the most abstract subject. But my ninth-grade math teacher was absurdly easy to stump with lateral thinking puzzles, which specifically go after mental flexibility. Abstraction is part of flexibility, sure, but it's not the only part.

[Dave McDougall]

What I meant by "Abstraction" is the ability to extrapolate higher-level concepts from practical knowledge (which lends itself well to the application of higher-level concepts to practical circumstances).
Lateral thinking puzzles are closer to this process than math because they involve a particular relationship to puzzle-solving that I think is what's measured on most IQ tests. (Let me know if I'm wrong, as I haven't taken an IQ test since I was 5).

[Dave Russell]

This all rings very true to me. Here's a slightly different slant on the subject. I've frequently met senior managers in business who seem to be far from the "brightest and best" people in the organisation in terms of analysis, lateral thinking, clarity of expression or other factors. So their intellect (subjectively and superficially) appears "low". However they often have the right skills through their experience and learning to sum up situations, take tough decisions and show compassion and leadership in implementation. Their AQ is high.