“Klout matters to employers.”
When I read that bold headline in Todd Bacile’s post, Florida State University class using Klout to determine student grades, I had a visceral reaction. Why? First, because Klout really only matters to a very small percentage of employers in a niche of the business world, and second, because I wouldn’t want to work for any company that used “minimum” Klout scores as part of their hiring process.
A huge red flag. And while the article clearly explains why Mr. Bacile decided to use Klout and how he used it to help teach his marketing students how to engage using social media and not to game Klout, it concerns me that, as a culture, we are willingly allowing Klout to pull the wool over our eyes. We are a culture of lemmings? We seem easily lead off any cliff that has even the slightest hint of “scientific” validity. In other words, we’ll believe anything with a number attached to it.
Which is more valid, your IQ number or your Klout number?
Hint: this is a trick question. My graduate degree is in School Psychology. I was carefully trained in how to administer standardized IQ tests. As a result, I understand how the scores from these tests can fluctuate based on the test administrator’s tone of voice, the pacing of the questions, and even the color pencil used during portions of the test. But until recently, I had never really questioned the “validity” of the IQ test, except to state that the accuracy of the test could be +/- 5 points, even if administered perfectly. IQ has simply become accepted as a real measure of intelligence.
But what does an intelligence test really measure? What can it measure? In Technopoly, Neil Postman, citing three points made by Stephen Jay Gould in The Mismeasure Of Man, lays out a few of the dangers of abusing statistics in attempts to measure abstract concepts.
“The first problem is called reification, which means converting an abastract idea (mostly, a word) into a thing.” Like “intelligence”, Klout, is not a thing. And the thing it measures, influence, is not a thing. They are all just words with a very high degree of abastraction. “But if we believe it to be a thing, like the pancreas or liver, then we will believe scientific procedures can locate it and measure it.”
And based on what I see daily online, many, many people believe the abstract concept of influence is actually being measured by Klout. We believe it almost as much as we believe “intelligence” is being measured by IQ tests. I was shocked by the number of really smart people praising Klout’s update last week as being “more accurate” and including “real-world” influence metrics, and delighted their score had improved. Which brings us to the second problem.
“The second problem is ranking. Ranking requires a criterion for assigning individuals to their place in a singel series.” And what is better for ranking than a single objective number? Nothing. In fact, using a single number leads us to believe that Klout, influence, is not only a thing, but a single thing. As if this abstraction can have any meaningful objective number formally attached to it.”
The third problem is that it forces us to answer the wrong questions.
“In doing this, we have formulated our question, ‘Who is the fairest of all?’ in a restricted and biased way. And yet this would go unnoticed, because as Gould writes, ‘The mystique of science proclaims that numbers are the ultimate test of objectivity.’ This means that the way we have defined the concept will recede from our consciousness – that is, its fundatmental subjectivity will become invisible, and the objective number itself will become reified.”
We have been measuring intelligence using standardized tests for more than 100 years. And while we have generally come to accept them as valid, experts continue to point out the problems associated with attempts to measure IQ. E.L. Thorndike, the second President of the Psychometric Society, observed that intelligence tests had three defects: “Just what they measure is not known, how far it is proper to add, subtract, multiply divide and compute rations with the measures obtained is not known; just what the measures signify concerning intellect is not known.” Basically, people who administer these test don’t really know what they are doing or what they are measuring. And Joseph Weizenbaum, former Professor Emeritus of computer science at MIT added,
“Few ‘scientific’ concepts have so thoroughly muddled the thinking of both scientists and the general public as that of the ‘intelligence quotient’ or ‘IQ.’ The idea that intelligence can be quantitatively measured along a single linear scale has caused untold harm to our society in general, and to education in particular.”
Is influence any different? I don’t think so. And Klout’s recent changes to include “real world” influence in their metrics is laughable. What Klout has done, quite simply, is hypercharge the process of reification for a number that has no validity whatsoever. And we are falling for the fallacy of ambiguity and making the mistake of treating an abstraction as if it were something concrete, a real thing.
Soon, if not already, the gross subjectivity of an influence metric will become invisible and we’ll simply believe the number. I wish it weren’t true, but article’s like Mr. Bacile’s and the increased number of smart people passing out “I just gave so-and-so a +K in…” tweets like candy force me to believe otherwise.