Listening To Learn – Numbers Can Be Deceiving

When an organization hires me to speak, I take the responsibility very seriously. My job is to deliver valuable content and help the organization achieve their goals. I like to know whether I’ve done my job. And while the kind words of those in attendance, especially my friends, are certainly one key way to judge success, it’s pretty subjective.

In my “Listening As Strategy” keynote presentation at CRS Sell-A-Bration I talked about “listening to learn.” I use lots of tools  to accomplish that goal in the social space and one of them is Rowfeeder. Before Sell-A-Bration began, I set Rowfeeder up to monitor and archive the tweets using the #sab11 hashtag.

One of my goals in capturing the data was to get a feel for who was moving the conversation, but I was also interested in how much my presentation impacted the twitter stream. Below is a screen capture of the top keywords and their frequency of use within the #sab11 tweet stream.

In this case, except for the names of the attendees doing the tweeting, the keyword in the title of my presentation, “listening” was the most used keyword at the event. Add to that the number of mentions of my Twitter name, @respres, and you begin to get a picture of whether I was successful or not.

But does this really help me understand how well I did my job?

This is where some other data needs to come into play. Four of the top five people being mentioned are close friends – myself, Brian Copeland, Maura Neill and Bobbi Howe. In addition, a very small percentage of the audience was actually tweeting during the event. So, those who were tweeting the most and being mentioned the most also have a high affinity for each other. This means they are more likely to tweet about each other and more more likely to want to “promote” each other’s presentation content to the world. The numbers, in my opinion, are clearly skewed.

I would need some other measures to give me a better feel for how well I performed. For example, It would be nice to know the total number of tweeters engaged during my hour of the conference compared with other hours. Not just total tweets by hour, but unique tweeters. I would also like to see average tweets per user by presentation hour. And of course, sentiment analysis would help lend direction to the volume. I have none of that available.

So what have I really learned in all of this listening? I have learned three things. First, I learned that you need larger numbers, both in terms of tweets and tweeters to get truly objective data. Second, I learned that understanding the relationships between the influencers is necessary to assess the validity of the numbers. And third, I learned that numbers don’t always tell the whole truth.

Oh, and I learned that my friends probably liked the presentation.