The two best measurement tools in this sector (in my opinion) are Klout and Twitalzyer. Having enjoyed the valuable information provided by these tools plus achieving a relatively high score I set out to understand (reverse engineer) what makes these tools tick. During my tests and trials I continually drove up my score to some very impressive levels and then one day BANG my daily Twitalzyer impact score plummeted to a new low.
This occurred (Nov. 8, 2010) the day before PubCon an event where I’d be tweeting a lot from and which in turn would generate a lot of retweets (2 factors both tools highly value). I quickly set out to understand why this was happening so I could get it resolved quickly. What I discovered was that Twitter had deleted nearly 12,000 of previous Tweets. Now since all tools don’t just look at what you did today, but what you’ve done over various different pre-defined measurement periods this killed my score. After all, Twitalyzer thought that in my 2 years on Twitter I had only tweeted a half dozen times.
It’s important to note that I have Twitalyzer on manual update. So during PubCon I did not perform any updates knowing that any update would impact my overall score. The disappearance of tweets happened to several of us at the conference and we had all seen it before. So we just left it as expecting it to be fixed in 1 or 2 days. It wasn’t so I set out to contact Twitter.
Of course there is no email or telephone number to contact support at twitter just a Twitter ID. So I browsed their website and was fortunate enough to find a post on Twitter about this problem, add my own post and within 1 day the missing tweets restored.
So I then proceed to update my Twitalyzer account and it appeared to work but something still seemed wrong. At this point I contact Twitalyzer and they told me that there was now something else wrong. That their queries through the Twitter API were timing out or something else was wrong either way, they couldn’t retrieve any information on my Twitter activity.
I was once again tweeting the issue to @support on Twitter and posting in the support area of their website. In no case did I hear anything back, but data started becoming available. Great I thought but the data still looked wrong. A further discussion with Twitalyzer, I found out that they were only getting partial info now on me. For low volume users this wouldn’t be a problem but I’m on the high end of how many tweets I do a day plus how many retweets and references by others (all 3 important measurement tools) so their 7 days averages were (shall we say) extremely messed up.
What was the impact of this on my Klout score during this period? Nothing as Klout was limited to a manual update once every 7 days. So I didn’t update my account during this mess. But they Klout changed how it worked and started doing automatic daily updates. This change in their policy coincided with a new algorithm which gave me an immediate boast which lasted a short period of time, but now that their measurement period starts after my Twitter problem started, my score has been declining steadily.
Klout 30 day trend taken Nov. 17, 2010
Klout 30 day trend taken Dec 30, 2010
So it’s been nearly 2 months since this problem 1st started, I’ve tweeted it to @support on Twitter at least 20 times, I’ve posted it in their help area and at best I might get a day once in awhile where these tools can properly index my Twitter activity.
To help visualize the problem, look at the two charts below (obtained from my Twitalzyer account) that displays my Twitter activity for the previous 7 days. This includes the number tweets, mentions and retweets – 3 key Twitter measurements. The first one was taken at approximately 1 pm on December 30th and the other around 5 pm. Note the reported activity on the Friday, Saturday and Sunday. While I’ll admit I did very little tweeting on the Saturday, Friday was a typical day of approximately 50-60 tweets not the zero reported for both days. Pay close attention to the change in the reported activity on the Sunday. The number of tweets drops from 29 updates to 2 updates. How is that possible? Simple Twitter is blocking Twitalyzer from retrieving information beyond the past 300 updates. The result is what appears to be 2/3 days out of 7of limited impact/influence on Twitter which translates into an overall lower Twitalzyer Impact & (while can't prove it) Klout scores.
Taken December 30, 2010 (at approximately 1:00 pm)
Taken December 30, 2010 (at approximately 5:00 pm)
In a nutshell, Twitter appears to be throttling the API access to my user activity which is killing my influence scores.
To see for yourself, conduct a simple search on search.twitter (http://search.twitter.com/search?q=aknecht) for my twitter ID and you'll see the problem. At times you’ll get a blue whale on the 1st page results, another time you might get 1st page results but the blue whale will show itself when you go to page 2.
While both tools still provide me with valuable information on the effectiveness of what I’m doing on Twitter, Twitter is killing my various influence scores. Why do I care? Because many companies are looking at peoples scores (usually Klout) to decide on which company to hire to help them with their social media campaigns and some conferences look at these scores to see who they should invite to speak at their events. While this is not what these tools were made for, people are using them as such and all of us in this industry now eed to worry about our influence scores.
My options are limited. I know Twitter is a free tool and as I always say “What is the cost of free?” – so I don’t have a support person to call, I can’t withhold payment etc. My contacts at Twitalzyer and Klout also have the same problem when it comes to contacting Twitter. So all I can do continue to tweet to twitter support, post in the support blog and hope & pray that someday someone at Twitter reads this post and fixes my account.
The alternative is to start a new Twitter account and try and get my current 1,700 plus follower to switch which if happened over the course of 1 or 2 days would look like a spam account and would likely be shut down by Twitter. So as in the immortal words of Otis Redding “I guess I’ll remain the same...”
We are seeing a drop in scores almost across the whole population related to a decrease in activity over the holiday season. We are watching this closely to make sure it't not a twitter data issue. We are using the stream feed rather than the rest feed so we shouldn't be missing anything.
I wouldn't bother opening new account. I would give this a couple weeks and see if the scores start trending back up.
Great research and insights on this post!
Thanks for the feedback. My issue started in November so I'm thinking it isn't a holiday issue.
By the way my overall activity didn't decline that much over the holidays, but I will admit for the past week, retweets and references were down.
Great analysis of what was happening and why! I would argue that the various "influence"-type scores calculated by Twitalyzer and Klout DO matter, and they *should* matter to Twitter. If a brand has decided to put resources towards engaging with consumers using Twitter, they're going to need to measure whether they've been able to "move the needle" with that investment, and it would be foolish for that to simply be based on follower counts or retweets. If a brand does all the right things to develop an influential/impactful/engaging Twitter presence, but Twitter winds up undermining the performance measurement of that effort, then those resources could very easily be reduced in the future.
Post a Comment