Given all the rumors about Google possibly talking to Twitter about search, or the Mountain View giant taking on the world of real-time, you would think that Twitter's dramatic growth and user adoption would see the microblogging company sitting on a gold mine of a database, as it amasses tweets from around the world and makes them searchable through the search.twitter.com product (formerly Summize). But it appears that the considerable expansion of the company's user base has led to strain on its index, rendering practically anything beyond realtime analysis completely useless, fraught with missing data and error pages.The promise of Twitter's advanced search capability is tremendous - letting you dice your queries by the sender and recipient, and even limiting the date range for said tweets, the location, hashtags or even emoticons. And at one time, it was a valuable resource. Now, depending on which account you're viewing, the data set could be as small as a week, or oddly, in some cases, not available at all.
For example, if I search Twitter to find out how many times Erin Vest (@queenofspain) has mentioned the word "Obama", it would show me five total results spanning the last four days. Modifying the same search to start with May 1, 2009 or January 1 and continue to today completely fails, saying I probably "mistyped the address".
Searching Twitter for Erin's Mentions of Obama
Twitter Says Erin Has Said Obama Five Times
Modifying the Date to the Start of January
The Familiar Fail Page From Twitter Search
Out of curiosity, I performed the same search for "Obama" from Erin's Twitter account on FriendFeed, finding nearly 500 results, going back to March of 2008.
FriendFeed Shows Erin's Tweets Referencing Obama
Similarly, Twitter's advanced search says that I have never sent a tweet referencing Adam Ostrow (@adamostrow), yet FriendFeed confirms that I have.
Twitter Says I Have Never Sent a Note to Adam Ostrow
FriendFeed Shows My Tweets to Adam Ostrow
And lest you think Twitter had left behind us early adopters, archiving only tweets from the celebrities, I was surprised to find that you can't find Oprah's famous first tweet. I searched for the phrase "FEELING REALLY 21st CENTURY" from Oprah and found no results.
Where Is Oprah's First Tweet, Twitter?
I Know The Tweet Exists, Right?
In fact, searching for any tweets from Oprah at all showed no results. Oddly, in parallel, I could see 8 days worth of tweets from Ashton Kutcher and at least a few weeks' worth for my account.
Sorry, Oprah, Twitter Stopped Indexing Your Account
Back in February, I said that Twitter was best suited for following topics and listening to its search engine, and less for following people, and I do use Twitter search every day. But if they are to truly reach their potential, the company has got to find a way to find all the data that today, is missing and hard to find. If it's a scalability issue, Twitter has practically become a utility, like e-mail, and a solution is necessary, even if it means teaming up with a company that knows how to grow and scale. Be the suitor Microsoft, Google, Apple or anybody at that level, each offers a better alternative to the rapid dissolution of features and data integrity we are seeing today.
Of note, we did peruse the open API issues page in regards to search for Twitter, as well as reviewing the Get Satisfaction community for Twitter's Search product, but no comments have been made public about this data being unavailable that I can yet find.
We've seen Twitter go up, come down, remove features and add them back. Is this a temporary blip, or should we never again expect search to work the way it's advertised? I hope it comes back soon, and that Twitter becomes a reliable site to exchange messages, knowing they will be preserved, but their track record makes me very nervous that it may never happen.