Twitter's struggles with handling high user load have been well documented. To help the embattled site stay up between Fail Whales, they've at times reduced features to just try and keep afloat, blocking the replies tab, reducing pagination, and blocking Instant Messaging, for example.They've also visibly tweaked the rate for authenticated API hits, first down from 70 to 20 and back up to 100. But until recently, unauthenticated API requests were unlimited, which all changed Wednesday night around 5 p.m. Pacific Time, when Twitter ratcheted them down to the same 100 per hour per IP address, effectively crushing many external services that relied on Twitter for their data. And this was done without public mention on the Twitter blog.
As you can see in this Google Groups thread, Alex Payne of Twitter told developers last Thursday:
"In our continued effort to keep things fast and prevent abuse we're planning on introducing rate-limiting by IP for unauthenticated API requests. We'll allow 100 unauthenticated requests per IP per hour, just as we currently do with authenticated requests. Please let us know if you foresee any ghastly issues with this change."And while that sounds all nice and dandy, there were a number of developers who did see "ghastly issues", ranging from Kee Hinckley, CEO of Somewhere, Inc., Richard Cunningham of FriendBinder, and Joel Strellner of Twitturly, each of whom runs a product that depends strongly on unauthenticated API requests from the microblogging service.
But it didn't stop Twitter from making the change, even as Jodee Rich of PeopleBrowsr wrote, "this will blow us out of the water."
Since throttling down the unauthenticated API requests, services that rely on this data are a mere shadow of their former selves.
Twitturly says on status page: "Due to recent changes in Twitter's API, Twitturly is slowly dying."
Similarly, Twist's Twitter charts only show a fraction of the data they did at the beginning of the week. (See: "morning", "starbucks" and "computer".)
Twist's Twitter Search for "Morning"
Twist's Twitter Search for "Lunch vs. Dinner"
In response to the changes, FriendBinder's Cunningham said he will now be changing all API requests to Twitter to be authenticated, to work around the problem. He wrote me, in an e-mail, "There are also some parts of the API we cannot even attempt now. User authenticated requests limits are shared with other sites that the user has used - so we might not be able to get any updates for them if some other site hit the limit for them."
You can see this yourself by trying sites that aim to help you bulk follow those Twitter users who are following you. If you use a site like Twitter Karma, and then try to use Less Friends, you will get a nasty note that says you have exceeded your 100 requests per hour limit. And if you're like me, having fallen way behind on following folks, you might be revisiting one of these sites religiously every hour for the next few days until you're caught up.
Less Friends and Twitter Karma Get a Mere Fraction of the Updates
Last month, Jesse Stay wrote that developers were bailing on Twitter, thanks to outages and broken APIs. With changes like this, despite improved uptime of late, the end to Twitter's problems is nowhere in sight, and services that hoped to tap into their API this way are sore out of luck.
Note: These restrictions do not impact the four golden partners of Twitter's XMPP feature, including Zappos, Summize, FriendFeed and Twittervision.