The more people and blogs you follow on social networks and through RSS, the more likely it is that you are going to see duplicate data, be it via retweets, forwards, or through many of your friends sending the latest viral videos or images. A new product under development, called Cadmus, looks to filter your real time streams to group similar posts in your feeds to reduce the noise. The service currently works on your Twitter account, your FriendFeed account, or on any number of blogs you add. You can also add many RSS feeds at once via OPML.
Adding Supported Services to Cadmus
In my testing of Cadmus, I found it correctly detected retweets, replies from others to the original sender, copies of tweets sent to FriendFeed, and other topically-related items, even if they did not share keywords. Cadmus was even able to find similar updates that were hours or days apart.
The Results: A Quieter Feed By About 10 Percent
On average, each refresh of Cadmus filtered around 10 percent of my updates. For runs that included 3,000 or so updates, 300 individual items would be grouped or filtered - and testing of a smaller account in the low hundreds also showed a similar 10 percent filter rate. In fact, the more updates I filtered, the higher the percentage filtering would be found. In a run comprising more than 8,000 items, almost 1,000 were "related".
Cadmus Knew Both Micah and Chris Were Watching a Show
Cadmus Saw Guy Talking About an Article TechCrunch Mentioned
Cadmus Linked Thomas Power's Like of Loic's Share
The authors, Anomaly Innovations, who are chronicling Cadmus' development on their blog, show even higher filtering rates, of almost 30 percent, if you looked at an entire week's worth of updates. And while most of the filtered items only had one or two related posts, you can see an extreme version from the last week here.
To use Cadmus, you need to log in at http://thecadmus.com/, using OAuth, to your Twitter account, and you can add as many supported services as you like to the system. Once you have scanned your stream, click the reload button in the top right to get a newly filtered stream. If you're tired of seeing duplicates, want a stream with less noise, or just want to lump similar items, it is an interesting development - one I expect to get better as they continue to update.