Web spiders have been around for more than a decade. First utilized for search engines, such as Webcrawler, and eventually growing into the search robots from Yahoo!, Inktomi, Google and Excite, these inanimate programs crawled the Internet, constantly indexing in an attempt to make sure their represented search engine was the biggest, had the most pages, and hopefully, was the most relevant for the search you were executing.Since that time, Web bots have been reprogrammed for specific tasks, from shopping search engines such as Shopzilla, Froogle, and MySimon, to mainstream news, such as Google News - which prides itself on being filtered by bots, not humans. Google News is a key example of a "20%" project so lauded by the search giant, who gives its engineers one full day a week to work on basically anything. One aspiring coder took on the news, and the result is now viewed by millions daily.
Now, Web bots are being even more finely honed, and through newer tools, they are able to be integrated in to well-designed pages that take into account popularity of links, frequency of mentions, etc. and can be tailored to more vertical topics. Techmeme is one of the best - covering all things tech. Part of the Memeorandum family, Techmeme also features sister sites including Ballbug for baseball, WeSmirch for entertainment news and gossip, and the original memeorandum, for politics. Funny how the first focuses of the bots were on those topics where I tend to waste the most time. (Baseball, Politics, Entertainment and Tech) If they come out with flavors dedicated to just Apple or say, electronic music, then I'm sure I'm being stalked.
In the meantime, check out Techmeme and see if a bot is better at finding out the news and organizing it than you and I ever could be.
Listening to ''Can U Feel the Funk'', by Darren Emerson (Play Count: 6)