For the better part of this year, I've seen some odd traffic to my site from Limelight Networks, a content delivery network similar to the more well-known Akamai. Multiple times a day, I would see a visit, with no originating page, drop in on the site, look at a page or two, and then, just as quickly, it would leave. Later, this pace quickened, to the point that if I saw a visitor came in with no referrer, I assumed it was llnw.net. But, in the last few days, the rate has dramatically accelerated, to the point where this spider from llnw.net is more than 10 percent of my total traffic, and it rates as the number one domain accessing my site, ahead of even Comcast.
So what the heck is it doing?Well, it's not 100% clear. The assumed spider drops in and advertises itself as Linux UNIX, running Mozilla 1.8.1.11, and displaying a monitor with the odd square resolution of 1300 x 1300. It generally looks at one page, takes off, and then comes back in a few minutes to get another one. No real rhyme or reason, and it's just as happy to suck down old pages as new ones.
So, is it caching my blog so that customers of Limelight Networks can access the pages faster? Is it taking a graphical snapshot, in the same way that www.archive.org has done to show how Web sites looked over time? I'm not exactly sure.
One theory, voiced in the forums at Webmaster World, titled "Unusual Traffic from Limelight Networks", says the activity is from a robot called "Searchme", a LLNW client. Going to www.searchme.com shows this as a possibility. Searchme, Inc. says, " Searchme delivers more meaningful and targeted search results to its users," and that its "intuitive category suggest technology provides users with a dynamic and rewarding search experience by delivering relevant results that are tailored specifically to their unique areas of interest."
But... alas, no search engine and no demo, yet.
LLNW.net keeps hitting the site, every few minutes.
While I'm sure my site and others get hammered all day long from Google and Yahoo! spiders, they don't trigger my Web statistics software to think they're actual people, as LLNW is doing. It could be because their spider acts so human-like that the JavaScript code I use to track accesses is fooled. But regardless, the activity is picking up steam, and at this pace, I wouldn't be surprised to see LLNW take in one of every five visits here, even if they are "junk" visits.
LLNW.net is grabbing 13% of my traffic, beating out even Comcast!
The question is, if it is Searchme who is pushing this spider, and if they are indeed planning to reveal their work at some point, will the world have the need for yet another search engine? I have no idea. But if these oddities are any indicator, something's going on worth watching.
For those of you who have blogs and Web sites that track this detail of activity, are you also seeing the LLNW traffic, and has it increased over time? Also, has anybody ever seen Searchme in action who can let us know what they're doing?