Real Valley Stories: Search Marketing With the Open Directory Project

Editor’s Note: Part 12 in an irregular series of stories from my many years in Silicon Valley. Part 11 talked about the time I got called into HR's office to meet with lawyers over industrial espionage. This time, a story involving gray hat search engine marketing in the early days of the Web.

DMOZ is now closed. 

Believe it or not, before the world of automated spiders that crawled the entire Web and ranked the results for your searches, much of the way we found content on the Internet was thanks to manual updates from an invisible army of directory editors. Yahoo! defined the initial dot-com era, with its hierarchical oracle making or breaking traffic downstream, as sites were organized and shuffled into categories by unseen text tweakers, much like the editors of Wikipedia try and keep its tens of millions of article pages up to date, with a seemingly fluid mass of edits to keep the live encyclopedia current.

But Yahoo! wasn't the only Web directory. Rich Skrenta and others, also behind Web 2.0 efforts Topix and Blekko, introduced the Open Directory Project, referred to as DMOZ, for it was hosted on the directory subdomain of the Mozilla.org site, with an objective of harnessing contributions from around the world (like Wikipedia), to build a directory, similar to Yahoo!, that could plug into any site that wanted to host one. In a time when many sites were seeking Internet traffic and riches by acting as the front page for the Web, attaching the open directory project to your portal could give you an edge and not require you to bring on a ton of staff.

As with Yahoo!'s directory, a company's inclusion in the DMOZ directory could act as a binary gate as to whether or not potential users would find you. In 1999, working as a Web marketing manager for a Web startup that offered internet faxing and conference calls, I found myself irked to see that our services were not included in DMOZ. Making things worse, the categories I would expect to see us listed in seemed slapped together and without an official owner. Given my understanding of the space and knowing our many competitors, I registered an account and requested to moderate the relevant category.

The DMOZ Internet Fax Listings from 1999 (via Archive.org)

Not too long afterward, I was given the option to update the category, including those of our competitors. As it was nearly two decades ago, I honestly don't remember if I used my company email or a Yahoo! email or equivalent, but I didn't try and disguise where I worked. My application had gotten through.

FaxCube from the year 2000.

When I did log in, I found the content in a state of abandonment. There wasn't much you could do with a site's listing. Give it a title, a link, and a short description, literally about a dozen words. It was fairly impossible to differentiate services from one another, especially in a commoditized space where the core function was pretty straight forward. But I cleaned up the descriptions for all the entries, including our competitors, to accurately display their offerings. Some offered email to fax services, while others offered the reverse. Some offered broadcast faxing. Some required a proprietary fax viewer, and others were completely Web based. That kind of thing.

When content was edited in DMOZ, edits would later propagate downstream. Sure enough, my colleagues noticed a spike in Web traffic to our main sites, with referrers coming from all the places DMOZ was set up. For no cost, I had a clear impact in our customer acquisition funnel, and maintaining the DMOZ became part of my ongoing work.

Later, DMOZ added the option to highlight two entries in the group, which were solely up to the moderator. This, of course, gave me the option to expand from a gray area, to clearly going over the line into promotion. It went without saying I thought our service was the best, and highlighted it at the top. I also chose to highlight a partner site (the Netscape Fax Center) that was white labeling our service, essentially the 1 and 1A positions.

This got even better when we soon realized DMOZ was fueling search results for AOL. The more people who searched for Internet Faxing on AOL, the better we were to see results.

DMOZ listings for Internet Telephony in early 2000. (Via Archive.org)

In early 2000, we introduced a new Web based conference call and meeting product. Addicted to the free traffic from DMOZ, my team asked me to see if I could get our content similarly included on whatever the equivalent was for Internet conferencing. I poked around, and, again, applied to be an editor for the Internet Telephony page. It wasn't a perfect fit, but it was pretty good.

Again, I got approved, and again, I added our sites, and again, I promoted our main site and a partner site to be included as the pair at the top of results. Again, we started to get plenty of Web traffic from DMOZ and its downstream partners, accounting for more than double digit percentage of our traffic to both properties. But this time, it wouldn't last long. Maybe it was the clear marketing copy promoting both services, but one day, I logged in and saw our service demoted to the standard results, with another in its place. I reversed the change, and it wasn't too long until there was a debate upstream as to whether these listings were in good faith.

PhoneCube, in all its glory, from 2000.

Not too long afterward, my moderation rights had been removed from both sections. I'd essentially been fired from my side job of gray hat SEO, long before most of us knew what that meant. And yes, my engineering peers lamented the loss of traffic, as our more organic listings, on AOL search and elsewhere, didn't carry as much weight, once we became one of the fold.

Had I just stayed content to include our services in the listings, or even just to stay a moderator of the less exciting Internet faxing space, it's possible I could have been editing text descriptions for our sites and our competitors indefinitely. But pushing our own products and our partner sites turned out too good to be true. The "Wild West" Internet traffic rush didn't last forever, and, as it turns out, neither did our products. I was laid off after we couldn't get a funding round closed at the beginning of 2001, and a few months later, the company was parceled off to become part of Oracle (see 2006 post). Some of my colleagues still work there nearly two decades later. As for DMOZ, it too shut down a few years ago, a relic in the world of Google and Facebook.

Disclosures: This whole post is a disclosure! I worked at 3Cube during this time. I work at Google now, a perceived competitor to whatever Yahoo! and AOL call themselves now. And as I work on Google Analytics, this is not a post about SEO to all you SEO/SEM folks.