There's no question that an ever-growing number of Internet users are spending more time creating and consuming data online. While a great deal of attention in recent years has been spent focused on the creation of and access to rich media files, including YouTube videos and photos, be they on Flickr, Smugmug or Facebook, the truth is that often much of the data consists of small files, which can number in the billions for the most popular of Web services. Add to the mix the potential for some files to go "hot" as they become popular, and a fickle audience that expects data to load instantly, and you can see why the issue of storage has been critical to every Web-based business.Traditionally, getting companies' storage issues "solved" through a rapid growth phase has been accomplished in one of three ways. The first is through the purchase and management of expensive or complicated storage networking hardware. The second is through using a storage service provider, like Amazon S3, and paying for data that is used. But the Internet's giants are taking a third route, creating customized file systems, and dedicating an incredible amount of engineers' efforts to test, create and validate. Through this, you have seen names like Haystack from Facebook, MobStor from Yahoo!, and yes, Google's GFS (Google File System), each built independently for the company's unique needs.
Into this world enters a new approach from MaxiScale, who claims its new software system can change the economics of Web-scale software with a promised 10x gain in performance, 100x the scalability and reduced costs. This new venture, unveiled today, was designed specifically with the Web's large data sets in mind, and is not based on any existing vendor's offering. The solution does not offer any customized hardware, and supports both Linux and Windows environments. The system also does not rely on RAID (Redundant Arrays of Independent Disks) to make sure that files are protected, instead replicating the files across storage nodes for backup.
Graphics Courtesy: MaxiScale
Last week, prior to my trip to London, I met with Gary Orenstein, the company's vice president of marketing, and Gianluca Rattazzi, the company's CEO and co-founder. I previously worked with Gianluca when he and I were at BlueArc, and he was the CEO and chairman of the board from 2002 to 2004. In fact, a good number of the team's engineers and leadership hail from BlueArc, so scrolling through the company's phone directory from their lobby was fun to see. As we talked about MaxiScale's approach, it was clear the team had considered the pitfalls that typically trip up companies' growth on the Web - namely price, scalability, latency, and availability.
Why have companies like Yahoo!, Facebook, Google and Amazon each created new approaches to the traditional world of file systems? Because their exceptional needs pushed requirements that standard offerings could not meet, growing too costly, or introducing issues of downtime or latency. MaxiScale calls this new world the "Era of Billions", be it tens of billions of photos at Facebook, streaming of a billion songs at MySpace, hundreds of billions of YouTube video views a month, or exceeding 100 billion ads served by AdMob - the company's first named customer. It's a new world of big numbers.Of course, not every Web company growing virally can ask a core team of engineers to build a new file system, and deploy massive data centers. That's why you see so many turning to S3, which is a great solution for small to medium size businesses, but can see costs grow as demands increase. MaxiScale's platform claims to grow with these companies, on standard hardware, optimizing file serving for all types of files, with a single namespace with highest reliability and predictable performance, even with growth to hundreds of petabytes. The idea? Give companies that are focused on the Web an alternative to expanding either their budgets or their staff, taking their eyes off the ball from what should be their core product development.
You can find MaxiScale at http://www.maxiscale.com or their blog, "Serving Size" at http://blog.maxiscale.com/. See also coverage from earlier today by SearchStorage and StorageMojo.
Disclosure: From January 2001 to April 2009, I worked for BlueArc, where Gianluca Rattazzi, Francesco Lacapra and many other MaxiScale employees were colleagues. I have no current financial relationship with the company or any of its employees.