Time Shifting In a World of Realtime

Nearly three short years ago, the buzz word du jour in tech was “realtime”. Real time discovery. Real time search. Real time serendipity. The explosion of interest in social sharing tools like Twitter, Facebook and FriendFeed (remember this was early 2009) had people (myself included) saying that “Delayed news will no longer be acceptable for early adopters, who will gravitate to the quickest sources of news, wherever they may be.” In practice, while this has occasionally been true, I’ve found a completely divergent innovation to play as big a role in the way I (and others) consume news content and entertainment - that of time shifting, which has remained valuable at a time when most real-time search engines have pivoted or vanished.

Best exemplified by TiVo and other DVRs, preceded by the creaky VCR, the act of consuming media at a time much after its initial airing is so commonplace that live viewings are so uncommon that friends often tiptoe around current storylines for top shows. In some social circles, only the most breaking drama series get the “day it actually aired” treatment - like Breaking Bad, Dexter or Homeland, while everything else goes to TiVo, to be consumed later. (Obviously, I saw the season finales for Dexter and Homeland last night)

News, with some exceptions, can be similarly stored away for later viewing, be it through RSS readers or on your social network of choice. One must not be glued to the real time stream to make sure you don’t miss anything. Instead, the RSS reader traps your own hand-picked links, ready for viewing when you get the opportunity, not necessarily tied to their time of posting.

On the big screen, movies may bank on a massive opening weekend, but with consumers having so many options for entertainment sources, it’s common to see people mention they’ll “wait for Netflix”, which could be months or years away, content to save a few dollars while also getting the comfort of watching in their own home. And if you do find yourself suddenly interested in a show your friends have been seeing which has been out a few seasons, don’t fret, as you can, in almost all cases, catch up - tapping into many options, be they Netflix, Hulu, Xfinity, iTunes or Android Market.

This fall, I made it a personal mission to watch all of Mad Men, after hearing people go on and on about its quality. I powered through it with many late-night Netflix marathons. After finally ordering Showtime, I caught up on this season’s Dexter on Xfinity, and then did the same for Homeland. If my wife misses her favorite shows, she can do the same, tapping into the various video repositories on the web, including the big three networks, typically slower to adapt to the innovation of the web.

I watch my evening talk shows 3 to 5 in a row, from Jon Stewart to Conan, fast forwarding through commercials and skipping uninteresting guests - efficiently getting the best and skipping the rest. It’s almost the same approach I take to my RSS reader or activity on the social networks, skimming, reading, clicking and leaving no prisoners. Even if I’m not constantly connected, and I do a good job of getting close, I don’t feel this sense of missing something.

Realtime reactions to breaking news events, kicked off by an initial discovery, and then rattling around search engines and social media, can’t be duplicated by time shifted content, but for most buckets of content, be they text, audio or video, the drive to be first and in the mix of the story as it is interpreted and curated, is not essential. Advents in information and content sharing over the last few years have instead made “on demand” a reality, getting me what I want when I want it, not when someone else decides for me.

Twitter Search Gets Modern Facelift, Index Still Limited

Since Twitter acquired Summize way back in 2008, the company's search engine has been one of the biggest question marks - for while use of the network has risen dramatically, the company has been unable to keep historical data beyond a few days. Recent enhancements, which Twitter termed personalization, help to separate quality updates from junk, but for the most part, the situation with the search engine remains the same. Today, the search engine saw flickers of life with a design revamp that brings the front-end of the engine in line with Twitter's newish Web interface. It also brings forth the "Promoted" search queries which the company is relying on for revenue.

Over two years ago, I talked about how Twitter's search engine became increasingly less useful over time thanks to a shrinking index and oddities, like being unable to find any tweets from specific users, or missing data, even when search operators were used. At the time, I asked if this would be a "temporary blip" which I hoped would "come back soon", but the company has prioritized other features. In the meantime, a deal with Google to provide realtime updates in their search results lapsed. So we're still stuck with the few days of results, just in a prettier format.

The new Twitter Search Front End, Including Top Trends by Your Geo

In addition to the cleaner look of Twitter search, the service also has a new example pop-up for search operators. While the practically ancient "flight :(" example held over from Summize remains, new are example searches including "from:alexiskold", "to:techcrunch" and "@mashable", nods to the GetGlue founder and top blogs who give Twitter a lot of press.

Also included? The optimistic operator: "superhero since:2010-12-27" which says it will return results "containing "superhero" and sent since date "2010-12-27" (year-month-day)". If you do run that query, you'll get responses dating back all the way to July 23, 2011. Where the rest of the 7 months' results are is anybody's guess.

Search Operator Options on the New Twitter Search

Despite one's social networking preferences, the data inside Twitter is extremely valuable. The company really could have a lead on being the realtime pulse of the planet. This makes prioritizing new tweets the most important, but I'd bet the world could benefit from more than a week's worth of content.

LiveShare Preps Real-Time Visual Communication Platform

Forget everything you think you know about the humdrummery of the competitive but me-too echo chamber of the social photo sharing space. While most of the world has gone gaga over faux artistry around filters that make amateur photographers feel like experts, CoolIris' LiveShare team has been creating one of the most intriguing visual platforms I've seen - taking what I already liked about their platform, including a fun way to share photos around events and groups and vaulting into a new world of real-time, with video, shared links, and text updates.

The result looks almost magical, even to the cynical observer who thinks he's seen everything.

Sitting in on a preview of their new platform, which is expected to become available in the next month or two, LiveShare looks like they've finally delivered on their name, as I saw the company's three cofounders create and share photos, video and URLs instantly between devices, and this content was available for immediate playback and interaction from the receiving party.

For all the hype about "real time" and "near real time" and instant, maybe the terms have gotten fuzzy, but if there was any kind of delay, to me it was imperceptible. When one person would hit publish, the item would immediately display on the other device. Multiple photos would cleanly lay themselves out in a sharp grid, and video never even considered buffering. The result is a new platform for real-time visual communications that I haven't seen anyone else attempt, and one that should be available on any modern platform, thanks to adherence to cutting-edge Web standards.

"By the time you move your finger from the button, it should show up," said Soujanya Bhumkar, explaining his interpretation of real-time.

In an internal note to the LiveShare development team, he wrote: "For the first time, you can capture moments and share them with friends and family with absolutely no delay at all - the instant your finger taps the camera icon your photo or video instantly appears in your personal stream on your phone, iPad and the Web even before the content is fully uploaded. No waiting, no uploading - there is nothing out there in this space that delivers such instant gratification."

LiveShare's origins are in the photo sharing space, and the company may be known for its media consumption, but as they said Wednesday, the natural evolution, based on user feedback, is to discover media highly relevant to the individual, and to assign social velocity to relevant people, so that instead of broadcasting to everyone or narrowcasting to a select few, you and your content is discoverable by those interested.

"If this is possible, my expectation for what live sharing is has completely changed," said Austin Shoemaker, cofounder and CTO of CoolIris. "No more progress bars and hourglasses. Reality and speed are matched to consumer expectations."

The new platform is engineered using WebGL on HTML 5, meaning that on day one, it works in Safari, Chrome, FireFox and Opera. Not only can you passively view content in the browser, but you can drag and drop content from your host PC onto any LiveShare stream and make it an active participant in any ongoing stream. "Consumption on the Web is not a second-class experience," Shoemaker added.

One doesn't have to think too hard about the dramatic use case of curating events around topics and locations with the new LiveShare platform, going beyond siloed areas for photos and other siloed areas for texts. As the company promises, they aren't trying to create yet another walled garden, but instead, to fill the gaps that remain in email and social networking. And I'll tell you, even for this Android-leaning geek, the demo of sharing content on the new LiveShare between two iPad 2s was so sleek, it was tempting to jump back on the bandwagon. Make sure you check out their demo video. It'll have you lusting after their new platform - coming soon.

Lazyscope RSS and Twitter Client Slims Down, Speeds Up

Last week's launch of Lazyscope brought an intriguing new Twitter and RSS client into the world which allowed for full story viewing in a right pane that supported rich media, effectively replicating Twitter's iPad experience on the desktop in a versatile Adobe AIR application. Just over a week later, the application has already seen significant updates that show its potential to become an option for Twitter power users - with the adoption of real-time user streams, a new "mini" single column size and keyboard shortcuts.

Like fellow desktop Twitter client TweetDeck, Lazyscope's support for real-time user streams eliminates the need for a refresh button, as updates flow into the application practically as they are posted to the microblogging network. The more active the users, the more frequent the updates, of course, as they cascade from the top down, simulating the stream in real time.

Lazyscope in Single Column View With User Streams Enabled

The option for a single column view that does not monopolize the screen is also important - as the content viewer (right pane) only opens when an article is selected, and closes when you are click the back arrow to return to the stream. In fact, the single column view is now the default view, giving you your screen real estate back.

Lazyscope in Full Expanded View

In addition to these two updates, Lazyscope, from the makers of Lazyfeed, has turned on keyboard shortcuts, letting you navigate the stream with arrow keys, reposting with the enter key, and my personal favorite, scrolling to the top of the screen with the space bar. As someone who uses keyboard short cuts all day long in Google Reader to go from story to story, doing the same in Lazyscope can make me even more efficient, and as they put it... more lazy.

If you haven't yet tried it, check out Lazyscope at http://www.lazyscope.com.

Tweetbeat Captures World's Events In Realtime, Spam Free

One of the most valuable and pervasive user-generated features of Twitter has been the evolution of hashtags. Whether used to signal discussion of a specific topic, or to participate in a shared event, searching for mentions of popular hashtags has become a frequent occurrence, aimed to capture the pulse of the world in real time. But, like most good things, they too can be used for ill will, as spammers and other jokers latch on to popular events and topics to push their nefarious agenda. Also, search results from Twitter, especially as the service grows in popularity, are not categorized or grouped. Tweetbeat, the latest introduction from search company Kosmix, delivered a solution last week that brings the world's events to the fore, with Twitter as the filter, spam free.

Just like the company's previous (and ongoing) efforts with the Tweetbeat Firsthand extension, which surfaces relevant Twitter accounts through the entire Web, Kosmix is again trying to make the pervasive noise from Twitter useful, bringing the best of the service forward, and pushing the dreck to the back.

Tweetbeat tracks the opening of The Social Network

Tweetbeat, launched last week at the TechCrunch Disrupt event, features what the company calls LiveFollow event streams to see real-time reactions from experts and others, Replay functionality to show tweets that took place during past events, and recommendations to find people who are engaging in events of shared interest.

The Disrupt event was not only a good launch conference for the product, but also a good test case, as Tweetbeat chronicled people's reactions to startup introductions, panels and yes, the AOL/TechCrunch news. Companies and people mentioned in the event were featured in a leaderboard, showing who was getting the most digital ink.

Tweetbeat tracks the Giants and Padres as they fight

But Tweetbeat is more than just tech conferences. Tweetbeat fires up for weather hotspots, movie releases, and all manner of sporting events. (See: http://tweetbeat.com/events. For me, one of the most compelling and fun uses of Tweetbeat is the service's dedicated NFL section, which shows real-time reactions to games across the league, with breakdowns by contest. You can even play back each game to ride fans' highs and lows. Even the baseball pennant race got a look in (See: Giants vs. Padres - Game 162 on Tweetbeat).

Tweetbeat NFL in Action for the Lions & Packers

Kosmix was among the earliest partners to have access to Twitter's full firehose feed, and they are leveraging that access to its fullest with Tweetbeat, pulling in about 3,000 tweets per second, filtering them for relevance and hiding those with low influence - a rank they have developed for every single active Twitter user, utilizing multiple factors, such as retweets and the influence level of those who have retweeted them. The result? An incredibly fast, no nonsense view of the world's reactions to real-time events.

Twitter's trending topics are well known to often be led by frivolous or off-color hastags, and you won't find that on Tweetbeat. While Twitter search may be a quick click away within the service to see what people are saying that very second, Tweetbeat is capturing it all and ranking it for usefulness. You just might find the event you are searching for is not just on Tweetbeat, but organized with style. So check it out at http://www.tweetbeat.com. With the NFL season under way, and fall TV shows hitting the air, you know I'll be sneaking over to see what the world is saying.

Disclosure: Kosmix, the parent company of Tweetbeat, is a client of Paladin Advisors Group.

Why My6sense: Right Time, Right Place, Right Context

After a post talking about the "Me Me Me" portion of social media, it seems only right to follow up and talk about news with me. If you caught Marshall Kirkpatrick's story on ReadWriteWeb (See: My6Sense & The Geek Who Rode His Blog to the Edge of the World) or caught the CinchCast interview I did with Robert Scoble, you know that I have joined digital intuition pioneer my6sense as the company's vice president of Marketing, after working with the company for a year as an advisor as part of my work with Paladin Advisors Group. By joining my6sense, I am the company's first employee in the United States, as the bulk of the company is at their Israeli headquarters, and I get the opportunity, once again, to take a major participatory role in a tech startup which has the very real potential to change the way we consume information, of all kinds, in all places.

As you may know from my previous coverage of my6sense, the company has an iPhone application today that prioritizes content from your subscriptions and social streams, based on your own behavior, automatically. Even more importantly, the company has an "Attention API", which can plug into any content provider, stream client or social network, breaking us out of the box of editorially-divined content prioritization, or best guess social circle recommendations, or the most-common, simple chronological sorting.

Truth is, even for the most focused of us information consumers, we are missing interesting content. More people are sharing more information more easily in more places, and it is impossible to read every Tweet, every Facebook share, every RSS entry or every Google Buzz post. Our intuition tells us quickly what are the most important items, and we find ourselves skipping the rest, or getting annoyed at what we find less valuable, considered "noise". I fear it is only going to get worse.

Solving for this information overload dilemma has seen new companies debut interesting filters, or rely on popularity or community to push items that may interest you to the top, but this is not true human ranking function, or it expects you to explicitly list your interests by keyword or subject. From all the companies I've talked to, interviewed, tested, and written about, my6sense has the best underlying technology, which works implicitly, leveraging your own behavior and not only already works very well, giving me a great mobile feed reading experience, but also sets up the opportunity to make this the official prioritization and personalization engine for the Web.

Even if we wholly adopt the belief that there is value in real-time reactions and recency, there is much to be said for the right content to find its way to you at the right time, in the right context. Factors that make you decide whether to read one article or update over another can be dependent on many different things - the source, the author, the headline, the subject, or even the time of day or location, and my6sense's Phds are working on divining the best stuff just for you - not anybody else. And in the year-plus that I have used the iPhone app, I practically never use its alternate time-based view, as the content is unfiltered and usually off-topic.

What makes me excited enough about my6sense to put Paladin on the backburner, despite the firm's doing very well, and having just hired a new partner, is that this is the right time to start plugging in prioritization and personalization for the entire Web. If the entire story were "just" an iPhone application, it wouldn't be interesting. But consider the benefits of personally curated content sites from major media outlets, social networks and clients. Consider how valuable it would be for those sites if users found content more relevant and stayed just percentage points longer per site, read more stories, and clicked on more relevant ads. Consider the real value of not contributing to the world of information overload, but cutting through the noise like a warm knife through butter, find the best and ignoring the rest.

My6sense already has the best technology, from all I have seen and heard, to make this happen. Now comes the opportunity to help do my part, an active role, to transform the company from technology leader to market leader.

And if you are curious what this means for the future of louisgray.com, I will of course be changing my About page and will offer disclosure if there is ever a time that warrants it, just as I always have. I have tried to bring solid content and ideas and smart technology here since 2006, when I worked for a hardware company, and in the last year, as a new media Partner at Paladin. The perspective changes, but the goals remain the same. Please do continue to keep me honest, and always let me know if you think I have crossed the line.

If you haven't yet had a chance to catch the call I did with Robert Scoble from Monday night, it is embedded below.

Real-Time News Needs to Reward Authenticity, Curation

At the Search Engine Strategies San Francisco event Wednesday, Echo's Khris Loux sat down with me and representatives from UltraKnowledge, CBS Interactive (formerly CNET) and CNN.com to discuss the rise of citizen journalism and social media, intertwined with traditional journalism, and how mainstream media outlets can adapt their practices to include curation as part of their offerings. Our panel's determination could be summarized as saying the ability for the public to launch stories and extend storylines related to realtime has removed the ability for mainstream media to be gatekeepers, furthering the need to accurately determine authenticity and truth. And where mainstream media cannot have exceptional access, be it to geography, sources or timing, part of their new role is to discover and highlight high quality content, regardless of its source, in effect, adopting the role of curator.

Best exemplified by CNN.com's iReport, citizen journalists are being given a chance to report on equal footing with the mainstream media. While iReport gives these would-be reporters a place to piggyback on a major brand, many others have taken to their own blogs, social networks or Twitter to report live, and unlike the traditional approach where media authors would issue a story and call it a day, the life of a story now often starts when it hits the Web, spawning comments, shares and retweets. But as citizen journalists and bloggers are given the same capability to gain visibility, mainstream media's much-celebrated structure to put articles and stories through the vetting process sometimes falls astray in the race to get the story out quickly.

CNN's iReport Has "Vetted" about 5% of all Submissions Today

All time, about 7% of all Submissions are "Vetted".

A little over two years ago, I highlighted how quickly incorrect news can spread in the age of microblogging. (See: Smart People, Stupid Tweets. Fake News Spreads Fast on Twitter.) While many of us know not to believe everything on the Web, even if it is in Wikipedia, there remains a need to separate fact from fiction. With so many publications out there reporting on the same information at the same time, without having additional insight, the task of curating the very best of the analysis and repeaters can become a very serious task.

In January, I broached this issue, asking, "Can You Filter for Quality News Amidst Instant Analysis?" and earlier still, in December, highlighting how some news organizations are in fact incentivized for behavior which may lead away from taking the extra effort sometimes required to get the story right. (See: Growing Grumblings on Tech News Don't Address Incentives)

As excited and promotional as I am about the ability for anyone to create and share content, I am equally worried that incentives to promote the discovery of accuracy are not clearly defined, and it is often very difficult to find a real authority on a given topic in a world of instant analysis. When a story breaks in a real-time platform, like Twitter, the person who first reported the earthquake, or was present on the scene of an event, is only an expert for a short time, and when that topic has faded, new influencers for new stories are needed. This leads to a need for trusted sources to harness the truth from the originator of news, wherever it may be, mass media or citizen journalism, and rapidly verify accuracy.

But the sheer volume of feedback can overwhelm even the most well-intended. CNN.com reported one recent popular article achieved more than 30,000 comments - a practical impossibility to moderate by humans, leading to the desire to somehow separate the wheat from the chaff and surface the most accurate and supported insight. As one knows, trusting the community is possible, but usually results in an across the board maligning of those who share opposing views. See any political blog or sports blog for this particular variety of trench warfare. See any comment stream on Yahoo! News or on YouTube to see the quality of the masses at work.

The answer lies in finding a way to reward the truth-seekers in the real-time Web, and to reward media, be they an established brand or an emerging one, for discovering accurate insights, or simply harnessing them. The world of real-time storytelling is evolving before our eyes, and in some cases, its sheer ephemeral nature promises to introduce as many issues as it solves. We, as readers, creators and sharers, should be working diligently to keep our streams clean and provide value to all those who come in contact with our content.

For more thoughts, see Echo's essay: Real-time Storytelling

48ers.com Offers Realtime Social Search Across Networks

For the most part, realtime search engines, including OneRiot, Topsy and Twazzup, have focused on indexing the world of Twitter. That makes some sense, given the significant number of updates flowing through the microblogging network, but it covers only part of the worlds real-time sharing spectrum. A relatively new site, called 48ers, is looking to return results not just from Twitter, but also Facebook, Google Buzz, Digg and Delicious, and more interestingly, it lets you show results from each network on its own, should you choose. While Twitter may still win in terms of the total number of discovered results, they are intertwined with people who have chosen to post in other places.

Much like practically every search engine that has debuted after Google's spartan interface displayed the UI of the future, 48ers.com's home page is relatively light, showing only a search field, and in the age of Twitter, Trending topics. Click on a Trending Topic, such as "Hannah Montana Forever" and you see results that match the query.

A 48ers.com Search for VC Funds Shows Results from Buzz, Twitter

No big deal, right? True. But if you look at the top left of the search result page, you see you can now filter the search by any of five networks. Click on Facebook for the same query, and you will see shares on Facebook with the same match. Similarly, clicking on Google Buzz, Digg or Delicious will further segment the results by their source.

Searching 48ers for Oakland A's Updates from Buzz

Clicking on the results of any query takes you to the status update, if on Twitter or Facebook, the Digg submission, the Delicious bookmark, or the Google Buzz entry. (For example, this special Google Buzz entry on Hannah Montana)

Breaking News updates via 48ers from Facebook and Twitter

Like Twitter's vanilla search engine, 48ers offers useful searches, including "breaking news" and the more social ""totally recommend", which gets used surprisingly often.

Where 48ers comes up short is in continuing the realtime flow. Twitter's search engine, despite its failings, prompts to refresh as new entries are posted. FriendFeed's has them flow from the top. 48ers does not do that, so while the initial query may be made on realtime sources, the results themselves are static.

You can find 48ers on the Web at http://www.48ers.com or on Twitter at @48ersdotcom.

Supertrackr Tracks "Anything" on the Web, Instantly

Thanks to advances in the Web's real-time infrastructure over the last few years, our acceptance of latency or delay in discovery has practically been eliminated. We don't want to wait minutes or hours or days for news and information, and the freshness of content is essential. Often, when people want a near-instant reaction to world events, they're not looking to a filtered editorially driven source, such as CNN or Yahoo! News, but instead, they are looking to social networks like Twitter and Facebook, and often blogs, to get the very latest.

This drive to be completely up to date has led to the development of tools that drive instant notification of "matches" to saved queries. Early in Twitter's infancy, one had the option to "track" a word and get instant notification of its being tweeted. But due to Twitter's rapid growth and some infrastructure holes, the feature has long since been dead. In its place has risen third-party tools and products that tap into Pubsubhubbub, a real-time notification protocol powering most of the world's blogs today, and many different content sites.

Among the most aggressive promoters of Pubsubhubbub and real-time notifications has been Julien Genestoux of Superfeedr, an infrastructure for real-time parsing of feeds in the cloud. Superfeedr recently launched a "Track" tool to instantly find Atom-based entries that match keywords pushed to a developer's application.

Chatting With Supertrackr and Tracking Results for Android

One of the simpler and more interesting applications of this tool that I have seen is called Supertrackr. Developed by @Harper, Supertrackr operates within any Jabber-based IM client (or Google Talk), and lets you follow or remove keywords, via something of a command line, and get updates instantly. The app taps into Google's App Engine and Superfeedr to drive results into your chat window as new entries.

Removing Android, and Seeing Updates on Other Keywords

For practical purposes, you probably don't want to "track" popular terms like "twitter" or "android" or "e-mail" because your chat window from Supertrackr would get flooded and you would have new entries before you even acknowledged the last ones. I tried, and it works, as advertised. Operating Supertracker is as easy typing "/track word" to follow a term and "/remove word" to stop. So if you are in a Jabber-based IM client or use Google Talk often, you could be leveraging Supertracker to find all URLs discussing hot topics, or following your company or brand.

Twitter may not have brought Track back, despite Steve Gillmor's insistence they do, but other developers are filling the gap to get us closer to that monitoring utopia of instant notifications on anything from any source anywhere. Check out Supertrackr at http://www.supertrackr.com/ or Superfeedr at http://superfeedr.com.

Lazyfeed Gets Social, Interactive, With Real-time Remix

Lazyfeed's July 2009 debut was arguably one of the most interesting new services to hit the information consumption market in the last few years. Built upon the belief the people wanted to follow topics, rather than feeds, and get updates in real time, the product offers a unique, and lazy, way to see streams of info on topics you like flow into your field of view, while prompting you to find more related sources and topics. Today, after months of incubation, Lazyfeed launched a new interface that brings the product out of the world of solitary RSS feed reading, and into the world of social.

True to the product's origins, Lazyfeed still lets you discover new articles, in real time, based on keywords you select. But you can now group those keywords into something called a "Channel", and you can make those channel mixes public. Connect with other friends using the service, and now you have a community building around those terms which you follow - sharing you interests with other connections in your social graph.

The New Lazyfeed With Channels, Followers, Posts and More

For example, I created new channels on Google, Apple and techno music. In time, I could build out my Lazyfeed to include updates for traditional media, sports, blogging and social media, or practically any topic. If it gets blogged, Lazyfeed will find it.

Creating the Google Channel With Keywords

Two of My Channels and Updates By Keyword 

With the added social elements of Lazyfeed, I can now see what topics my friends are following, and what specific posts have caught their eye - and now, every user has a public profile page. For example, mine is http://www.lazyfeed.com/user/louisgray. Is this yet another place to follow the same people? Maybe. But it's a new category of content sharing that hasn't been made possible before - with the possible exception of Google Reader sharing.

Posting from Lazyfeed to Facebook Is Simple

From within any Channel, I can now make new "posts" in response to articles that I have found particularly interesting. These posts, with my comments, will be shared with my new Lazyfeed network. This way, we can discuss topics we both find of note, and can make the once passive approach of real-time feed consumption one that you are now doing with your peers. Additionally, you can send these posts to Twitter or Facebook to alert others to your activity on Lazyfeed.

Lazyfeed has come a long way since its initial debut nearly a year ago. Now that you don't feel you're doing this in a solo environment, it's likely the community elements will spur major interaction and traffic. Find me here: http://www.lazyfeed.com/user/louisgray

Google Buzz Team Tweaks Tweet Imports, Hints at Filtering

As Google Buzz users discover their own approach to using the service, and being selective as to which programs they pull into the site, many have been dissatisfied with the way Buzz has handled Twitter. Unlike other aggregation services, including Cliqset and FriendFeed, Buzz's imported tweets are not imported in real-time, and can seem very disjointed - made worse by their being imported in clumps. Today, one Googler explained they were going to solve the clumping issue in what was termed a public experiment, making things "less awful" in the interim before more capabilities are released, including filtering of imports by source - a popular option seen on sites like FriendFeed and Facebook.

Why Buzz does not pull in tweets in real time is extremely curious, given Google's much-publicized relationship with Twitter, their focus of late on recency in search, and of course, Buzz's centering around standards, such as PubSubHubbub, a big player in the real-time world. That Twitter does not support PubSubHubbub is obviously one contributor, but other services have managed to find a way to get Twitter updates with practically zero delay, through leveraging Twitter's APIs. The result is a truly unsatisfactory experience, contributing to many people removing Twitter as a service feeding Buzz.

The way Buzz polls Twitter for updates now can mean older updates get pulled in as new along with more recent items. To date all "new" updates were added concurrently, but now they will be redistributed using the timestamp of the tweet from Twitter, in effect shuffling those updates throughout the network. This will make Buzz's mysterious "unread" counts even less relevant, and should be seen as a band-aid.

Updates from Twitter Now Shuffle In By Their Official Timestamp

Josh Wills, an engineer on the Buzz team, explained:

"Personally, I really like Twitter, and I want better integration between Twitter and Buzz. My hope is that this is a temporary change until we work out a way to play nicely together that is beneficial to everyone."

Josh went on to say that the changes were intended to "make things less awful" while things get "worked out" to make the tweets appear in real-time. Whether that's a business development decision or an engineering one, to make things "play nicely together" was not explained, but he added that he believed "real-time twitter updates don't make sense w/o the highly-anticipated source filtering."

The ability to hide sources, such as Twitter, from one's feed in Buzz will be as important as it has become to hide services in FriendFeed, or to stop seeing Farmville updates in Facebook, should you not be interested. While Buzz has been fairly close-lipped in terms of talking about future features, Josh's comment referring to the filtering shows it's on their radar, and no doubt coming, rather than being speculated about endlessly by hopeful users.

Older Tweets that do enter Buzz can of course be made more prominent if they gain conversation, and therefore, get bumped to the top of your feed - but from what I've seen, most engagement comes on native items on the network.

Ping.fm Taps Superfeedr to Broadcast Realtime RSS Feeds

While the mass broadcast of status updates across multiple networks is something done best in moderation, the opportunity to distribute one's RSS feeds, from a blog or any other source, to potential viewers, no matter where they are, is core to the nature of the technology itself. Today, Ping.fm, the Seesmic-owned utility that allows for multi-network pollination, announced a new feature that gives RSS feeds the same flexibility as traditional status updates.

The solution is powered by Superfeedr, the real-time cloud based feed parser and publisher who I have long been bullish on thanks to their support and promotion of Pubsubhubbub, a protocol behind real-time data transmission from network to network.

With Superfeedr having already proven capable of acting as the back-end for real-time RSS distribution across the Web, Ping.fm's turning to Julien Genestoux and team makes logical sense, rather than trying to reinvent the process. Now Ping.fm is much more than a massive broadcaster of status updates, but a vehicle for RSS too.

See the official announcement here: Ping.fm Provides RSS Support

Users vs. Companies: Conflicts over the Real-Time Web?

If 2009 was the year of real-time Web, with practically every major service finding ways to bring content to its users instantly, 2010 is about optimizing the new real-time world, expanding interoperability between sites, finding more ways for users' content to be discovered, and taking the potential of real-time out of the status world and into the real world. Today, at the South by Southwest Interactive event in Austin, Texas, one panel asked if we were making serious progress in this vision, and if companies, feeling increased competitive pressures, are short-changing users in the process.

Marshall Kirkpatrick of ReadWriteWeb, who moderated the panel, featuring representatives from Collecta, Google, Gowalla and Microsoft, said "the real-time Web is a big, complex and multi-headed beast," adding, "almost as many people you talk to on the subject will give a different perspective."

For most, the real-time Web represents reducing latency from the time updates are published and when they are experienced practically to zero. This can be anything from updates from blogs to downstream aggregators and RSS feed readers, status updates from social networks to other points in the ecosystem, or instant alerts from the Web at large that a saved search you requested has found a positive match.

But one of the existing problems with the real-time Web that has occurred is that despite the focus by many services to solve the same problem, many have done so without delivering true data interoperability - and other services are trying to solve for real-time without having full access to users' public data.

"Back in the day, you couldn't send e-mail from AOL to Compuserve, and today, you can't send data from Google Buzz to Facebook," said Brett Slatkin of Google's App Engine team, and co-author of Pubsubhubbub. "Part of what we are trying to work on is breaking down these barriers that connect to different sites. If I am on Buzz and Marshall is on Identica and Jack is on Twitter, we should all be able to communicate."

Standards have evolved in the real-time Web space, from OAuth to PubSububbub, WebFinger and Salmon (as documented here), but that's not to say there aren't still heated debates over these standards, or even which version of standards should be supported. (See this article for a discussion of OAuth 2.0)

"I try to be a practical person, and when I hear about a family of specifications, it sounds like a family of work," said Dare Obasanjo of Microsoft. "There is clearly a place where we have a common pain that we can work on. There is a bunch of shared pain, and the way you have to get real-time service is to work on APIs, and that is a clear starting point for standards. Pubsubhubbub can help solve that problem, but I get concerned when you have to implement certain specs to solve that problem."

"These specifications we agree on should be useful on their own," answered Slatkin. "When you implement a specification like HTML, you are not buying into an ideology."

As the real-time Web's protocols are debated and deployed, so too does the application of these services. Google Buzz and Facebook have received scrutiny for their aggressiveness in converting assumed private data to public, and Netflix recently canceled an algorithm development contest thanks to concerns of assumed privacy violations.

"When talking about privacy, right now, unfortunately, the social networking market is failing, and they have little incentive to encourage user privacy," said Obasanjo. "I am waiting to see when people find what they thought were private updates as part of trending topics on Google and Bing. Users and companies are in conflict."

Obasanjo gave the example of Twitter needing its users to be public in order to drive value into the system. After all, if users were all private, there would be no trending topics, and thus it is Twitter's best interests for updates to be public. "There is a factor that if a user wants to be private, it subtracts value from the system," he said.

Beyond these concerns, known benefits of the real-time Web are scratching the surface of what could be done with more expanded to real-time data from other sources, it was argued. Slatkin forecast a time where you could query supply chains for inventory and purchase locally instead of from Amazon.com, turning economies of scale on their head. Scott Raymond of Gowalla talked about intersecting real-time Web technologies with geodata to show trending locations and the hot parties of the moment, by decaying the relevance of checkins over time. Jack Moffitt, CTO of Collecta, said a development environment for new tools and applications that leveraged zero latency was becoming "very interesting".

"All these guys are working on realizing the potential right now, working on real-time data," Kirkpatrick said. "Brett Slatkin said it was important people focus on the unforseen future that systems we worked on to support undiscovered use cases - things are going to get real crazy real soon."

Web-wide adoption of RSS and Atom standards has eliminated the problem of publishers providing their data, and tools like Pubsubhubbub are working to get data from one site to another faster. "Polling doesn't scale and you need a push notification to deliver it. It's possible we will have multiple winners, and we have to consider privacy considerations that people won't want their data available to everyone," said Moffitt.

The element of real-time is being layered across the Web, and it seems to be happening even if developers aren't completely in agreement over the tools needed to optimize the experience or if the debates on privacy versus public data are solved. And there's a lot of room for real-time to grow outside of the statusphere and to more traditional markets. The question is can developers provide solutions that don't have users running to the FCC?

Activity Streams Aim to Be DNA of the Future Web

When the well-respected open source advocate Chris Messina announced he was joining Google in January, many folks were concerned that his being absorbed in to the big company Borg would mean a cessation or redirection of some of his projects targeting the next generation Web, possibly in exchange of proprietary efforts to promote the company's products. Today, at the South by Southwest Interactive event in Austin, Texas, he spoke on how he and others in the community, both at Google and outside of it, are working to bring more meaning to our social networks, activity, and feeds, through extending today's data portability standards to include more information and more relevance. Messina walked through a history of the Web's publishing, from static portals of a decade ago, to today's RSS and Atom-powered sites, and suggested a future with even more information, based on streams, that tells a story.

Messina, after expressing his excitement about working with a team he believed was leading the industry in things he cares about, including the Data Liberation Front, letting data move from one site to another, said he was focused on what he called "generative structures" that were the underpinnings and DNA of how information is shared, updated and transmitted.

As he recounted, in 1999, portals ruled the Web, and people, myself included, would put their data in sites like My Netscape and My Excite, customizing these sites with headlines from third party services, primarily tapping RSS, which offered the headline of a story, a link and its description. In an era when publishers wanted to not give away their data, it was "the best we could do", he said. By 2005, a new extension of RSS was promoted, called Atom, which was still the essential concept of syndicating data from one Web site to another, but also adding an author and an identifier for the atomic bit of information.

Now in 2010, little has changed. Most news feeds of today, be they on Facebook, on customized portals, or the headline and link model that dominates Twitter, are fairly simple, and they don't indicate intent. As Messina said, "It's not all that different from the last 10 years, and that gets kind of depressing".

But what has changed is the increase of sharing rich media in these places, on platforms designed for "dead tree media". He said we should be able to show what we did, who with and why we were doing it, and that needs to happen through new richer formats for the social Web.

Activity Streams, an extension to the Atom Feed format, is looking to accomplish this by extending Atom and RSS with new aspects, including a verb and an object type. The world of FriendFeed, which supported a unified feed of 58 different services, where people could have one single stream that represented their identity online helped guide much of ActivityStreams' framework. As entries flowed to FriendFeed, they largely represented actions of posts, shares, bookmarks, reviews, from different sites. But ActivityStreams is aiming to do more than just syndicate data from one home page to another, as RSS and Atom have done for a decade, but also display intent and meaning.

"If your goal is to help people produce meaning, knowledge and culture, you have the basics for a pretty compelling social application and can motivate people to act," Messina said.

But with more streaming of information from many different sites, it can exacerbate the assumed problem of information overload - and tools need to be further developed to help us consume the data.

"We snack on information. It may feel like overload, but the tools haven't caught up," Messina said. "The solution to data overload is more metadata and we are at that point where can start generating that. We take the basic construct from 1999 and weave in some additional information - data about data."

As I outlined in my summary of DeWitt Clinton's talk on Google Buzz at the beginning of the month, Activity Streams are playing a big role in this new network, and these streams are intended to be open, not just for a company like Google, but other social networks that are tracking individual's activity and intent. The goal is to make discovery of intent data ubiquitous and transitive between sites, in the same way that RSS and Atom focused on publishing of data from one site to another.

Messina called the work on Activity Streams as iterative "baby steps", but ones that focus on getting today's rich media activity a home with a rich experience, and to make this process easy for service providers.

"If you have a Web site that has people doing things on it, and they have a feed they are taking with them, it is fairly trivial to add ActivityStreams information," he said. "Essentially we have the verbs and object types represented."

You can find out more on the continued development of ActivityStreams at http://activitystrea.ms.

Twitter Unleashes the Firehose to Seven New Partners

As Twitter promised back in December at LeWeb, the company has now extended access to the firehose, the full feed of public tweets, to a full array of partners - going beyond the initial deals with search leaders Yahoo!, Google and Microsoft. The partners announced today include personal favorites Twazzup, Collecta and Kosmix, taking Twitter more into the realm of platform rather than destination.

Twitter, through its continued rise in traffic, use and the public consciousness, has seen the data flowing through its system grow increasingly valuable, as the company aims to live up to its lofty goal as the pulse of the planet. This rich data set, now open for partners to tap into, can potentially rival traditional search engines in terms of value, especially where real-time reaction and sentiment analysis comes into play.

Kosmix, behind the personalized newspaper service, Meehive, and extended topic pages on its own search tool, wrote to say, "Having full access to the entire tweet streams will give us the chance to surface all relevant and interesting information on what is buzzing on the Web for any given topic."

Financial deals were not disclosed by Twitter, or by its partners, but the company looks to have found a way to recognize revenue and gain income off of the public stream. In this age of real-time data, the faster and more relevant, the better. That Twitter is letting more of its secret sauce spread across the Web will make it harder for other sites, namely Facebook, to look as nimble.

Disclosure: Kosmix has previously done business with Paladin Advisors Group, where I am managing director of new media.

Slideshow: Finding Signal In the Real-Time Noise

Finding Signal in the Real-Time Noise

View more presentations from Louis Gray.

With more data being created and shared in more places by more people, each of whom is more active in each network, it is no surprise that many people are being overcome by the sheer noise of social media and RSS. On Tuesday, I had the opportunity to stop by the Bay Area Real-Time Web Meetup in Oakland and talk about some strategies and tools to help cut through the noise and find the signal. From my6sense (who I am advising as part of Paladin Advisors Group) to Lazyfeed, Mozzler, Cadmus, Twazzup, and others, each can help find personalized or socially recommended content that is more relevant than the unfiltered flow.

The full deck is embedded above, via Slideshare. If you want me to participate in your event, or talk to your company, let me know.

Can You Filter for Quality News Amidst Instant Analysis?

With real-time news and distribution increasingly becoming the norm for many product launches and events, the interpreters of the news who pass their findings your way don't often have additional information that was gleaned as exclusive. As events are live streamed directly to the public, or presentations are mass broadcast to large groups simultaneously, those headlining the news are armed not with inside sources and first person quotes, but instead with immediate analysis of the events to try and put their own unique spin on history, separating themselves from their peers.

Nearly three years ago, I wrote a post titled Launching Products in the Age of Instant Analysis, highlighting the tech media's instantaneous interpretation of products like Google Finance and semi-regular introductions from Apple. You could later see tech bloggers swarm around Cuil and Microsoft's Origami platform, declaring each of them as failures, fairly accurately.

In the ensuing time, news has gotten even more rapid, and even more participants are helping to broadcast events, not just through their blogs and publications, but on Facebook, Twitter and many other social media properties. Those that attend live gatherings get an edge over their competition through speed of publishing, speed of transmission of photos and videos, higher quality videos, a better angle, or through an innovative use of a new Web service. We saw this with the recent Nexus One announcement from Google, and will inarguably see it again this Wednesday, when Steve Jobs entertains a hungry press corp looking for the latest from Apple.

Wednesday's entertainment will be much like many similar events before it, no doubt. Scads of gadget bloggers and Macintosh faithful will be there, flanked by general tech reporters, and the mainstream press. There may be a QuickTime stream, and if not, the absence of said stream will be made fairly irrelevant thanks to near-immediate live blogging, best exemplified by the work at Engadget or MacRumors Live in years past. Other sites will try to do the same, but to choose their work over the seasoned pros should only be done out of compassion by family members or significant others of the writers themselves.

Essentially, as Steve Jobs and his team flip through each successive Keynote slide, or walk through each carefully rehearsed product demo, those waiting and watching will all find out at the same time. The news you see or read through your computer screen will not be substantively less accurate or less speedily discovered than by those who were actually there. The press corp, in an effort to drive home their own relevancy and highlight their value through a key thing called "access", will drive home their recaps of the proceedings. Those driven by the quantity of posts will give each major product or feature its own story. Quotes off the cuff may be spun as insight themselves, and others will simultaneously post how each introduction will place in history, or judge its success or failure relative to the meteoric expectations they drove themselves. Subsequent stories will fawn over the first views of the product at stores, we will see glorious unboxing videos, disassembly of the parts, and estimates to total costs and product margins. Speculation will ensue as to whether the company built too many machines or not enough, if it's taking too long to reach customers, or if they priced the product too high. (They never complain that it's too cheap, after all)

All these angles and all this spin from all these places... While in the past, media properties were judged by their ability to get access to individuals and find things out that were not known by anyone else, or before they were supposed to be aired, the flattening of the media and the measurement of what some would say are the wrong metrics (be it page views, retweets or whatever your favorite statistic of the day is), have delivered a scenario by which there is more noise than ever, and as consumers, we must filter like mad.

Robert Scoble, a peer, and fellow geek and curator, said that he will be skipping out on Wednesday's news, and watching the news ink feeding frenzy as it seeps through our computer screens. His goal won't be necessarily to report on the news, which will be common knowledge as soon as he starts typing, but to find the best interpretations of the news from his favorite sources. He publicly will be doing something we all must learn to do - separating the news discovery artists from the news spin artists. With tools like Twitter and other networks making it ever easier to hit the publish button, our ability to screen, filter and decide what information is good for us is going to be increasingly tested.

Synaptic Web: Realtime is Just the Beginning

Guest Post by Chris Saad (@chrissaad | Blog)
Vice President, Strategy, Echo

I like the Synaptic Web because it's different. Unlike other projects or initiatives I've been involved with, the Synaptic Web is not an open standard, advocacy group or business. It is not even something Khris Loux, Eric Blantz or I invented. It is not even something we are advocating for or developing. It is simply something we observed and wrote down.

We wrote it down because we needed a framework from which to design, develop and refine our own product. We then shared it at SynapticWeb.org because others seemed interested in learning more about it. Since then we have continued to refine it and collect examples of other people taking our observations and expanding upon them. We hope this post is just another step in continuing the discussion here in the comments, and across the Web.

The Synaptic Web is not new. It is not a recent trend. It is something that started before the advent of HTTP and HTML gave birth to the actual Web and will continue long after the boom of 'real-time' fades away. It is a simple observation that the most interesting aspect of the Internet generally and the Web specifically is not the nodes on the network, but the type, density and flexibility of the connection between them.

We use the metaphor of the Synapse (the connection between brain cells) as a way to try to relate this idea to the real world - a literal mental model if you will, and over the years, we've observed these connections have morphed in surprising ways. Nodes have atomized (i.e. nodes are now more granular and there are more of them) and the nature of the connections are changing at an accelerating rate.

This new-found density of nodes and the rapidly changing nature of the connections between them has made the SynapticWeb easier to observe and document, not to mention more important to pay attention to.

What are the nodes?

Nodes used to be computers on the network. They then expanded to include documents on the web. Then pieces of content became nodes in their own right and now, even pieces of data - people, places, photos, videos, status updates, geo data - can be considered self contained single units.

Every single one of these things now has a canonical URI allowing us to uniquely reference them and to connect them together in new and novel ways.

What are the connections?

Connections used to be TCP/IP connections between computers. They then expanded to include hyperlinks between pages, then 'friendships' between people and now include real-time streams between all sorts of objects. Soon the implicit inferences made from those streams will increasingly become visible and viable connections in their own right.

Synaptic Web

Our premise is that while most have observed the changes in the nodes, the more interesting opportunity for innovation is to observe the change in the connections and how those changes might produce new user experiences.

In the age of social media, we asked ourselves 'what objects can become more social'. The result was vertical social networks that focused on these social objects. Obvious examples include Flickr for photos, YouTube for video, MySpace for Music and Facebook for Friends.

The fact of the matter, though, is that being social is only one kind of connection. An interesting and important one, but not the only one.

Links between 'social objects' and people are fantastic but what about the connection between you and your interests, your interests and products, products and ratings, ratings and pricing, pricing and quality, quality and availability?

These type of connections used to be made in private, carefully curated databases. In some cases there were algorithms placed on top to try to make rudimentary recommendations but the connections rarely spanned these closed systems much less allowed for curation at 'Internet Scale'. The connections were too brittle, controlled and insular.

Now, with an abundance of real-time gesture data, the growing trend to map everything to a URI, the availability of semantic technologies, the game like crowd-sourcing of tags and more, these connections are becoming far more rich and interesting.

An example of a great Synaptic application is Microsoft Photosynth.

Its predecessor, Quicktime VR, allowed one user to create a single continuous photo and then allowed viewers to scroll across that image like being inside a kind of 3D sphere.

Instead, Photosynth allows hundreds of people with thousands of photos (atomized nodes) to, acting independently, automatically generate a 3D mesh of the scene they were capturing. The connections between the photos are totally organic. The authors had no idea about this potential application when taking the photo, the photos were not arranged in a special way and the 3D mesh on which they are placed was totally generated by connecting previously disconnected nodes together and looking for the patterns.

Another very visual example is Augmented Reality. Where once we might have looked at the Yellow Pages, which only connected two types of data (zip code + business type) to create a static list of directory entries. Instead, the latest Augmented Reality app can take Location, Orientation, Directory information, Personal Interests and Business Type to layer meta data on the real world through a camera lense (typically a mobile phone).

As you can see by these two examples, just by adding just a few more nodes to the equation, and connecting those nodes more organically, the result is a dramatically different experience.

These are very real, and very current, examples of the synaptic density of the web becoming far more complex and powerful.

Real-time is perhaps the hyped aspect of the Synaptic Web. To me, the 'real-time web' is analogous to Synapsis firing in the brain. The rate, route and repetition of real-time data between nodes (usually people) has introduced countless new user experiences and content discovery opportunities.

Obviously Real-time is is just one way that connections between our nodes are changing and we have yet to scratch the surface on the possibilities.

What's Next

The goal of this post is simply to introduce the idea of the Synaptic Web and highlight the value of paying attention to the connections between nodes. There are countless ways in which this thinking might impact product design, user behavior and business models so it would be impossible to provide exhaustive coverage in one piece.

I encourage you to read the SynapticWeb strawman and continue the conversation in the comments, on your own posts, at conferences and in your own products. Who knows where it might take us!

Conclusion

So in summary, the Synaptic Web is not something that necessarily needs our support or conscious development - it is happening with or without us. It has been happening for a long time and its continuation is inevitable.

People might refer to it as different things, they might focus on various smaller aspects of the broader theme, but in the end the notion is the same - it is the connection between the nodes - and the way they are becoming more dense, more flexible, more implicit, more visible and more open - that is truly exciting and will continue to make an impact on people's lives through the Web.

I can't wait to see where it goes next.

---

Learn more about other people's take or to contribute to this theme please check out SynapticWeb.org. You can find Chris Saad at his blog, Paying Attention, and on Twitter. Chris is also Vice President, Strategy at Echo.