CrunchUp Live: Real Time Search Panel

img_01802We’re here at the Real-Time CrunchUp, listening to the Real Time Search Panel featuring:

Edo Segal, Futurity Ventures
Sean Suchter, Microsoft
Danny Sullivan, Search Engine Land
Vipul Ved Prakash, Topsy
Kimbal Musk, OneRiot
Matt Cutts, Google
Gerry Campbell, Collecta
Erick Schonfeld, TechCrunch (Moderator)

Follow my live notes below (paraphrased). You can also watch the live video stream here.

ES: The NSA has been doing real-time search for decades. I want to take the last 5 minutes of everything published on the planet and collect it, that’s feasible. It’s like a consciousness, not memory.

Erick: What is real-time search right now? Is it just Twitter, or is it everything on the Internet that is just recent.

KM: Traditional search is like going to a library. And Google are the best librarians. But real-time is the right search, right now.

DS: I don’t think it is all Twitter, it’s just that Twitter is the leader. What’s interesting is that real-time search engines are all different — unlike say a Google image search, and Bing image search. We still really need to define real-time search. Twitter is basically real-time search right now because that’s where all our content is right now.

Erick: The second you put a filter on it becomes less real time, right?

GC: Sometimes you just want highlights from the game, sometimes you want the whole game. Filtering doesn’t really slow things down. At Collecta we’re still holding back about 90% of the content because we’re trying to figure out what is useful to people.

KM: Drinking from the firehose is a ticking timebomb. Even filtering 90% of the Iran election is a tiny slice that’s useful. And in the future, it will be even more spam. That’s why you need to filter, too much spam.

MC: As you saw at Searchology a few weeks ago, Google is interested in real-time. Back when I joined Google in 2000, we were only updating our index every few months. By 2003 it was every few days. By 2007, it was 10-15 minutes. We’re still working in that direction, but it’s about getting the right mix.

Erick: What about Marissa Mayer and Larry Page talking so much about real-time?

MC: We’re doing a lot of exciting things in that area.

Erick: What about Microsoft?

SS: We’re definitely interested in real-time. You saw what we are doing already with pulling in tweets.

ES: Right now Twitter is the only game in town.

VVP: Twitter feed is one of the largest and interesting sources of information, but a large part of the web is moving into this real-time world.

Erick: How much data are you seeing from Twitter alone?

KM: 20% of our data. We have a panel we use to monitor some users. Our own data is 50%. And other comes in from Digg and some other sources. If you just stick with Twitter it’s biased data, and spam.

GC: Twitter is all about the head too, it’s hard to use it to get the tail.

Erick: More about spam on Twitter?

MC: On the web, spam is anonymous, but Twitter tries to put an identity. It’s interesting to have been doing this for 8 years to see the same things happening now, but it’s going faster now.

Q&A

Q: How are we going to get the search data out of Facebook?

ES: It’s happening already, just slowly. Twitter is forcing their hand a bit.

SS: There’s a big promise in what Facebook is trying to do with making status updates public.

ES: It’s a matter of months away. And it’s a big threat to Twitter.

Q: What about making the real-time results more relevant?

MC: I agree. The debate on date vs. authority will be solved soon. Date only is a recipe for disaster.

GC: Rank and filter are not exclusive. It’s all about deciding which content to show, no matter what the filter is.

KM: At OneRiot we do Pulse Rank, with a ranking done at search time.

ES: We are struggling to figure out where we are. But where we’re going is important. One has to assume that the ways people use real-time search will be different from using a search box. Most of the game is on our phones, because we’re connected all the time. The next buzz is augmented reality. We’ll be doing it in two years from now in a conference like this.

End of the panel.