The recent victory of IBM’s Watson computer against human competitors in an exhibition round of Jeopardy got computer scientist Stephen Wolfram thinking about how regular search engines might fare in such a match-up. So he took 200,000 known Jeapardy clues and ran them through six search engines (Google, Bing, Ask, Blekko, Wikipedia Search, and Yandex). He excluded known Jeopardy sites from the results, and didn’t test his own Wolfram Alpha because it is not designed for those kinds of queries.
What he found is that the search engines did fairly well, depending on how you measure success. Google did slightly better than the rest, but Bing and Ask were close behind. On average, Google got the correct answer somewhere on its first results page 69 percent of the time, versus 68 percent for Ask and 63 percent for Bing. Google got the right answer somewhere in the title or snippet of text of the very top result 66 percent of the time, versus 65 percent for Bing (and Ask dropped to 51 percent).
In comparison, most humans answer 60 percent of Jeopardy clues correctly, while the top player of all time, Ken Jennings, answered 79 percent correctly. So it is conceivable that a system could be created using regular search engines that could beat most humans. But Wolfram cautions:
Of course, the approach here isn’t really solving the complete Jeopardy problem: it’s only giving pages on which the answer should appear, not giving specific actual answers. One can try various simple strategies for going further. Like getting the answer from the title of the first hit—which with the top search engines actually does succeed about 20% of the time.
Answering Jeopardy clues correctly and consistently is a hard problem for computers to solve because of all the variations and nuances of human language. Yet “just using a plain old search engine gets surprisingly far,” concludes Wolfram.
Stephen Wolfram is a distinguished scientist, inventor, author, and business leader. He is the creator of Mathematica, the author of A New Kind of Science, and the founder and CEO of Wolfram Research. His career has been characterized by a sequence of highly original and significant achievements. Born in London in 1959, Wolfram was educated at Eton, Oxford, and Caltech. He published his first scientific paper at the age of 15, and had received his PhD in theoretical physics from...
IBM, acronym for International Business Machines, is a multinational computer technology and consulting corporation. The company is one of the few information technology companies with a continuous history dating back to the 19th century. IBM manufactures and sells computer hardware and software, and offers infrastructure services, hosting services, and consulting services in areas ranging from mainframe computers to nanotechnology…
Google provides search and advertising services, which together aim to organize and monetize the world’s information. In addition to its dominant search engine, it offers a plethora of online tools and platforms including: Gmail, Maps, YouTube, and Google+, the company’s extension into the social space. Most of its Web-based products are free, funded by Google’s highly integrated online advertising platforms AdWords and AdSense. Google promotes the idea that advertising should be highly targeted and relevant to users thus providing...
Bing is a decision (search) engine from Microsoft officially announced on May 28, 2009. It combines technology from the Farecast and Powerset acquisitions, as well as new algorithms and a more colorful page design, to attempt to understand the context behind the search, which Microsoft claims gives users better results. Bing as a brand is also an attempt to eliminate the confusion caused by Microsoft’s “Windows Live” branding. Bing is now everything “search” related, whereas Windows Live encompasses the remnants...
Ask.com is a top 10 US website and digital brand (as ranked by Nielsen) with more than 100 million monthly users globally. Founded in 1996, Ask.com was originally known as Ask Jeeves and specialized in natural language search. In Sept 2001, the company acquired Teoma Technologies, and began shifting Ask’s algorithms away from natural language search. AskJeeves was acquired by IAC in March 2005 for $1.85 Billion and renamed Ask.com in 2006. In July 2010, Ask.com returned to its Q&A...
Austin, TX
Seattle, WA
San Diego, CA
Menlo Park, CA
Boston, MA
Disrupt Europe: Berlin Hackathon
Berlin, Germany
