Two Google scientists presented a paper (pdf embedded below) at the World Wide Web Conference in Beijing last week that outlines their vision for the future of image search.
Notably, the new image search technology doesn’t just index text associated with an image in determining what’s in it. Google is now talking about using computers to analyze the stuff in photos, and using that to associate it in a ranked way with keyword queries. In effect, they’re talking about something similar to PageRank for images (but without the linking behavior).
Today when we talk about search all we really mean is text search. That’s sort of like only being able to see in one color. And when we search for image, video and audio content, the only data that search engines use to do those searches is the text that is associated with those files. That’s like trying to describe the color green when you can only see in red.
To date Google and others have spent a significant amount of effort on making the metadata around rich content better. One example of this is Google’s Image Labeler game that uses human labor to properly tag images. Innovative, yes. But it’s still trying to “describe green” when you you can’t actually see it.
Once computers are able to analyze rich content as easily as they can analyze text, a whole new dimension to search will emerge. Humans will no longer be needed to do the heavy lifting in describing what is included in rich content, and that means that content will no longer be invisible on the Internet.
PageRank For Image Search
Googlers Yushi Jing and Shumeet Baluja argue that Google is now ready to see beyond text. In their paper they talk about their efforts to apply state of the art image recognition software to figure out what stuff is in an image. “Commercial search-engines often solely rely on the text clues of the pages in which images are embedded to rank images, and often entirely ignore the content of the images themselves as a ranking signal,” they say. Their experiments in actually digging into the images themselves “show significant improvement, in terms of user satisfaction and relevancy, in comparison to the most recent Google Image Search results.”
Google is looking at the visual characteristics of popular images and then determining rank based on similarities between images. In the figure below, the largest two images contain the highest rank.
http://www.docstoc.com/docs/wrapper.ashx?doc_id=529160&swf_url=http%3A//content1.docstoc.com.s3.amazonaws.com/p307-jingA.pdf.swf&showrelated=0&showotherdocs=0&showstats=0&enableFullScreen=1
PageRank for Product Image Search – Get more documents