I/O Spotlights Google’s Machine Learning Smarts

Few will argue that this year’s I/O keynote was Google’s flashiest yet. If you were expecting people to jump out of blimps to bring you a new version of Google Glass, the event surely left you disappointed. Instead, Google used its relatively low-key keynote to announce an evolutionary update to its mobile operating system, a new effort to bring Android to the Internet of Things and a number of new tools for developers to better monetize, advertise and analyze their apps.

The three new products that stuck out for me on the consumer side of Google’s announcement, though, were Google Photos, Now On Tap in Android M, and — though it wasn’t announced in the keynote — the latest update to Inbox for Gmail. To some degree, all of them draw on the natural language understanding, deep machine learnings expertise, and immense Knowledge Graph database Google has focused on so much in recent years.

Google Photos, for example, features what is undoubtedly the best photo search engine currently available. Just do a search for ‘trees,’ ‘flowers,’ or ‘mountains’ and see what it finds in your photo collections. Photos, of course, is the stand-alone version of the photo feature of Google+, which was already able to do much of this, but the company has improved search in this version.


Google Now on Tap can understand what’s happening in an app and then give you additional contextual information and answer your questions about it. In Google’s demo, which drew quite a bit of applause from the audience, the presenter brought up a Skrillex song in Spotify (and oddly enough, not Google Music), launched Now on Tap and then asked “what’s his real name?” To do this, Google has to understand what’s playing in the app, understand who the ‘his’ refers to, and then go deep into its knowledge base to give you the answer.


Inbox, which somehow got left out of the keynote , now relies on some of this knowledge and natural language understanding to automatically generate reminders for you when it notices somebody is asking you to do something. It can also recognize emails about upcoming trips and bundle them together into a single group.

Microsoft is starting to do a bit of this with Cortana now and I’m sure Apple is thinking about similar tools, but at the end of the day, Google’s products feel like they are quite a bit ahead of the curve. Nobody else, after all, can draw on a project like Knowledge Graph with data about more than a billion of entities. And for better or worse, nobody knows as much about you and your online (and maybe even offline) habits than Google, so it can put all of this info into context for you.

Google I/O’s keynote may have lacked any real surprises (though the ATAP keynote the day after made more than up for that), but these three features alone show that Google is far ahead of its competitors when it comes to understanding its users.