Google Assistant is getting an AI-powered update. At today’s Made By Google live event, the company introduced Assistant with Bard, a new version of its popular mobile personal assistant that’s now powered by generative AI technologies. Essentially a combination of Google Assistant and Bard for mobile devices, the new assistant will be able to handle a broader range of questions and tasks, ranging from simple requests like “what’s the weather?,” “set an alarm” or “text Jenny,” as before, to now more intelligent responses provided by Google’s Bard AI.
This includes being able to dive into your own Google apps, like Gmail and Google Drive, to offer personalized responses to queries on an opt-in basis. That means you could do things like ask Google Assistant questions like “catch me up on my important emails I’ve missed this week,” and the digital helper can dig up emails you need to know about.
This feature builds on the update Bard released in mid-September, which allows the AI chatbot and ChatGPT rival the ability to integrate with Google’s own apps and services, including Gmail, Docs, Drive, Maps, YouTube and Google Flights and hotels, through “Bard extensions.” Users who have already opted in to allow Bard to access their Gmail, Drive and Docs won’t have to do so again when using the feature in Assistant, but those who haven’t yet tried extensions would need to give Bard permission before it could respond to those personal queries in the Assistant app.
In addition to finding things in your inbox, Google suggests the expanded capabilities could be used for personal tasks, like trip planning, creating a grocery list or writing a caption for social media, for example. With the launch of the new experiment, Google aims to examine how people use Assistant with Bard before launching the functionality broadly to the general public across Android and iOS.
And because Bard is now on mobile devices, users can interact with it in a variety of ways.
“It can hear through the microphone. It can speak to you through voice output. It can see through your camera. And it can even take actions to help you out,” explains Sissie Hsiao, vice president of Google Bard and Assistant, in an interview about the new functionality. “And of course it’s on the device that you have with you at all times, which is your phone,” she says.
The exec positions the expansion as a major leap for the company’s digital assistant, which has before been limited to more basic tasks.
Image Credits: Google
“Google Assistant, over the past seven years, has been helping hundreds of millions of people get things done through natural and conversational methods. So things like setting alarms, asking for weather, or making quick calls using a simple ‘Hey, Google.’ And now with generative AI coming there’s new opportunities to deliver an even more intelligent, more personalized, more intuitive digital assistant. And we think it should extend beyond voice.”
In fact, users can interact with Google Assistant with Bard in three ways. They can ask it questions and follow-ups with their voice, type in their queries, or they can leverage the camera through Bard’s Google Lens integration. The latter allows users to take or upload pictures to accompany their queries.
Hsiao says people have been using this feature in a number of unique ways — like taking pictures of their clothes and shoes and asking Bard how to style them, taking pictures of apps and asking Bard to write the code scaffolding.
“We want Bard to be multimodal,” she explains. “It can see. It can hear. It can speak to you.”
In addition, on Pixel devices and select Samsung phones, you can long-press on the power or home button, respectively, to bring up a pop-up, floating window that offers a conversational overlay on the page you’re viewing, allowing Bard to respond to what you’re seeing on the screen. For example, you could pull up Bard over a picture of a hotel and ask it if the hotel is available to book this weekend.
Image Credits: Google
With Bard’s integration into Google Assistant, it’s not limited in any way from the web version — which means it can now also double-check your answers if there’s concern about AI hallucinations — a problem modern AIs face as they construct incorrect answers based on false information. That feature was also rolled out in mid-September.
Google says Assistant with Bard will initially launch in a limited set of markets — and not only English-speaking ones. It hasn’t yet determined which markets or languages will be first to receive the update, however. In the coming months, it will roll out more broadly to iOS and Android mobile users before exploring the possibility of bringing the upgraded Assistant functionality to other platforms.
