Google today is releasing a more capable version of Bard, its generative AI chatbot and ChatGPT rival, which now lets you double-check its answers, collaborate with others and, notably, integrate with Google’s own apps and services, including Gmail, Docs, Drive, Maps, YouTube and Google Flights and hotels.
The latter is available through new Bard Extensions and only in English for the time being. First announced at Google I/O, the company had not immediately rolled out extensions because it wanted to make sure it did so in a way that would offer a safe and trustworthy experience.
“We wanted to make sure that the way that we bring this to users is extremely rooted in the three principles that we have, as it relates to the trust that we build with our with people that use Bard, which is around transparency, choice and control,” explains Jack Krawczyk, product lead for Bard. “So we’re going to start off with saying when Bard interacts with Gmail, Drive and Docs, it’s only when a user has opted in to say it’s okay,” he says. And the user can revoke that permission at any time.
In addition, the company wants to ensure users understand how that data is and is not used. If you’re using personal data that’s been brought in from Gmail, Google Drive or Docs, that information is not used for reinforcement learning. Says Krawczyk, that’s a critical element in order to maintain user trust.
He notes that opting in to use Gmail with Bard isn’t providing Bard with the ability to store your entire Gmail inbox. Instead, on a per-prompt basis, it can be directed to find information in your inbox by using its ability to generate a call to Gmail to find something you’ve asked for. In addition to not being used for reinforcement learning, Google says no human reviewers will see the email Bard accesses either.
“It’s similar to how we’ve approached spam filtering in Google services in the past — your personal information isn’t read, because we believe that that trust is the most critical pillar upon which we build,” Krawczyk explains. “It’s early, and you lose some precision, some of the broader capabilities because you don’t have that. But we think it’s a long arc to make this technology helpful. And we would rather do it right. Do it from the position of building trust from the beginning.”
Once the extensions are connected, however, you could ask Bard to read your important emails and summarize what you missed. But there may be areas where it falls short, as the feature is further developed. For instance, it wouldn’t be capable of finding your wine club memberships and their next delivery dates if they don’t specifically say “wine club” in the email somewhere.
The new extensions can also work together. For instance, if you were planning a trip with friends, Bard could retrieve the dates from your Gmail thread that worked best for everyone and then look up real-time flight and hotel information, get Maps directions to the airport and even surface YouTube videos of things to do at the destination — all in the same conversation.
Image Credits: Google
For the extensions that don’t leverage personal data — YouTube, Flights, Hotels and Maps — you’re opted in automatically but you can choose to opt-out. The company says it eventually wants to support third-party services through this same Extensions model, but wants to first test and learn from the feature using its own first-party apps and services.
Another new feature updates the “Google it” button in Bard to double-check the chatbot’s response — an improvement that Google says taps into work from Google Research and DeepMind. When you tap the “G” icon, every sentence that Bard has written is validated against Google search to see if there’s content on the web to substantiate the answer. When the statement is evaluated, you can click on the highlighted phrases to learn more through Google Search.
Image Credits: Google
But if the AI is unsure, the sentence may be highlighted in orange to indicate that it knows this part of the answer might be wrong. This should help users better understand when the AI is “hallucinating” — that is, when it provides a response based on false information. This is a problem with modern AI that may confidently generate output even when it doesn’t have the supporting data.
“We are pretty excited about taking this step toward building trust with language models,” notes Krawczyk. “Of course, we certainly want to be transparent when we’re not confident or even when we make a mistake,” he says. The feature will also help the AI to improve as it learns what it gets wrong from user feedback and then uses that to create a better model.
A third update allows Bard users to collaborate with one another. Now, when someone else shares a Bard chat with you through a public link, you’ll be able to continue the conversation and ask Bard additional questions about that same topic. You can also use this as a starting point for your own ideas, says Google.
Image Credits: Google
Along with these new releases, Google is expanding access to Bard’s existing English language features, including the ability to upload images with Google Lens, get Search images in responses and modify Bard’s responses — to more than 40 new languages.