Airbnb leans on reviews to make listings more reliable as it tests review summaries using generative AI

Airbnb has given hotels a run for their money with offerings that are often more flexible, sometimes cheaper, and more characterful than their traditional hospitality counterparts. One area where Airbnb has stumbled, however, is consistency: Hotels are still considered to have better service and customers still regularly complain that Airbnb listings are not accurate.

Now, to try to bridge those gaps — and at the very least to better manage everyone’s expectations — today Airbnb is unveiling an overhaul of guest reviews. Alongside that, Airbnb is adding two more ways for hosts to help surface more nuance about their properties, by way of a new top-rated badge and tools to communicate more information about properties.

The updates are coming as part of the company’s latest winter release, and as we talked about those with Airbnb’s Brian Chesky, the CEO and co-founder also revealed some of its future roadmap, including tests it’s running using generative AI to write review summaries.

Overall, Chesky said the company’s main priority was to bring more predictability to Airbnb, since the reason people often stay in hotels is because they know what they’re going to get in terms of service.

“Our homes are one of a kind — that’s a strength [for us], but their reliability is the big question,” he said in an interview. “We can’t inspect every home, and we don’t want to try to inspect every home. So we’re just going to be investing a lot more into a review and reputation system. That’s why we’re upgrading reviews, and we’re going to be doing a lot more stuff like that.”

Airbnb ratings

Image Credits: Airbnb

The redesigned reviews page will feature a carousel at the top that will display overall and category-specific ratings. Users can now also sort reviews by most recent or highest rated.

The new badge being rolled out is a signal of how Airbnb is trying to bake in customer service incidents and customer feedback to understand if the property is worth a shout-out. Called “Guest Favorites,” the badge will be used to designate highly rated properties. Currently, the properties with this label have an average rating of 4.9, less than a 1% cancelation rate, and a less than 1% incident rate.

The company didn’t specify the exact criteria by which it picked the 2 million “Guest Favorites” property in the first batch. But Airbnb said that it considered “signals that have the loudest guest voice.” such as reviews and cancelation rates.

You can think of Guest Favorites as the property equivalent of the “Superhost” badge that hosts can earn by maintaining a 4.8 overall rating across their properties; less than 1% cancelation rate; and at least 10 short-term stays or 100 nights stayed in the past year. “Guest Favorites” will be more property-specific and subject to changing more frequently: They are updated weekly, while the “Superhost” badge is evaluated quarterly.

Whether customers ever understand the distinction between those two badges remains to be seen. But ultimately, there are other reasons to have launched Guest Favorites: The more frequent updating makes hosts more accountable, and the information gives more transparency to would-be guests. It also gives Airbnb another way to understand its customer base, and also surface properties based on the “Guest Favorites” filter.

Guest Favorites

Image Credits: Airbnb

Still, the five-star rating system, popularized by hotels — ironically, the very industry Airbnb aims to disrupt — is far from perfect, something Chesky admitted.

For starters, there is a lot of opportunity for gaming right now. A property with a 4.6-star rating on Airbnb, for example, is actually at the bottom 20% of all listings. In 2022, the U.S. average review score was 4.75, and it was just above 4.6 in the rest of the world, according to short-term rental analytics firm AirDNA. In other words, given that many of those “lower” rated properties are not actually that nice, this undermines the whole idea of the ratings in the first place.

Bram Gallagher, an economist at AirDNA, said that heterogeneity is a big challenge for companies like Airbnb, but it’s also a big draw for customers.

“I don’t think Airbnb wants to commodify listings by making them uniform. A move towards providing more details would relieve some of the uncertainty in consumers that might be preventing them from going to a short-term rental,” he said.

On the other side, though, Airbnb is stuck between a rock and a hard place in trying to improve this.

“We didn’t want to blow up the five-star rating system because we have more than 317 million reviews and two out of three people leave a review on our platform,” Chesky explained. “[So] we will try to create more context on the rating with labels like guest favorites.”

Airbnb EU review scores by price tier

Image Credits: AirDNA

On the host end of things, Airbnb is adding more tools to keep them sweet. An easily accessible “Listings” tab on the home page has tools to add amenities, as well as an AI-powered photo tour to arrange images uploaded by the host by different rooms automatically.

Airbnb listings

Image Credits: Airbnb

Investing in AI and future roadmap

Airbnb has introduced a few AI-powered features in this release, but Chesky said that the plan is to add more. Top of the list, he said, is a test using generative AI to produce review summaries. He noted that those summaries can also take into account prior info such as customer service contacts.

He also mentioned that generative AI can improve areas such as search, identifying amenities of a property from photos and customer service. In particular, it can help with language translation between a host and guest who are fluent in two different languages. Additionally, AI can help in quickly parsing the policy pages to understand what might be relevant for an issue — both for users and customer service agents.

“Imagine a Japanese guest staying with a German host. Neither of them speaks the same language and they are calling a call center, and very few people speak both languages. Agents are trying to deal with a resolution within minutes while dealing with many policies [applicable to the situation] that could be like very many pages long. So I think what AI can do just generally, is it can reimagine our service reimagined customer service,” Chesky said.

In a set of interviews in the last few months, Chesky said that Airbnb is trying to overhaul its core services before focusing on other things. He told TechCrunch that there might be one or two releases focusing on improving affordability, reliability and customer service.

Airbnb registered $3.4 billion in revenues for Q3 2023, with an 18% growth year-on-year. However, the company presented cautious guidance saying that it is seeing “greater volatility early in Q4.”

In a note to investors, Bernstein said that the firm adjusted its night booking guidance for the financial year 2023 from 455 million to 449 million because of the guidance.

November 9, 7 AM PT: The story has been updated to clarify that Airbnb hasn’t publically defined the exact criteria for properties to get a “Guest Favorite” label. The numbers released by the company were reflective of average ratings and cancelation rates for the initial batch of properties with the “Guest Favorite” label.