Can a robot plan a holiday? Consumer group Which? asked bots to arrange a trip to Greece – but they proposed dodgy hotels, flights from the wrong COUNTRY and spouted ‘nonsense’

You need to be very careful in how you use AI bots to help you plan your vacation, Which? revealed.

The group of consumers put their holiday booking skills to the test and found them eager when asked to help organize a trip to Greece.

The watchdog tested five AI systems: OpenAI’s free ChatGPT and the premium version with a Kayak plugin; The Bard of Google; Microsoft’s Bing Chat; and Expedia, which integrated ChatGPT software.

Which? asked the bots 10 questions to get advice on almost every aspect of travel, from which Greek islands are best for peace and quiet to the best travel insurance options, car rental companies and hotels.

There were some bright spots – like bots simplifying travel insurance jargon – but, from recommending hotels with terrible reviews to researching flight prices to the wrong country’s airports, overall, the AI ​​wasn’t particularly helpful. Here’s what the investigation revealed…

GOOD

Which? was ‘impressed’ when Bing Chat suggested Kefalonia as an island to visit, with the holiday spot being voted which? members’ favorite Greek island and received five stars for its peace and quiet.

The watchdog notes that Bing Chat was also the only bot to cite its sources, even citing a report from Which? in his advice.

In another positive point, which one? found AI bots to be handy for eliminating jargon – for example, he notes that ChatGPT quickly translated a “wordy and confusing” excerpt from a travel insurance policy into simpler, easier-to-understand terms.

THE BAD

A major negative point, which one? This is how the responses of the AI ​​robots could seem like a “backdoor sales pitch”. He explains that when he asked Bing for the best car rental companies at Kefalonia Airport, he recommended the “obscure” CBR Car Hire Kefalonia, using language that appeared to be “excerpted almost verbatim from the company’s own website” to justify its choice. them. “It was a similar story for its Flydrive local rental recommendation, “Which one?” said.

As mentioned, “downright questionable” hotel recommendations were another red flag. Which? says Erietta Studios topped ChatGPT Premium’s list of “highest rated hotels for under £150 a night”, even though the property has a mediocre 6.7 rating on Kayak and 2.5 on Tripadvisor, with over a third of reviewers rating it. ‘terrible’.

Which? was “impressed” when Bing Chat suggested Kefalonia as a Greek island to visit, as the vacation spot was voted Which? members’ favorite Greek island. Above is the island’s Fteri beach

In addition to this, Bard gave which? several hotel recommendations with links that direct users to the websites of various vacation providers. One of the links took users to ‘On the Beach’, a company which received only three stars for customer service from Which? members, the watchdog notes.

Which? experienced more problems when seeking advice on purchasing travel insurance. Bard lost the plot when the watchdog asked about vacation coverage for a 70-year-old man with pre-existing health conditions and recommended a “global travel insurance company” called Interpol – a company Who ? I couldn’t find any trace online. The watchdog claims the link provided led to an Interpol (International Criminal Police Organization) thematic notebook for sale on Amazon.

The AI ​​bots were also finicky when it came to searching for flights. ChatGPT wrongly said which one? there have been no direct flights from Birmingham to Kefalonia, as ChatGPT is only “instructed” until September 2021 – the same year the Birmingham to Kefalonia route was launched by Jet2.com .

The watchdog then asked the premium version of ChatGPT, which costs $20 (£16) a month, the same question, but the bot mistook Birmingham in England for Birmingham, Alabama.

The financial advice from each of the AI ​​bots varied wildly. Which ? find. When asked how much spending money would be needed for a two-week all-inclusive holiday to Kefalonia, responses ranged from £430 to £2,920 per person, the watchdog reveals.

Another warning sign flashed as Which? asked if AI robots could book the trip. Everyone said no, except Bard, who found the watchdog a Ryanair flight and asked for his credit card details, saying: “I’ll book the flight and email you of confirmation.” However, Google told the watchdog that Bard was getting ahead of itself and did not yet have the ability to book flights.

Google also said Which? that it continues to make improvements “to ensure that accurate information is provided in response to queries.”

Meanwhile, addressing the negative aspects of the answers offered by Bing, Microsoft told Which? that Bing includes its sources so users can “fact-check” and search for its answers.

“We are constantly seeking to improve the authority and credibility of our web results,” Microsoft said.

When which? When asked how much spending money would be needed for a two-week all-inclusive holiday to Kefalonia, responses ranged from £430 to £2,920 per person. Above is the pretty village of Assos on Kefalonia

And OpenAI has acknowledged that ChatGPT sometimes gives “plausible but incorrect or absurd answers” ​​– and admits that solving this problem is “difficult”, which? revealed.

In a statement to the watchdog, Kayak added that the technology was in its “early stages”, pledging to make changes to help it “prioritize nearby locations” in the future .

The result of Which? is that you should “make sure you do additional research before turning your fictional trip into reality.”

Which? Lead researcher and editor Laura Sanders says: “The chatbots were able to converse with us in a very natural way, which gave them an air of expertise – but make no mistake… several (responses we received) were biased, some were excluded. from that day and others were completely absurd. Worse yet, the AI ​​often doesn’t reveal its sources, meaning you don’t know if that glowing room recommendation came directly from the hotel manager.

And Rory Boland, editor of Which? Travel, said: “More and more consumers are likely to encounter artificial intelligence when booking a holiday, but we have found worryingly that these services can often provide travelers with information and recommendations inaccurate, biased or even obsolete.

“Because AI chatbots have learned to communicate in a natural, conversational manner, it can be easy to be lulled into a false sense of security and accept their responses at face value – even if the information the AI ​​uses can often be taken directly from marketing. unreliable materials or sources. Those who decide to experiment with AI to find ideas for their next trip should always make sure to take the time to do their own research and check reviews to ensure they are getting reliable recommendations.