They thought there was something strange in his voice. It was not unknown but, after a while, it began to spread everywhere.
Science presenter Liz Bonnin’s accent, as regular BBC viewers know, is Irish. But this voicemail, apparently granting permission to use her image in an advertising campaign, seemed to place her on the other side of the world.
It turns out that the message was fake: generated by AI to imitate Bonnin’s voice. Her management team found out about this after seeing the presenter’s face in online ads for an insect repellent spray this week, something she did not sign up for.
“At first she sounds like me, but then I sound a bit Australian and by the end she’s definitely an English woman. It’s all fragmented and there’s no cadence to it,” said Bonnin, best known for hosting Bang Goes The Theory and Our Changing Planet.
“It feels like a violation and it’s not a nice thing,” he added. “Thank goodness it was just insect repellent spray and it wasn’t supposedly advertising something really horrible!”
Howard Carter, CEO of Incognito, the company behind the failed campaign, claims that someone he thought was Bonnin sent him several voicemails. He said these voicemails “confirmed” to him that he was really talking to her.
He had previously sought her endorsement before being approached by a Facebook profile adopting Bonnin’s identity. She claims messages exchanged between the two led him to believe she was real despite thinking the profile was “a little suspicious.”
The person who assumed Bonnin’s identity gave Carter a phone number and email address. They also provided him with contact details for someone posing as the Wildlife Trusts, the charity where Bonnin serves as chairman. He said the deal was negotiated through WhatsApp and emails. He also claims that he spoke on the phone with one of the scammers posing as Bonnin on at least one occasion.
On March 13, Carter received a contract via email that he believed was electronically signed by Bonnin. The company sent £20,000 to an account linked to a digital bank on March 15, bank statements show. Five days later images of Bonnin were sent for use in the campaign.
Other emails sent by Incognito went unanswered. The campaign launched on Monday using quotes and images sent by scammers. Hours later, Bonnin said on X that he did not agree to any endorsement with the company.
The person posing as Bonnin texted Carter on Tuesday to apologize and said he was not managing his social media and also cited security and server issues. At this point, Carter realized that he had been tricked.
“I am very sorry for what the company has gone through. It’s not fun for them, but it’s a violation on both of their parts. It’s a reminder that if it seems too good to be true and too easy, or a little strange, you need to triple or quadruple check,” Bonnin said.
He added: “If someone contacts you and says, ‘Hey, let’s not go the professional route,’ beware!”
Carter claims she didn’t get the deal signed through Bonnin’s management agency because the person posing as her said “she was doing us a favor, as long as we do it directly with her and don’t involve her main agency.” “.
In a voice note heard by The Guardian, the AI-generated Bonnin says a series of stilted sentences, such as “I must thank you for providing clarity on the direction you envision for our collaboration.”
Two artificial intelligence experts who evaluated the voice note agreed that it was likely to have been artificially generated. Surya Koppisetti, Senior Applied Scientist at an Image Sensing Startup Defender of reality said: “There are gaps and recitation speed issues that are consistent with AI-generated speech. The accent of the dialogue is inconsistent.” Koppisetti added that the speech quality was “unusually clear despite the noise.”
Michael Keeling, senior data scientist at AI technology company Faculty, said that the “constant and monotonous” artificial background noise in the message “is a classic way to make something seem more realistic.” “If you’re listening to this on your phone on a busy street, it’s a lot easier to escape that way,” she added.
Bonnin said the incident is a “warning message” about the potential dangers of AI. “There are many ways AI can be used to benefit society; We have heard that it can be used to identify cancers, but it is not sufficiently regulated either. The technology will get better and more sophisticated,” she added.
Incognito said they reported the incident to the police and their bank. In a statement they said: “We hope [our CEO] Falling for this elaborate scam will be a warning to others not to fall for the same or similar tricks. Sophisticated criminals using artificial intelligence and other computer-generated communications have targeted Howard and Incognito several times.
“It is understandable that many companies do not want the public to know that they have been deceived due to shame and embarrassment. As an ethically and transparently run company, we believe it is our duty to alert people and their businesses to the rise of sneaky scams like this. Howard and all other members of our organization apologize to Liz Bonnin and her associates for any harm we may have inadvertently caused.”
The BBC did not respond to a request for comment. The Wildlife Trusts declined to comment.