METROMy brother held his phone to my ear. “This will be creepy for you,” he warned. A instagram reel which showed a teenager at a rally featured a news show-style voiceover. A quiet female voice, with an almost imperceptible Mancunian accent, said: “The recent protest by a British student has become a powerful symbol of a deepening crisis in the UK education system.” I sat up straight, eyes wide open.
As a presenter for a YouTube news channel, I was used to hearing my voice on the screen. Only this wasn’t me, even if the voice was unquestionably mine. “They are forcing us to learn about Islam and Muhammad in school,” he continued. “Listen. “This is disgusting.” It was chilling to hear my voice associated with far-right propaganda, but more than that, as I dug deeper into how this scam is perpetrated, I discovered how far-reaching the consequences of fake audio can be.
AI voice cloning is an emerging form of deepfake audio and the third fastest growing 2024 scam. Unwitting victims have their voices played by experts without their consent or even knowledge, and the phenomenon has already led to security checks at banks. omitted and the people disappointed to send money to strangers they thought were family. A friend who had recognized my voice had sent the clip to my brother.
After some digging, I was able to trace it to a far-right YouTube channel with around 200,000 subscribers. It was supposedly an American channel, but many of the spelling errors in the videos were typical of non-native English-speaking disinformation accounts. I was horrified to discover that eight of the channel’s 12 most recent videos had used my voice. Going back even further, I found a video from five months ago that used my voice and showed a 10m view count. The voice sounded almost exactly like mine. Except there was a slightly strange rhythm to my speech, a sign that the voice was generated by AI.
This increasing sophistication of AI voice cloning software is of great concern. In November 2023, a deepfake audio of London Mayor Sadiq Khan allegedly making inflammatory comments about Armistice Day was widely circulated on social media. The clip almost caused “serious disorder,” Khan told the BBC. “The timing couldn’t have been better if the goal is to sow discord and cause problems.” At a time when trust in the UK political system is already at record levels lowWith 58% of Britons saying they “almost never” trust politicians to tell the truth, being able to manipulate public rhetoric has never been more damaging.
The legal right to have one’s voice lies within a murky gray area of poorly legislated AI issues. TV naturalist David Attenborough was at the center of an AI voice cloning scandal in November: he described himself as “deeply disturbed” to discover his voice was being used to deliver American partisan news bulletins; In May, actress Scarlett Johansson clashed with OpenAI after a text-to-speech model of its product, ChatGPT, used a voice that Johansson described as “eerily similar” to her own.
In March 2024, OpenAI delayed the launch of a new voice cloning tool, deeming it “too risky” for general release in a year with a record number of global elections. Some artificial intelligence startups that allow users to clone their own voice have introduced a policy of caution, allowing them to detect the creation of voice clones that imitate political figures actively involved in election campaigns, starting with those in the United States. and the United Kingdom.
But these mitigation measures are not enough. In the United States, concerned senators have proposed a bill that would crack down on those who play audio without consent. In Europe, the European Identity Theft Observatory System (eitos) is developing four tools to help police identify deepfakes, which they hope will be ready this year. But addressing our audio crisis will not be an easy task. Dr Dominic Lees, an expert on AI in film and television who advises a UK parliamentary committee, told The Guardian: “Our privacy and copyright laws are not up to date with what this new technology presents.”
If one problem is the decline in trust in institutions, another is the growing distrust between communities. The ability to trust is fundamental to human cooperation in our increasingly globalized and intertwined personal and professional lives; However, we have never been so close to undermining it. Hany Farid, a professor of digital forensics at the University of California at Berkeley and an expert in deepfake detection, told the Washington Post that the consequences of this audio crisis could be as extreme as mass violence or the “theft of elections.”
Could there be any advantage to this new ability to easily clone voices? Maybe. AI voice clones could allow us to seek comfort by connecting with the deceased loved onesor help give a voice to those with medical conditions. The American actor Val Kilmerwho has received treatment for throat cancer, returned in 2022 for Top Gun: Maverick with his voice restored by AI. Our ability to innovate can serve those with nefarious goals, but it also serves those who work for good.
While I willingly shared my voice on screen when I became a presenter, I did not agree to give up this integral and precious part of me to anyone who wants to use it. As broadcasters, we sometimes worry about how a cold or winter virus might affect our recordings. But my recent experience has given another, much more sinister meaning to the concept of losing your voice.