Home Tech ‘Just the beginning’: X’s new AI software fuels racist abuse online, experts warn

‘Just the beginning’: X’s new AI software fuels racist abuse online, experts warn

0 comments
'Just the beginning': X's new AI software fuels racist abuse online, experts warn

A rise in online racism fueled by fake images is “just the start of a looming problem” after the latest release of X’s artificial intelligence software, online abuse experts have warned.

Concerns arose after computer-generated images created using Grok, X’s generative AI chatbot, flooded the social media site in December last year.

Signify, an organization that works with prominent sports groups and clubs to track and report online hate, said it has seen an increase in reports of abuse since Grok’s last update and believes the introduction of photorealistic AI will much more frequent.

“It’s a problem now, but it’s really just the beginning of a problem to come. It’s going to get a lot worse and we’re just at the beginning; I expect that in the next 12 months it will become incredibly serious.”

Grok was launched in 2023 by Elon Musk and recently got a new text-to-image feature called Aurora, which created photorealistic AI images based on simple prompts typed by the user.

An earlier, less advanced version called Flux sparked controversy earlier this year because it was found to do things that many other similar programs would not do, such as rendering Copyrighted characters and public figures in compromising positions.consume drugs or commit acts of violence.

There have been several reports about the latest Grok update being used to create photorealistic racist images of various football players and coaches. One image shows a player, who is black, picking cotton, while another shows the same player eating a banana surrounded by monkeys in a forest. A separate image shows two different players as pilots in the cockpit of a plane with the twin towers in the background. More footage shows a variety of players and coaches meeting and chatting with controversial historical figures such as Adolf Hitler, Saddam Hussein and Osama bin Laden.

Callum Hood, head of research at the Center for Countering Digital Hate (CCDH), said X had become a platform that incentivized and rewarded the spread of hate through revenue sharing, and AI images made this even more so. easy.

“What X has done, to a degree that no other mainstream platform has done, is offer cash incentives to accounts to do this, so accounts on

A key concern described by many is not only the relative lack of restrictions on what users can ask for, but also the ease with which directions given to Grok can bypass AI guidelines through “jailbreaking,” which includes describing the physical characteristics of whoever the user is. the prompter wants in the image, instead of just naming them.

a summer report published by the CCDH found that when given different hate prompts, Grok created 80% of them, 30% of which he created without rejection and another 50% of which he created after a leak.

The Premier League has said it is aware of the images and has a dedicated team assigned to help find and report racist abuse directed at athletes, which may lead to legal action. The Premier League is believed to have received more than 1,500 such reports last year and have introduced filters for players to use on their social media accounts to help block large amounts of abuse.

An FA spokesperson said: “Discrimination has no place in our game or in wider society. “We continue to urge social media companies and relevant authorities to address online abuse and to take action against offenders of this unacceptable behaviour.”

X and Grok have been contacted for comment.

You may also like