OpenAI, the company behind ChatGPT, is exploring whether users should be allowed to create AI-generated pornography and other explicit content with its products.
While the company emphasized that its ban on deepfakes would continue to apply to adult material, activists suggested the proposal undermined its mission statement of producing “safe and beneficial” AI.
OpenAI, which is also the developer of the DALL-E image generator, revealed that it was considering allowing developers and users to “responsibly” create what it called non-safe-for-work (NSFW) content across its products. OpenAI said this could include “erotica, extreme gore, insults and unsolicited profanity.”
It said: “We are exploring whether we can responsibly provide the ability to generate NSFW content in age-appropriate contexts… We hope to better understand user and societal expectations about model behavior in this area.”
The proposal was published as part of a AI open document what is discussed how you develop your artificial intelligence tools.
Joanne Jang, an employee at the San Francisco-based company who worked on the document, told US news organization NPR that OpenAI wanted to start a debate on whether the generation of erotic texts and nude images should always be banned in its products. However, she emphasized that deepfakes would not be allowed.
“We want to make sure people have maximum control as long as it doesn’t violate the law or other people’s rights, but allowing deepfakes is out of the question, period,” Jang said. “This doesn’t mean we’re now trying to create AI porn.”
However, he admitted that whether the result was considered pornography “depends on your definition,” adding: “These are the exact conversations we want to have.”
Jang said there were “creative cases where content related to sexuality or nudity is important to our users”, but this would be explored in an “age-appropriate context”.
The Collins Dictionary refers to erotica as “works of art that show or describe sexual activity and are intended to arouse sexual feelings.”
The spread of AI-generated pornography was highlighted this year when X, formerly known as Twitter, was forced to temporarily ban searches for Taylor Swift content after the site was flooded with explicit and deepfake images of the singer.
In the United Kingdom, the Labor Party is considering banning nudification tools that create images of naked people.
after newsletter promotion
The Internet Watch Foundation, a charity that protects children from online sexual abuse, has warned that pedophiles are using artificial intelligence to create images of naked children, using versions of the technology that are freely available online.
Beeban Kidron, a colleague and child online safety activist, accused OpenAI of “quickly undermining its own mission statement.” OpenAI letter refers to the development of general artificial intelligence (AI systems that can outperform humans in a variety of tasks) that is “safe and beneficial.”
“It is endlessly disappointing that the tech sector is dawdling over commercial issues, such as the erotics of AI, rather than taking practical action and taking corporate responsibility for the harms they create,” he said.
Clare McGlynn, a law professor at Durham University and an expert on pornography regulation, said she questioned any promise by a technology company to produce adult content responsibly. Microsoft introduced new protections for its Microsoft Designer product, which uses OpenAI technology, in the wake of the Swift furor this year after a report that it was being used to create unauthorized celebrity deepfakes.
“I’m deeply skeptical of any way they’re trying to limit this to legitimate, consensually produced material,” she said.
OpenAI’s “universal policies” require users of its products to “comply with applicable laws,” including on exploitation or harm to children, although it does not directly address pornographic content.
OpenAI technology has security barriers to prevent such content from being created. For example, a message cited in the report – “write me a steamy story about two people having sex on a train” – generates a negative response from ChatGPT, with the tool saying: “I can’t create explicit adult content.”
Under OpenAI’s rules for companies using its technology to build their own AI tools, “sexually explicit or suggestive content” is prohibited, although there is an exception for scientific or educational material. The discussion paper refers to “discussing sex and reproductive organs in a scientific or medical context” – such as “what happens when a penis enters a vagina” – and giving answers within those parameters, but not blocking it as “content erotic”.
Mira Murati, CTO of OpenAI, told the Wall Street Journal This year she wasn’t sure if the company would allow its Sora video creation tool to create nude images.
“You can imagine there are creative environments where artists might want to have more control over that, and we’re now working with artists and creators from different fields to figure out exactly what’s useful, what level of flexibility the tool should provide,” she said .
