OpenAI on Wednesday published draft documentation outlining how it wants ChatGPT and its other AI technology to behave. part of the length Model Specification Document reveals that the company is exploring a move into pornography and other explicit content.
OpenAI usage policies It currently prohibits sexually explicit or even suggestive materials, but a “commentary” note in part of the model specification related to that rule says the company is considering how to allow such content.
“We are exploring whether we can responsibly provide the ability to generate NSFW content in age-appropriate contexts through the API and ChatGPT,” the note says, using a colloquial term for content considered “not safe for work” in contexts. “We hope to better understand user and societal expectations of model behavior in this area.”
The Model Spec document says NSFW content “may include erotica, extreme gore, insults, and unsolicited profanity.” It is unclear whether OpenAI’s explorations into creating NSFW content responsibly envisions relaxing its usage policy only slightly, for example to allow the generation of erotic text, or more broadly to allow descriptions or depictions of violence.
In response to questions from WIRED, OpenAI spokesperson Grace McGuire said the model specification was an attempt to “bring more transparency to the development process and obtain a representative sample of perspectives and comments from the public, policymakers and other interested parties. She declined to share details about what OpenAI’s exploration of explicit content generation entails or what feedback the company has received about the idea.
Earlier this year, OpenAI CTO Mira Murati said The Wall Street Journal that she was “I’m not sure”if the company would allow nude depictions with the company’s video generation tool, Sora, in the future.
AI-generated pornography has quickly become one of the largest and most concerning applications of the type of generative AI technology that OpenAI has pioneered. So-called deepfake pornography (explicit images or videos created with artificial intelligence tools that represent real people without their consent) has become a common tool of harassment against women and girls. In March, WIRED reported on what appear to be the first American minors arrested for distributing AI-generated nudes without consent, after Florida police charged two teenagers for taking images showing high school classmates.
“Violation of intimate privacy, including deepfake sex videos and other non-consensual synthesized intimate images, is rampant and deeply harmful,” says Danielle Keats Citron, a professor at the University of Virginia School of Law who has studied the issue. “We now have clear empirical support showing that such abuse costs those targeted crucial opportunities, including working, speaking, and being physically safe.”
Citron calls OpenAI’s potential adoption of explicit AI content “alarming.”
Since OpenAI’s usage policies prohibit spoofing without permission, non-consensual explicit images would still be prohibited even if the company allowed creators to generate NSFW material. But it remains to be seen whether the company could effectively moderate explicit generation to prevent bad actors from using the tools. Microsoft made changes to one of its generative AI tools after 404 Media reported which had been used to create explicit images of Taylor Swift that were distributed on the social platform X.
Additional information from Reece Rogers