OpenAI, maker of ChatGPT, has recently disclosed plans that could revolutionize its technology’s applications, signaling a potential shift in its traditionally stringent content policies. According to draft documentation released last week, the company is exploring how to 'responsibly' introduce not-safe-for-work (NSFW) content through its platforms. The new policy is highlighted in a commentary note within the extensive Model Spec document, sparking a complex discussion about the future of AI in generating sensitive content, Wired reports.
"We’re exploring whether we can responsibly provide the ability to generate NSFW content in age-appropriate contexts through the API and ChatGPT," the note reads. "We look forward to better understanding user and societal expectations of model behavior in this area."
Current usage policies prohibit the generation of sexually explicit or even suggestive materials. However, the document suggests a nuanced consideration: the possibility of allowing NSFW content in age-appropriate contexts. This potential pivot is not about promoting explicit content indiscriminately but rather understanding societal and user expectations to guide model behavior responsibly.
OpenAI is considering how its technology could responsibly generate a range of different content that might be considered NSFW, including slurs and erotica. But the company is particular about how sexually explicit material is described.
In a statement to WIRED, company spokesperson Niko Felix said "we do not have any intention for our models to generate AI porn." However, NPR reported that OpenAI's Joanne Jang, who helped write the Model Spec, conceded that users would ultimately make up their own minds if its technology produced adult content, saying "Depends on your definition of porn." -Wired
The concern extends beyond the direct implications of NSFW content. Danielle Keats Citron, a law professor at the University of Virginia, has emphasized the broader societal repercussions, noting that intimate privacy violations can severely impact targeted individuals’ lives, restricting their opportunities and personal safety.
Of course, there are already a lot of NSFW AI content generators using things like Stable Diffusion, many of which border on (or worse) virtual child exploitation that we're sure this guy would defend.
"Intimate privacy violations, including deepfake sex videos and other nonconsensual synthesized intimate images, are rampant and deeply damaging," she said. "We now have clear empirical support showing that such abuse costs targeted individuals crucial opportunities, including to work, speak, and be physically safe." According to Citron, OpenAI's potential embrace of NSFW content is "alarming."
OpenAI’s announcement addresses an ongoing debate about the balance between technological innovation and ethical responsibility - particularly when it comes to setting precedents for how AI technologies might handle sensitive content in the future. The engagement with various stakeholders, as OpenAI spokesperson Grace McGuire told the outlet, noting that the Model Spec was an attempt to "bring more transparency about the development process and get a cross section of perspectives and feedback from the public, policymakers, and other stakeholders."
Earlier this year, OpenAI’s chief technology officer, Mira Murati, told The Wall Street Journal that she was “not sure” if the company would in future allow depictions of nudity to be made with the company’s video generation tool Sora.
AI-generated pornography has quickly become one of the biggest and most troubling applications of the type of generative AI technology OpenAI has pioneered. So-called deepfake porn—explicit images or videos made with AI tools that depict real people without their consent—has become a common tool of harassment against women and girls. In March, WIRED reported on what appear to be the first US minors arrested for distributing AI-generated nudes without consent, after Florida police charged two teenage boys for making images depicting fellow middle school students. -Wired
While OpenAI's usage policies prohibit impersonation without permission, the decisions made by OpenAI could have far-reaching effects. Of course, they also realize that if they don't compete in this space, someone else's AI will simply dominate, leaving OpenAI as the gimp.
