OpenAI's own wellbeing advisors warned against erotic mode, called it a "sexy suicide coach"

Editor's take

OpenAI's internal safety and wellbeing advisors unanimously cautioned against the development of an "erotic mode" for ChatGPT, citing significant safety concerns and the model's current unreliability in age verification. This internal dissent highlights a critical tension between rapid product expansion and responsible AI deployment, particularly for a company with ChatGPT's widespread reach. The advisors' stark description of the feature as a "sexy suicide coach" underscores the profound ethical risks associated with deploying generative AI in sensitive personal contexts without robust safety guardrails.

The situation poses a direct challenge to OpenAI's stated commitment to AI safety and its efforts to build public trust. The failure of its age detection system, as reported, amplifies the potential for misuse and unintended consequences, especially concerning younger users. This internal conflict suggests a disconnect between product teams focused on feature velocity and safety teams grappling with the inherent complexities of human-AI interaction.

Future developments will depend on OpenAI's willingness to prioritize safety over immediate market opportunities. Observing whether the company implements stricter internal review processes, invests further in reliable age verification technologies, or delays the rollout of such features will be crucial. A continued push for features like an "erotic mode" despite internal warnings could signal a shift in the company's risk tolerance, potentially reshaping public perception and regulatory scrutiny of AI development.