San Francisco Demands Apple and Google Delete AI ‘Nudify’ Apps From App Stores

Editor's take

San Francisco's City Attorney has formally requested Apple and Google remove AI-powered "nudify" applications from their respective app stores, citing their widespread misuse for non-consensual deepfakes targeting women and girls.

This action highlights the growing tension between AI's creative potential and its capacity for harm, particularly concerning privacy and exploitation. The onus now falls on platform providers, like Apple and Google, to actively police their digital marketplaces for harmful AI applications, a task complicated by the sheer volume of apps and the evolving nature of AI capabilities. This situation directly impacts users, especially vulnerable populations, and forces a reckoning for app store moderation policies.

The next crucial development will be the platforms' response to these cease-and-desist letters and the establishment of clear, enforceable guidelines for AI-generated content. Whether Apple and Google proactively implement stricter vetting processes for AI apps or wait for further regulatory pressure will signal the industry's commitment to mitigating AI-driven abuse. The effectiveness of these measures in preventing future instances of deepfake exploitation will be the ultimate test.