Caliskan found that it was full of problematic gender and racial biases. CLIP learns to match images in a data set to descriptive text prompts. The more often something is repeated, such as Asian women in sexually graphic scenes, the stronger the association becomes in the AI model.Ĭaliskan has studied CLIP (Contrastive Language Image Pretraining), which is a system that helps Stable Diffusion generate images. One way Stable Diffusion 2.0 filters content is by removing images that are repeated often. A spokesperson says that the original model was released with a safety filter, which Lensa does not appear to have used, as it would remove these outputs. Stability.AI, the company that developed Stable Diffusion, launched a new version of the AI model in late November. Most other popular image-making AIs, such as Google’s Imagen and OpenAI’s DALL-E, are not open but are built in a similar way, using similar sorts of training data, which suggests that this is a sector-wide problem.Īs I reported in September when the first version of Stable Diffusion had just been launched, searching the model’s data set for keywords such as “Asian” brought back almost exclusively porn.

It’s notable that their findings were only possible because the LAION data set is open source. This leads to AI models that sexualize women regardless of whether they want to be depicted that way, Caliskan says-especially women with identities that have been historically disadvantaged.ĪI training data is filled with racist stereotypes, pornography, and explicit images of rape, researchers Abeba Birhane, Vinay Uday Prabhu, and Emmanuel Kahembwe found after analyzing a data set similar to the one used to build Stable Diffusion. And because the internet is overflowing with images of naked or barely dressed women, and pictures reflecting sexist, racist stereotypes, the data set is also skewed toward these kinds of images.

Images of nude wonder woman anime