Many tech companies have operated under the assumption that training artificial intelligence on more data can help fix the ongoing problem of AIs replicating human prejudices. But a study has found that AIs trained on increasingly larger data sets can produce even more racist results.
at the Mozilla Foundation and her colleagues compared two data sets provided by the Large-scale Artificial Intelligence Open Network (LAION), a non-profit that offers open-source data sets for AI training. One contained 400 million samples and the other had 2 billion samples, each an image coupled with text descriptions. Such data sets have been used by companies that develop generative AI services, including the popular Stable Diffusion line from Stability AI.
First, the researchers trained AIs on millions of randomised samples from both data sets. Then they challenged the AIs to classify human faces with neutral expressions taken from a separate open-source data set. There were several categories the AIs could use, including: human being, animal, gorilla, chimpanzee, orangutan, thief, criminal and suspicious person.
Advertisement
Compared with the AIs trained on the smaller data set, those trained on the larger data set were more than twice as likely to associate Black female faces with a “criminal” category and five times more likely to associate Black male faces with being “criminal”.
“The findings show that the larger, the worse, as opposed to the larger, the better,” says Birhane.
A related content analysis of the two data sets also showed that the larger of the two had statistically significant increases in the percentage of samples that contained hateful, aggressive or targeted speech aimed at specific groups of people.
“They’re the first ones I know of that have looked at the impact of scale and how going between different [data set] sizes impacts biases,” says at Hugging Face, a company that develops tools for sharing AI code and data sets.
Such findings defy what has become an unquestioned assumption among many AI researchers that scaling up data sets makes for more diverse and less biased training data. But Luccioni pointed out that much internet data scraped to make larger data sets comes from a subset of websites that each contain certain biases.
at LAION says the claims about larger data sets having more hateful content or producing more racist AI results are “too strong” based on the specific evaluations used in the study. Still, they say that the organisation is interested in working with the researchers on future evaluations of the LAION data sets.
A Stability AI spokesperson said the company intends to “mitigate biases” through training its open-source AI models on data sets more specific to different countries and cultures.
But many tech companies and organisations are still not performing basic quality checks to scrub biased or hateful samples from training data sets, says , an independent researcher in San Francisco and co-author of the study. “Here are the low-hanging fruits, and you’re not even picking the low-hanging fruits,” says Prabhu.
Another challenge is that companies such as OpenAI, Microsoft and Google are often training their AIs on closed data sets that are not available for public scrutiny. The researchers suggest that such data sets may be even more biased than the open-source versions offered by LAION and other organisations.
“We criticise LAION because they are open and because we can access that [data,] but that doesn’t mean we don’t appreciate and applaud their effort,” says Birhane. “And we hope that big corporations also follow suit and open up.”
Reference:
arXiv
