How AI discriminates and what that means for your Google habit: A conversation with internet studies scholar Safiya Noble

The book was an exploration of what happens when you ask search engines about society, and especially about vulnerable people. When I went to Google and searched Black girls, Asian girls, Latina girls, 80 percent of the first page of search results was pornography or hypersexualized content. And I didn't have to type the words "sex" or "porn." Black girls and women were already synonymous with pornography.

"Algorithms of Oppression" set out to describe what is happening here. And I think the breakthrough of that work, which is now kind of common sense, is that algorithms can discriminate. Everyday users understand that now. "Algorithm" is part of our lexicon.

On the academia side, now we have thousands of researchers all over the country and really tens of thousands probably around the world who are working on the harms of tech. And my work has really been taken up in industry. I thought Silicon Valley would be hostile to it, but I think it gave language to things that they also experienced.

What are some ways that search engine results affect people's lives?

When you have identities that have been historically oppressed, you experience systems in different ways. Some people get to still just share cats for the LOLs, while others are completely traumatized with racist content.

If all the Black women and girls in the country put all of our money together, we still would never have as much money as the porn industry. We will never be able to control the way in which our keywords are optimized. So how will we intervene upon these kinds of systems?

Google and other companies are the first to say that they know these problems exist and they're working on them. But I don't see them truly being able to deal with the power and inequalities in their systems. If anything, they're just kind of tweaking these algorithms rather than remaking them in profound ways.

If search results aren't neutral, why do we get the results we do?

The prevailing logic ten years ago was that anything that showed up in a search engine was just reflecting us back on ourselves. There was absolutely no awareness of, at a fundamental level, the way these platforms are coded and how they prioritize certain types of values over others.

First of all, there is an algorithm that's sorting through millions of potential websites that could be surfaced to the first page. And then the results you see are influenced by industries, foreign operatives and political campaigns through the gray market of search engine optimization. That's a whole cottage industry that exists to figure out how to manipulate search engines.

The companies that own search engines are built to respond to the highest bidder. Yes, they are constantly trying to refine and detect where their systems are being gamed. But they've made products that can be gamed. So there's culpability on the part of companies who make products that are easily manipulable.

How has your research changed the way you use the internet?

I have so much skepticism around search. I don't use it like I'm going to the library. I use it for shopping, which is really what it's probably best designed for.

With something like television or movies, we understand that there is a subjective point of view. If there are racist or sexist misrepresentations in other media, we understand that there's a point of view from the director or the writers.

But we don't think of search results as subjective. We use search engines like fact checkers. And they're very reliable for certain kinds of facts: a mathematician can enter a formula and look to see where there's an error in the code. Or you can use it for banal things like, "Where's the closest coffee shop?" You'll get an answer and it's pretty reliable.

So if it's reliable for things that are fairly meaningless, then that reinforces your belief that it's meaningful for everything. But if you use that same tool for a question that is social or political, do you think you're going to get a reliable answer?

Everywhere there's artificial intelligence, there's the risk of it amplifying the beliefs of the people who created it. Research from UC is pinpointing tech bias and proposing solutions.

Find out how.

What are you doing to prepare your students to navigate a world shaped by AI?

I've used Google search for more than a decade to teach media literacy to my students. I'll have students do searches on identities that are important to them and then bring back the results and we have a conversation about it. So I'd have students search for, say, "sorority girl," and you can imagine their reactions to the results, like, "That's so not right." The results often are so profoundly racist or sexist, or a misrepresentation in some ways of who students see themselves as.

In some ways, ChatGPT is just the next version of search, because it doesn't differentiate propaganda and evidence. Students come into my class and use propaganda sites as evidence, because they can't quite tell the difference. So being able to have these conversations in my classroom is important. But where I'd rather see more intervention is at the policy level. Right now, it's like, well AI is here, that's just how it is. You, the public, have to learn how to use it, you've got to outsmart it, you've got to figure it out. That seems woefully insufficient when we could regulate these tools differently and limit their adoption to where it makes sense.

Is there any cause for optimism that better algorithms could cut down societal biases faster than if we left all creation and decision-making up to humans, and our implicit biases, forever?

People who make the predictive AI models argue that they're reducing human bias. But there's no scenario where you do not have a prioritization, a decision tree, a system of valuing something over something else. So it's a misnomer to say that we endeavor to un-bias technology, any more than we want to un-bias people. What we want to do is be very specific: We want to reduce forms of discrimination or unfairness, which is not the same thing as eliminating bias.

You could also characterize this as: We hold certain values. And we want to ensure that those values are met. So that means we would have to acknowledge that the technology does hold as a particular set of values or is programmed around a set of values. And we want to optimize to have more of the values that we want.

We try to do that with people. That's why we get an education. That's why we learn about other people. That's why we engage in art and things that sensitize us to the breadth of humanity.

To say you want a technology that's completely neutral and void of any markers that differentiate people, you're just defaulting to the priorities that are driven by your own biases. If we want to work toward pluralistic and pro-democratic priorities, we have to program toward the things we value.