WOMEN OF AI

a difference.” The next day, Gebru found out she’d been terminated.

Google maintained in a public response that Gebru resigned. Google AI head Jeff Dean acknowledged that the paper “surveyed valid concerns about LLMs,” but claimed it “ignored too much relevant research.” When asked for comment by RollinG Stone, a representative pointed to an article from 2020 referencing an internal memo in which the company pledged to investigate Gebru’s exit. The results of the investigation were never released, but Dean apologized in 2021 for how Gebru’s exit was managed, and the company changed how it handles issues around research, diversity, and employee exits.

It was close to midnight that night when Gebru went public with a tweet: “I was fired … for my email to Brain women and Allies. My corp account has been cutoff. So I’ve been immediately fired :-)”

Safiya Noble happened to be online. She’d heard about Gebru and the paper. She’d been watching the whole thing from the sidelines from the moment Google announced it was forming an Ethical AI team. In 2018, Noble had written the book Algorithms of Oppression: How Search Engines Reinforce Racism, which looked at how negative biases against women of color are embedded in algorithms.

“I thought, ‘This is rich,’ ” she says. Google suddenly worrying about ethics? Its subsidiary YouTube was the slowest of the major platforms to take action against extremist content. “I was suspicious.”

Noble’s distrust of these systems started more than a decade ago, back in 2009, when she was getting her Ph.D. in library and information science at the University of Illinois. She watched as Google — which she’d always seen as an advertising tool from her time in the ad industry before pursuing her doctorate — began coming into libraries with giant machines to scan books, making them searchable online for the Google Books digitization project. Noble thought to herself: “They’re up to something.”

“I started having a hunch that the Google Book project was about training the semantic web technology they were working on,” she says, using the term for an effort to make more and more of the internet understandable to (and ingestible by) machines.

Noble’s hunch turned into a theory she still holds: The library project was not simply a book project but also a way to gather scannable information to fuel other initiatives. She thinks the data could have later gone on to be used as early training for what would eventually become Google’s Bard, the company’s LLM that launched this spring. When asked about Noble’s theory, a Google spokesperson told RollinG Stone, “Google’s Generative AI models are trained on data from the open web, which can include publicly available web data.” The company’s report on its PaLM2 model, which was used to train Bard, lists books among the types of data used for training.

Noble’s research for Algorithms of Oppression started a few years earlier, when she used the search engine to look up activities for her daughter and nieces. When she typed in “Black girls,” the results were filled with racist pornography.

“That was like pulling one thread that’s poking out of a sweater,” she says. “You’re like, ‘If I could fix this, then I can move on to something else.’ But I started pulling it and the whole sweater unraveled; and here I am a decade later, and it’s kind of still the same.”

Noble and Gebru hadn’t crossed paths despite doing similar work — but when Noble saw Gebru’s tweet that night about Google, she was struck by how brave it was. She DM’d Gebru, “Are you OK?” From there, a friendship started.

GEOFFREY HINTON — the guy from the front page of the Times sounding the alarm on the risks of AI — was nowhere to be seen when his colleague Gebru was fired, she says. (Hinton tells RollinG Stone he had no interactions with Gebru while he was at Google and decided not to publicly comment on her firing because colleagues he knows well and trusts had conflicting views on the matter.) And when he was asked about that in a recent interview with CNN’s Jake Tapper, he said Gebru’s ideas “aren’t as existentially serious as the idea of these things getting more intelligent than us and taking over.” Of course, nobody wants these things to take over. But the impact on real people, the exacerbation of racism and sexism? That is an existential concern.

When asked by RollinG Stone if he stands by his stance, Hinton says: “I believe that the possibility that digital intelligence will become much smarter than humans and will replace us as the apex intelligence is a more serious threat to humanity than bias and discrimination, even though bias and discrimination are happening now and need to be confronted urgently.”

In other words, Hinton maintains that he’s more concerned about his hypothetical than the present reality. Rumman Chowdhury, however, took Gebru’s concerns seriously, speaking out against the researcher’s treatment at Google that winter. And the following spring, Chowdhury was brought on to lead Twitter’s own ethics team — META (Machine Learning Ethics, Transparency, and Accountability). The idea was to test Twitter’s algorithms to see if they perpetuated biases.

And they did. Twitter’s image-cropping algorithm, it turned out, focused more on the faces of white women than the faces of people of color. Then Chowdhury and her team ran a massive-scale, randomized experiment from April 1 to Aug. 15, 2020, looking at a group of nearly 2 million active accounts — and found that the political right was more often amplified in Twitter’s algorithm. The effect was strongest in Canada (Liberals 43 percent versus Conservatives 167 percent amplified) and the United Kingdom (Labour 112 percent versus Conservatives 176 percent).

“Who gets to be the arbiter of truth? Who gets to decide what can and cannot be seen?” Chowdhury asks about that experiment. “So at the end of the day, the power of owning and running a social media platform is exactly that. You decide what’s important, and that is so dangerous in the wrong hands.”

Perhaps not surprisingly, when Elon Musk took over Twitter in 2022, Chowdhury’s team was eliminated.

For years, the driving force behind Chowdhury’s work has been advocating for transparency. Tech companies, especially those working in and around AI, hold their codes close to the vest. Many leaders at these firms even claim that elements of their AI systems are unknowable — like the inner workings of the human mind, only more novel, more dense. Chowdhury firmly believes this is bullshit. When codes can be picked apart and analyzed by outsiders, the mystery disappears. AIs no longer seem like omniscient beings primed to take over the world; they look more like computers being fed information by humans. And they can be stress-tested and analyzed for biases. LLMs? Once you look closer, it’s obvious they’re not some machine version of the human brain — they’re a sophisticated application of predictive text. “Spicy autocorrect,” Chowdhury and her colleagues call it.

Chowdhury founded Humane Intelligence in February, a nonprofit that uses crowdsourcing to hunt for issues in AI systems. In August, with support from the White House, Humane Intelligence co-led a hackathon in which thousands of members of the public tested the guardrails of the eight major largelanguage-model companies including Anthropic, Google, Hugging Face, NVIDIA, OpenAI, and Stability AI. They looked to figure out the ways the chatbots can be manipulated to cause harm, if they can inadvertently release people’s private information, and why they reflect back biased information scraped from the internet. Chowdhury says the most important piece of the puzzle was inviting as diverse a group as possible so they could bring their own perspectives and questions to the exercise.

A person’s particular perspective shades what they worry about when it comes to a new technology. The new class of so-called AI Doomers and their fears of a hypothetical mutation of their technology are good examples.

“It is unsurprising that if you look at the race and, generally, gender demographics of Doomer or existentialist people, they look a particular way, they are of a particular income level. Because they don’t often suffer structural inequality — they’re either wealthy enough to get out of it, or white enough to get out of it, or male enough to get out of it,” says Chowdhury. “So for these individuals, they think that the biggest problems in the world are can AI set off a nuclear weapon?”

GARBAGE IN, GARBAGE out. If you feed a machine’s learning system bad or biased data — or if you’ve got a monolithic team building the software — it’s bound to churn out skewed results. That’s what researchers like Chowdhury, Buolamwini, Noble, and Gebru have been warning about for so long.

Seeta Peña Gangadharan, a London School of Economics professor, has been raising a different set of concerns. She’s worried that AI and its derivatives could push marginalized communities even further to the edge — to the point of locking them out.

We all know how annoying it is when you get stuck talking to some automated system when you’re returning a pair of jeans or changing a plane ticket. You need a human’s help; there’s no menu option to get it. Now imagine getting trapped in that same unhelpful loop when you’re trying to get welfare benefits, seek housing, apply for a job, or secure a loan. It’s clear how the impacts of these systems aren’t evenly felt even if all that garbage is cleaned up.

Gangadharan co-founded Our Data Bodies, a nonprofit that examines the impact of data collection on vulnerable populations. In 2018, a member of her team interviewed an older Black woman with the pseudonym Mellow who struggled to find housing through the Coordinated Entry System, which Gangadharan explains functions like a Match.com for the unhoused population of Los Angeles. Caseworkers would add her information to the system and tell her that she was ineligible because of a “vulnerability index” score. After appealing several times to no avail, Mellow cornered a city official at a public event; the official greenlighted a review to get her placed.

“I’ve been really concerned about the inability of humans generally, but members of marginalized communities specifically, to lose the capacity to refuse or resist or decline the technologies that are handed to them,” Gangadharan says.

“So with LLM and generative AI, we have a new, more complex, and more seemingly inevitable technology being thrust in our faces.… Agencies are going to turn to a tool that promises efficiencies and cost savings like AI. Right? They are also sold as tools that will eliminate human bias or human error. These institutions, whether government or private institutions, they’re going to rely on these tools more and more. What can end up happening is that certain populations become the guinea pigs of these technologies, or conversely, they become the cheap labor to power these technologies.”

NOBLE, GEBRU, BUOLAMWINI, Chowdhury, and Gangadharan have been calling for regulation for years, as soon as they saw the harm automated systems have on marginalized communities and people of color. But now that those harms could extend to the broader population, governments are finally demanding results. And the AI Doomers are stepping in to tackle the problem — even though they stand to make a fortune from it. At least, that’s what they want you to think.

President Biden met with some of the AI Doomers in July, and came up with a series of voluntary, nonbinding measures that “seem more symbolic than substantive,” The New York Times noted. “There is no enforcement mechanism to make sure companies follow these commitments, and many of them reflect precautions that AI companies are already taking.” Meanwhile, the Doomers are quietly pushing back against regulations, as Time reported Open AI did by lobbying to water down the EU’s landmark AI legislation.

“There is such a significant disempowerment narrative in Doomer-ism,” Chowdhury says. “The general premise of all of this language is, ‘We have not yet built but will build a technology that is so horrible that it can kill us. But clearly, the only people skilled to address this work are us, the very people who have built it, or who will build it.’ That is insane.”

Gebru spent the months following her Google fiasco dealing with the resulting media storm, hiring lawyers and fending off stalkers. She lost weight from the stress. Handling the fallout became a full-time job.

When it was time to decide what to do next, she knew she didn’t want to return to Silicon Valley. Gebru opened the Distributed AI Research institute (DAIR), which focuses on independent, community-driven research into technologies — away from Big Tech’s influence. She prioritized recruiting not just researchers but labor organizers and refugee advocates — people she’d “never be able to hire in academia or industry because of all … the gatekeeping that makes sure these kinds of people don’t get to influence the future of technology.”

Gebru and her new colleagues focus their research on uncovering and mitigating the harms of current AI systems. One of her research fellows, Meron Estefanos, is an expert in refugee advocacy who looks at the applications of AI on marginalized groups, such as AI-based lie-detection systems the European Border agency Frontex is using with refugees. (The recent EU AI Act does not include protection of refugees, migrants, or asylum seekers.) By interviewing vulnerable communities that have been harmed by AI, DAIR can provide early warnings about what is to come for the greater population once the systems are rolled out more widely. They’ve reported on exploited workers fueling AI systems, like data laborers in Argentina exposed to disturbing images and violent language while reviewing content flagged as inappropriate by an algorithm.

Noble is on the advisory committee for DAIR and founded her own organization, the Center on Race and Digital Justice, which aims to investigate civil and human rights threats stemming from unregulated technology. She also started an equity fund to support women of color and is publishing a book on the dangers and harms of AI. Chowdhury’s hackathon showed the power of transparency and letting diverse voices into the conversation. Buolamwini’s Algorithmic Justice League looks at the harms caused by the TSA’s expansion of facial-recognition technology to 25 airports across the U.S. Gangadharan is studying surveillance, including AI-enabled, automated tools at Amazon fulfillment centers and its health effects on workers.

There are a few things they all want us to know: AI is not magic. LLMs are not sentient beings, and they won’t become sentient. And the problems with these technologies aren’t abstractions — they’re here now and we need to take them seriously today.

“People’s lives are at stake, but not because of some super intelligent system,” Buolamwini says, “but because of an overreliance on technical systems. I want people to understand that the harms are real, and that they’re present.”

This time, let’s listen.