A NEW CODE OF ETHICS?
Tech workers now want to know: What are we building this for?
SAN FRANCISCO — Jack Poulson, a Google research scientist, recently became alarmed by reports that the company was developing a search engine for China that would censor content on behalf of the government.
While Poulson works on search technologies, he had no knowledge of the product, which was code-named Dragonfly. So in a meeting last month with Jeff Dean, the company’s head of artificial intelligence, Poulson asked if Google planned to move ahead with the product and if his work would contribute to censorship and surveillance in China.
According to Poulson, Dean said Google complied with surveillance requests from the federal government and asked rhetorically if the company should leave the U.S. market in protest. Dean also shared a draft of a company email that read, “We won’t and shouldn’t provide 100 percent transparency to every Googler, to respect our commitments to customer confidentiality and giving our product teams the freedom to innovate.”
The next day, Poulson quit the company. Dean did not respond to a request for comment, and Google declined to comment.
Across the technology industry, rank-and-file employees are demanding greater insight into how their companies are deploying the technology that they build. At Google, Amazon, Microsoft and Salesforce, as well as at tech startups, engineers and technologists are increasingly asking whether the products they are working on are being used for surveillance in places like China or for military projects in the United States or elsewhere.
That is a change from the past, when Silicon Valley workers typically developed products with little questioning about the social costs. It is also a sign of how some tech companies, which grew by serving consumers and businesses, are expanding more into government work. And the shift coincides with concerns in Silicon Valley about the Trump administration’s policies and the larger role of technology in government.
“You can think you’re building technology for one purpose, and then you find out it’s really twisted,” said Laura Nolan, 38, a senior software engineer who resigned from Google in June over the company’s involvement in Project Maven, an effort to build artificial intelligence for the Defense Department that could be used to target drone strikes.
All of this has led to growing tensions between tech employees and managers. In recent months, workers at Google, Microsoft and Amazon have signed petitions and protested to executives over how some of the technology they helped create is being used. At smaller companies, engineers have begun asking more questions about ethics.
And the change is likely to last: Some engineering students have said they are demanding more answers and are asking similar questions, even before they move into the workforce.
“What people are looking for — not just employees — they are looking for some clarity,” said Frank Shaw, a Microsoft spokesman. “Are there principles that get applied? Even if you don’t agree with the decision that gets made, if you understand the thinking behind it, it helps a lot.”
Amazon did not respond to a request for comment.
Poulson, whose work involved incorporating a variety of languages into Google search, said he did not initially think his research could be involved in Dragonfly — until he noticed Chinese had been added to a list of languages for his project.
“Most people don’t know the holistic scope of what they’re building,” said Poulson, 32, who worked at Google for over two years. “You don’t have knowledge of where it’s going unless you’re sufficiently senior.”
The difficulties of knowing what companies are doing with technologies is compounded because engineers at large tech companies often build infrastructure — like algorithms, databases and even hardware — that underpins almost every product a company offers. At Google, a storage system called Colossus is used by Google search, maps and email.
“It would be very difficult for most engineers in Google to be sure that their work wouldn’t contribute to these projects in some way,” said Nolan, who helped to keep Google’s systems running online smoothly. “My personal feeling was that if the organization is doing something I find ethically unacceptable, then I was contributing to it.”
Yet executives at tech companies have claimed that complete transparency is not possible.
“We’ve always had confidential projects as a company. I think what happened when the company was smaller, you had a higher chance of knowing about it,” Sundar Pichai, Google’s chief executive, said at a staff meeting in August, according to a transcript provided to The New York Times. “I think there are a lot of times when people are in exploratory stages where teams are debating and doing things, so sometimes being fully transparent at that stage can cause issues.”
Such policies have rippled beyond tech companies. In June, more than 100 students at Stanford, MIT and other top colleges signed a pledge saying they would turn down job interviews with Google unless the company dropped its Project Maven contract. (Google said that month that it would not renew the contract once it expired.)
“We are students opposed to the weaponization of technology by companies like Google and Microsoft,” the pledge stated. “Our dream is to be a positive force in the world. We refuse to be complicit in this gross misuse of power.”
Alex Ahmed, a doctoral candidate in computer science at Northeastern University in Boston, said she organized a student discussion on campus this month to debate whether they should work for tech companies that made decisions they believed to be unethical.
“We’re not given an ethics course. We’re not given a political education,” Ahmed, 29, said. “It’s impossible for us to do this unless we create the conversations for ourselves.”
Bridget Frey, chief technology officer at online real estate company Redfin, said job candidates had increasingly raised ethical questions in interviews. This summer, interns questioned Redfin’s chief executive, Glenn Kelman, about whether the way the site displays school information and test scores could contribute to socio-economic divides in neighborhoods. In response, the company added more context about the test score information.
Employees are now frequently asking, “If you don’t share the information with me, how can I make sure this isn’t happening here?” Frey said.