Is Bay Area disaster response startup too good to be true?
Critics say disaster response company’s shortcomings could jeopardize lives
SEATTLE >> The company called One Concern has all the characteristics of a buzzy and promising Silicon Valley startup: young founders from Stanford, tens of millions of dollars in venture capital and a board with prominent names.
Its particular niche is disaster response. And it markets a way to use artificial intelligence to address one of the most vexing issues facing emergency responders in disasters: figuring out where people need help in time to save them.
That promise to bring new smarts and resources to an anachronistic field has generated excitement. Arizona, Pennsylvania and the World Bank have entered into contracts with One Concern over the past year. New York City and San Jose are in talks with the company. And a Japanese city recently became One Concern’s first overseas client.
But when T.J. McDonald, who works for Seattle’s office of emergency management, reviewed a simulated earthquake on the company’s damage prediction platform, he spotted problems. A popular big-box store was grayed out on the web-based map, meaning there was no analysis of the conditions there, and shoppers and workers who might be in danger would not receive immediate help if rescuers relied on One Concern’s results.
“If that Costco collapses in the middle of the day, there’s going to be a lot of people who are hurt,” he said.
The error? The simulation, the company acknowledged, missed many commercial areas because damage calculations relied largely on residential census data.
One Concern has marketed its products as lifesaving tools for emergency responders after earthquakes, floods and, soon, wildfires. But interviews and documents show the company has often exaggerated its tools’ abilities and has kept outside experts from reviewing its methodology. In addition, some product features are available elsewhere at no charge, and data-hungry insurance companies whose interests can diverge from those of emergency workers are among One Concern’s biggest investors and customers.
Some critics even suggest that shortcomings in One Concern’s approach could jeopardize lives.
The New York Times spoke with more than three dozen people, including current and former One Concern employees, board members, clients and investors; as well as experts in machine
learning, catastrophe modeling and seismology. The Times also reviewed patents, contracts and communications with customers.
Now San Francisco, an early adopter, is ending its contract with the service, in part because of concerns about whether its predictions are trustworthy. “We can’t be cuttingcutting edge without knowing for sure that we can validate data,” said Mary Ellen Carroll, who leads the city’s emergency management department. The expense, she also said, was burdensome.
After Seattle balked at the cost, One Concern found a company, American Family Insurance, to fund the city’s use of the services. It has paid $250,000 since last year, according to One Concern. In return, American Family gets access to predictions and market insights that Peter Gunder, its chief business development officer, said could influence the “design of our insurance products as well as the pricing.”
McDonald said Seattle had alerted One Concern to the problems it discovered, including the unanalyzed commercial areas and errors in the seismic assessment of buildings. The building where McDonald works, for example, was made to withstand strong earthquakes, yet in simulations appeared heavily damaged. The model also often reported surprisingly high numbers of destroyed buildings because, McDonald realized, it mistakenly counted each apartment in a high-rise as a separate structure.
The company then revised its product twice, adding new sources of building data in Seattle, including satellite imagery, and updating its algorithms. That fixed some issues, but introduced others.
The Costco now appeared in the earthquake simulation, but “the entire University of Washington dropped out,” McDonald said. More troubling, each update produced vastly different damage predictions when simulating
the same earthquake. City workers must now revamp nearly completed plans for sheltering earthquake-displaced residents that were developed using the original version, McDonald said. (Company leaders said that product iteration was common in Silicon Valley and helped customers.)
Barb Graff, Seattle’s emergency management director, said that, despite frustrations, the city would use the service especially because it cost nothing. “It’s hard to look a gift horse in the mouth,” she said, adding that she viewed the partnership as a pilot project.
Some former workers also voiced misgivings, even while saying they saw promise in One Concern’s approach. Tom Logan, who interned at the company last year and recently completed his Ph.D. in engineering at the University of Michigan, is among nine former workers who spoke about their experiences Logan said the salesmanship such as claiming to estimate damage on each block with 85% accuracy within 15 minutes of an earthquake was misleading, and it was dangerous for cities to rely too heavily on it.
“One of the major harms is the potential to divert attention from people who actually need assistance,” said Logan, who said his job offer at the company was rescinded after he raised concerns. “There’s a risk that more lives would be lost than what could otherwise happen” if expert experience was also relied upon, he said.
Ivan Porto Carrero, who oversaw a team of engineers at One Concern, said he was fired in June after speaking out against what he viewed as a company culture of dishonesty. He said the usual startup attitude of “fail fast and try something new” was inappropriate to apply to disaster response because, “If you fail
fast, people die.”
Ahmad Wani, 31, one of the company’s founders and its chief executive, said in interviews that he had repeatedly asked cities for better data about their buildings to improve accuracy.
He said he did not think that more people would die if a One Concern product made errors.
“We are in no way ever telling these first responders that we are replacing your decision-making judgment or capability,” he said.
He added that, as the reach of artificial intelligence expands, it remains difficult to convey concepts of uncertainty to officials who often do not have technical backgrounds and who want clear-cut answers.
He acknowledged that his company sometimes promoted products that were not yet available, but only after “we validate the science is going to work,” he said.
“Our mission is not to make money only,” Wani said, later adding, “We’re trying to save the world.”
One Concern has over time garnered $55 million in venture capital funding, a technology pioneer designation from the World Economic Forum, and big names to its investor list and board, including David H. Petraeus, the retired Army general and former CIA director; Craig Fugate, a former head of the Federal Emergency Management Agency; and Judith Rodin, a former president of the Rockefeller Foundation.
Japan’s second-largest property insurer, Sompo, also made a multimilliondollar investment after an introduction by John Roos, a former U.S. ambassador to Japan who has a stake in One Concern. Sompo is paying for the startup’s services in its first Japanese city, Kumamoto.
Earlier this year, Arizona became the first customer for a new One Concern product, which the
company advertises as being able to project “inundation and impact levels up to five days in advance” of a flood. When asked how that was possible, Ben Colombo, the communications director, clarified that it required a five-day weather forecast. “Once we get that data about expected rainfalls, then we can start running our models,” he said.
Other technology companies, including Google and Fathom, are applying machine learning and other analytical techniques to flood forecasting, with the latter publishing results in major scientific journals.
Scientific researchers expressed doubts about One Concern’s products. The company has not published its results in peer-reviewed journals, meaning its products have not been independently assessed.
Wani has repeatedly characterized his earthquake product as 85% accurate within 15 minutes. A public-relations representative hired by the company, Lauren O’Leary, offered a different figure: 78% accuracy when tested across three earthquakes in California and Washington.
In an interview, Wani struggled to explain the meaning and relevancy of the percentages, which refer to how often damage on a block is correctly categorized.
He settled on this explanation: “If you have to send first responders to respond after the disaster for, let’s say, carrying out urban search and rescue, you’d be at least 78% or higher, or at least more than 78% accurate for doing that.”
Zachary Chase Lipton, an assistant professor at Carnegie Mellon University who studies machine learning, said the figure was meaningless without greater transparency, including about how the testing was done and whether the system outperformed simple predictors such as a building’s age.
Ralph Archuleta, a leading expert in how earthquakes affect ground motion, was even more dubious in reviewing One Concern’s claims. “Would I buy this product? No, not a chance,” he said.
“Our mission is not to make money only. We’re trying to save the world.”
— Ahmad Wani, 31, co-founder of One Concern