AI and Big Tech should not have control over health
In an interview with the Wall Street Journal earlier this year, David Feinberg, the head of Google Health and a self-professed astrology buff, enthused that, “If you believe me that all we are doing is organising information to make it easier for your doctor, I’m going to get a little paternalistic here: I’m never going to let that get opted out.” In other words, patients will soon have no choice but to receive personalised clinical horoscopes based on their own medical histories and inferences drawn from a growing pool of patient records. But even if we want such a world, we should take a hard look at what today’s health-tech proponents are really selling.
In recent years, most of America’s Big Tech firms – along with many startups, the Big Pharma companies, and others — have entered the health-tech sector. With big-data analytics, artificial intelligence, and other novel methods, they promise to cut costs for struggling health-care systems, revolutionise how doctors make medical decisions, and save us from ourselves. What could possibly go wrong?
Quite a lot, it turns out. In Weapons of Math Destruction, data scientist Cathy O’Neil lists many examples of how algorithms and data can fail us in unsuspecting ways. When transparent data-feedback algorithms were applied to baseball, they worked better than expected; but when similar models are used in finance, insurance, law enforcement, and education, they can be highly discriminatory and destructive.
Health care is no exception. Individuals’ medical data are susceptible to subjective clinical decisionmaking, medical errors, and evolving practices, and the quality of larger data sets is often diminished by missing records, measurement errors, and a lack of structure and standardisation. Nonetheless, the bigdata revolution in health care is being sold as if these troubling limitations did not exist. Worse, many medical decision-makers are falling for the hype.
One could argue that as long as new solutions offer some benefits, they are worth it. But we cannot really know whether data analytics and AI actually do improve on the status quo without large, welldesigned empirical studies. Not only is such evidence lacking; there is no infrastructure or regulatory framework in place to generate it. Big-data applications are simply being introduced into health-care settings as if they were harmless or unquestionably beneficial.
Consider Project Nightingale, a private data-sharing arrangement between Google Health and Ascension, a massive non-profit health system in the US. When the Wall Street Journal first reported on this secret relationship last November, it triggered a scandal over concerns about patient data and privacy. Worse, as Feinberg admitted just two months later, “We didn’t know what we were doing.”
Given that the Big Tech companies have no experience in healthcare, such admissions should come as no surprise, despite the attempts to reassure us otherwise. Worse, at a time when individual privacy is becoming more of a luxury than a right, the algorithms that are increasingly ruling our lives are becoming inaccessible black boxes, shielded from public or regulatory scrutiny to protect corporate interests. And in the case of healthcare, algorithmic diagnostic and decision models sometimes return results that doctors themselves do not understand.
Although many of those pouring into the health-tech arena are well-intentioned, the industry’s current approach is fundamentally unethical and poorly informed. No one objects to improving health care with technology. But before rushing into partnerships with tech companies, health-care executives and providers need to improve their understanding of the health-tech field.
For starters, it is critical to remember that bigdata inferences are gleaned through statistics and mathematics, which demand their own form of literacy. Another critical area is AI, which requires both its own architecture — that is, the rules and basic logic that determine how the system operates — and access to massive amounts of potentially sensitive data. The goal is to position the system so that it can ‘teach’ itself how to deliver optimal solutions to stated problems. But, here, one must remember that the creators of the architecture — the people writing the rules and articulating the problems — are as biased as anyone else, whether they mean to be or not. Moreover, AI systems are guided by data from the current health-care system, making them prone to replicating its own failures and successes.
At the end of the day, improving health care through big data and AI will likely take much more trial and error than techno-optimists realise. If conducted transparently and publicly, big-data projects can teach us how to create high-quality data sets prospectively, thereby increasing algorithmic solutions’ chances of success.
Above all, health-care providers and governments should remove their rose-tinted glasses and think critically about the implications of largely untested new applications in health care. Having been massively overhyped, big-data health-care solutions are being rushed to market in without meaningful regulation, transparency, standardisation, accountability, or robust validation practices. Patients deserve health systems and providers that will protect them, rather than using them as mere sources of data for profit-driven experiments.
Big-data health-care solutions are being rushed to market in without meaningful regulation, transparency, standardisation, accountability, or robust validation practices.