Apple Watch is smart, but not doc
Study questions reliance on device to assess, diagnosis heart ailments.
The Apple Watch has been quite successful as a smartwatch. The company would also like it to succeed as a medical device. The recently published results of the Apple Heart Study in the New England Journal of Medicine show there is still a long way to go.
An estimated 6 million people in the United States — nearly 2 percent — have atrial fibrillation, a type of irregular heartbeat that brings increased risk of events like clots, heart attacks and strokes. It is thought that about 700,000 of people with the condition don’t know they have it.
A selling point of the watch is a sensor that can monitor a wearer’s pulse and potentially detect atrial fibrillation.
To test the device’s ability to aid diagnosis, a group of researchers enrolled almost 420,000 Apple Watch wearers in a study. (Some of the researchers were Apple employees, and Apple sponsored the research.) Participants were monitored for about four months. Over that time, 2,161 of the study participants were notified of an irregular pulse, representing just over 0.5 percent of the sample.
Those people were offered telemedicine visits and, if their symptoms were mild, were offered electrocardiogram patches to wear for up to a week to help confirm a diagnosis of atrial fibrillation. Participants mailed the patches back and, if the results indicated an emergency, were contacted immediately and instructed to receive care. If the results were positive for atrial fibrillation but did not require immediate medical attention, the participants were offered a second telemedicine visit and instructed to see their regular physician.
But only 450 of the 2,161 people who were notified about having an irregular pulse returned their sensor patches for evaluation. This means that among those who signed up for the study, wore the watch and got a health alert, almost 80 percent ignored it.
Of the 450 participants who returned patches, atrial fibrillation was confirmed in 34 percent, or 153 people. Those 153 are about 0.04 percent of the 420,000 participants.
This doesn’t mean that the Apple device failed. It probably led some participants to be diagnosed sooner than they might have. How many, and how much of a difference this made in their health, though, is debatable.
Many news outlets reporting on the study mentioned a result: a “positive predictive value” of 84 percent. That statistic refers to the chance that someone actually has the condition if he or she gets a positive test result.
But this result wasn’t calculated from any of the numbers above. It specifically refers to the subset of patients who had an irregular pulse notification while wearing their confirmatory patch. That’s a very small minority of participants. Of the 86 who got a notification while wearing a patch, 72 had confirmed evidence of atrial fibrillation. (Dividing 72 by 86 yields 0.84, which is how you get a positive predictive value of 84 percent.)
Positive predictive values, although useful when talking to patients, are not always a good measure of a test’s effectiveness. When you test a device on a group where everyone has a disease, for instance, all positive results are correct. A flipped coin would have a positive predictive power of 100 percent in such a population, even though it’s a terrible test.
Other test characteristics like sensitivity (if you have a disease, how likely the test is to be positive) and specificity (if you don’t have a disease, how likely the test is to be negative) are more effective in evaluating a test’s overall quality. This study, unfortunately, wasn’t designed to determine those characteristics.
Other methods to screen and diagnose people with atrial fibrillation are available. A systematic review of mobile health devices for atrial fibrillation found 22 studies between 2014 and 2019 that reported on many of them.
Some had sensitivities and specificities pretty close to the ideal of 100. None are close to as large as this study, though.
Even blood pressure monitors, ubiquitous in physician’s offices, can screen for atrial fibrillation. A systematic review of them found that they had sensitivities greater than 85 percent and specificities greater than 90 percent.
Here’s the thing, though: Experts aren’t even sure if screening is a good idea to begin with.
After all, if we felt strongly enough about detecting asymptomatic people who might have atrial fibrillation, we could screen everyone with electrocardiograms. The U.S. Preventive Services Task Force has considered doing this among adults 65 and older, who are at higher risk for stroke. The group found that the evidence was insufficient to recommend doing so, because it’s not clear that this level of screening is better than current care. Just taking a pulse as part of a checkup is a pretty good screen by itself.
There is also a concern that electrocardiogram screening could turn up a lot of false positives, leading to misdiagnosis and unnecessary further testing, which incurs its own risks.
Moreover, the task force was focusing on a population where we might intervene: older people. Patients at high risk of stroke who have atrial fibrillation (i.e., older people) might be treated with anticoagulation. For younger ones at lower risk, it’s not immediately clear how we would treat them, or if we should.
And it’s younger people who are more likely to have a smartwatch.