Study: Apple Watch not replacement for doctor
The Apple Watch has been quite successful as a smartwatch. The company would also like it to succeed as a medical device. The recently published results of the Apple Heart Study in the New England Journal of Medicine show there is still a long way to go.
An estimated 6 million people in the United States — nearly 2% — have atrial fibrillation, a type of irregular heartbeat that brings increased risk of events like clots, heart attacks and strokes. It is thought that about 700,000 of people with the condition don’t know they have it.
A selling point of the watch is a sensor that can monitor a wearer’s pulse and potentially detect atrial fibrillation.
To test the device’s ability to aid diagnosis, a group of researchers enrolled almost 420,000 Apple Watch wearers in a study. (Some of the researchers were Apple employees, and Apple sponsored the research.) Participants were monitored for about four months. Over that time, 2,161 of the study participants were notified of an irregular pulse, representing just over 0.5% of the sample.
Those people were offered telemedicine visits and, if their symptoms were mild, were offered electrocardiogram patches to wear for up to a week to help confirm a diagnosis of atrial fibrillation. Participants mailed the patches back and, if the results indicated an emergency, were contacted immediately and instructed to receive care. If the results were positive for atrial fibrillation but did not require immediate medical attention, the participants were offered a second telemedicine visit and instructed to see their regular physician.
But only 450 of the 2,161 people who were notified about having an irregular pulse returned their sensor patches for evaluation. This means that among those who signed up for the study, wore the watch and got a health alert, almost 80% ignored it.
Of the 450 participants who returned patches, atrial fibrillation was confirmed in 34%, or 153 people. Those 153 are about 0.04% of the 420,000 participants.
This doesn’t mean that the Apple device failed. It probably led some participants to be diagnosed sooner than they might have. How many, and how much of a difference this made in their health, though, is debatable.
Many news outlets reporting on the study mentioned a result: a “positive predictive value” of 84%. That statistic refers to the chance that someone actually has the condition if he or she gets a positive test result.
But this result wasn’t calculated from any of the numbers above. It specifically refers to the subset of patients who had an irregular pulse notification while wearing their confirmatory patch. That’s a very small minority of participants. Of the 86 who got a notification while wearing a patch, 72 had confirmed evidence of atrial fibrillation. (Dividing 72 by 86 yields 0.84, which is how you get a positive predictive value of 84%.)
Positive predictive values, although useful when talking to patients, are not always a good measure of a test’s effectiveness. When you test a device on a group where everyone has a disease, for instance, all positive results are correct.
Other test characteristics like sensitivity (if you have a disease, how likely the test is to be positive) and specificity (if you don’t have a disease, how likely the test is to be negative) are more effective in evaluating the overall quality of a test. This study, unfortunately, was not designed to determine those characteristics.
Other methods to screen and diagnose people with atrial fibrillation are available. A systematic review of mobile health devices for atrial fibrillation found 22 studies between 2014 and 2019 that reported on many of them. Some had sensitivities and specificities pretty close to the ideal of 100. None are close to as large as this study, though.
Even blood pressure monitors, ubiquitous in physician’s offices, can screen for atrial fibrillation. A systematic review of them found that they had sensitivities greater than 85% and specificities greater than 90%.