Will your smartphone be the next doctor’s office?

The same devices used to take selfies and type out tweets are being repurposed and commercialized for quick access to information needed for monitoring a patient’s health. A fingertip pressed against a phone’s camera lens can measure a heart rate. The microphone, kept by the bedside, can screen for sleep apnea. Even the speaker is being tapped, to monitor breathing using sonar technology. ¶ In the best of this new world, the data is conveyed remotely to a medical professional for the convenience and comfort of the patient or, in some cases, to support a clinician without the need for costly hardware. ¶ But using smartphones as diagnostic tools is a work in progress, experts say. Although doctors and their patients have found some real-world success in deploying the phone as a medical device, the overall potential remains unfulfilled and uncertain.

Smartphones come packed a decade ago, according the Pew with sensors capable of monitoring Research Center. The covid-19 a patient’s vital signs. pandemic has also pushed people

They can help assess people for to become more comfortable concussions, watch for atrial with virtual care. fibrillation and conduct mental Some of these products have health wellness checks, to name sought FDA clearance to be marketed the uses of a few nascent applications. as a medical device. That way, if patients must pay to use

Companies and researchers the software, health insurers are eager to find medical applications more likely to cover at least part for smartphone technology of the cost. Other products are are tapping into modern phones’ designated as exempt from this built-in cameras and light sensors; regulatory process, placed in the microphones; accelerometers, same clinical classification as a which detect body movements; Band-aid. But how the agency gyroscopes; and even handles AI and machine learning-based speakers. The apps then use medical devices is still artificial intelligence software to being adjusted to reflect software’s analyze the collected sights and adaptive nature. sounds to create an easy connection Ensuring accuracy and clinical between patients and physicians. validation is crucial to securing Earning potential and buy-in from health care providers. marketability are evidenced by And many tools still need the more than 350,000 digital fine-tuning, said Dr. Eugene health products available in app Yang, a professor of medicine at stores, according to a Grand View the University of Washington.

Research report. Currently, Yang is testing contactless

“It’s very hard to put devices measurement of blood into the patient home or in the pressure, heart rate and oxygen hospital, but everybody is just saturation gleaned remotely walking around with a cellphone via Zoom camera footage of a that has a network connection,” patient’s face. said Dr. Andrew Gostine, CEO Judging these new technologies of the sensor network company is difficult because they rely Artisight. Most Americans on algorithms built by machine own a smartphone, including learning and artificial intelligence more than 60% of people 65 and to collect data, rather than over, an increase from just 13% the physical tools typically used in hospitals. So researchers cannot “compare apples to apples” with medical industry standards, Yang said. Failure to build in such assurances undermines the technology’s ultimate goals of easing costs and access because a doctor still must verify results.

“False positives and false negatives lead to more testing and more cost to the health care system,” he said.

Big tech companies like Google have heavily invested in researching this kind of technology, catering to clinicians and in-home caregivers, as well as consumers. Currently, in the Google Fit app, users can check their heart rate by placing their finger on the rear-facing camera lens or track their breathing rate using the front-facing camera.

“If you took the sensor out of the phone and out of a clinical device, they are probably the same thing,” said Shwetak Patel, director of health technologies at Google and a professor of electrical and computer engineering at the University of Washington.

Google’s research uses machine learning and computer vision, a field within AI based on information from visual inputs like videos or images. So instead of using a blood pressure cuff, for example, the algorithm can interpret slight visual changes to the body that serve as proxies and biosignals for a patient’s blood pressure, Patel said.

Google is also investigating the effectiveness of the built-in microphone for detecting heartbeats and murmurs and using the camera to preserve eyesight by screening for diabetic eye disease, according to information the company published last year.

The tech giant recently purchased Sound Life Sciences, a Seattle startup with an Fdacleared sonar technology app. It uses a smart device’s speaker to bounce inaudible pulses off a patient’s body to identify movement and monitor breathing.

Binah.ai, based in Israel, is another company using the smartphone camera to calculate vital signs. Its software looks at the region around the eyes, where the skin is a bit thinner and analyzes the light reflecting off blood vessels back to the lens. The company is wrapping up a U.S. clinical trial and marketing its wellness app directly to insurers and other health companies, said company spokesperson Mona Popilian-yona.

The applications even reach into disciplines such as optometry and mental health:

With the microphone, Canary Speech uses the same underlying technology as Amazon’s Alexa to analyze patients’ voices for mental health conditions. The software can integrate with telemedicine appointments and allow clinicians to screen for anxiety and depression using a library of vocal biomarkers and predictive analytics, said Henry O’connell, the company’s CEO.

Australia-based Resapp Health got FDA clearance last year for its iphone app that screens for moderate to severe obstructive sleep apnea by listening to breathing and snoring. Sleepcheckrx, which will require a prescription, is minimally invasive compared with sleep studies currently used to diagnose sleep apnea. Those can cost thousands of dollars and require an array of tests.

Brightlamp’s Reflex app is a clinical decision support tool for helping manage concussions and vision rehabilitation, among other things. Using an ipad’s or iphone’s camera, the mobile app measures how a person’s pupils react to changes in light. Through machine learning analysis, the imagery gives practitioners data points for evaluating patients. Brightlamp sells directly to health care providers and is being used in more than 230 clinics. Clinicians pay a $400 standard annual fee per account, which is currently not covered by insurance. The Department of Defense has an ongoing clinical trial using Reflex.

In some cases, such as with the Reflex app, the data is processed directly on the phone — rather than in the cloud, Brightlamp CEO Kurtis Sluss said. By processing everything on the device, the app avoids running into privacy issues, as streaming data elsewhere requires patient consent.

But algorithms need to be trained and tested by collecting reams of data, and that is an ongoing process.

Researchers, for example, have found that some computer vision applications, like heart rate or blood pressure monitoring, can be less accurate for darker skin. Studies are underway to find better solutions.

Small algorithm glitches can also produce false alarms and frighten patients enough to keep widespread adoption out of reach. For example, Apple’s new car-crash detection feature, available on both the latest iphone and Apple Watch, was set off when people were riding roller coasters and automatically dialed 911.

“We’re not there yet,” Yang said. “That’s the bottom line.”

“It’s very hard to put devices into the patient home or in the hospital, but everybody is just walking around with a cellphone that has a network connection.”

Dr. Andrew Gostine, CEO of the sensor network company Artisight.

KHN ILLUSTRATION