SYSTEM AND METHODS FOR DETERMINING HEALTH-RELATED METRICS FROM DATA COLLECTED BY A MOBILE DEVICE
Techniques and systems include predicting various health conditions using a photoplethysmography (PPG) signal or a video signal based on images of a patient's fingertip or other tissue or other body portion captured using a mobile device, such as a smartphone or tablet. The video signal may be transformed into a pseudo PPG signal to measure blood volume changes in the patient's blood flow to derive data indicating a disease state or health-related characteristic, such as blood oxygen level, blood glucose level, heart rate variability, hemoglobin, respiration rate, or arrhythmia. Techniques involve real-time environment assessment and problematic issue detection, training an artificial intelligence (AI) model to measure signal quality so as to select high-quality signals from a range of signals, and domain adaption and transfer learning to make use of publicly available datasets.
1 . A method of measuring or determining a disease state or a biomarker, the method comprising:
acquiring a series of images of a tissue using a camera of a mobile device;
transforming the series of images into pseudo photoplethysmography (PPG) signals;
assessing the pseudo PPG signals to determine the quality of the pseudo PPG signals;
determining whether to provide the pseudo PPG signals to a deep learning model or to discard the pseudo PPG signals based on the determined quality; and
measuring or determining the disease state or the biomarker based on the pseudo PPG signals provided to the deep learning model.
2 . The method of claim 1 , wherein the series of images comprises a video signal.
3 . The method of claim 1 , wherein acquiring the series of images includes controlling a flash and exposure settings of the camera.
4 . The method of claim 1 , wherein discarding the pseudo PPG signals comprises:
not providing the pseudo PPG signals to the deep learning model;
adjusting the flash or the exposure settings of the camera; and
initiating additional acquiring of images of the tissue using the adjusted flash or exposure settings.
5 . The method of claim 1 , further comprising:
measuring ambient light reaching the tissue using a photosensor onboard the mobile device;
measuring relative motion between the tissue and the camera using an accelerometer onboard the mobile device; and
grouping the measured ambient light and the measured relative motion as metadata.
6 . The method of claim 5 , wherein the quality of the pseudo PPG signals is based, at least in part, on the metadata.
7 . The method of claim 1 , wherein assessing the pseudo PPG signals to determine the quality of the pseudo PPG signals further comprises analyzing shapes and features of waveforms of the pseudo PPG signals, wherein the quality of the pseudo PPG signals is based, at least in part, on the shapes and features of the waveforms.
8 . The method of claim 1 , wherein the biomarker comprises measures of blood pressure, heart rate, SpO2, or other health-related quantity.
9 . The method of claim 1 , wherein the mobile device is a smartphone, wrist-worn wearable such as smartwatch or smart wristband, earbud, or tablet.
10 . A method of measuring or determining a disease state or biomarker, the method comprising:
acquiring a series of images of a tissue using a camera of a mobile device;
transforming the series of images into pseudo photoplethysmography (PPG) signals;
assigning a quality value to the pseudo PPG signals based on the quality of the pseudo PPG signals;
providing to a deep learning model the pseudo PPG signals having the quality value exceeding a threshold value;
discarding the pseudo PPG signals having the quality value less than the threshold value; and
measuring or determining the disease state or biomarker based on the pseudo PPG signals provided to the deep learning model.
11 . The method of claim 10 , wherein transforming the series of images into the PPG signals comprises applying a second deep learning model to the series of images, wherein an output of the second deep learning model is the PPG signals.
12 . The method of claim 10 , wherein the series of images comprises a video signal.
13 . The method of claim 10 , wherein acquiring the series of images includes controlling a flash and exposure settings of the camera.
14 . The method of claim 13 , wherein discarding the pseudo PPG signals comprises:
not providing the pseudo PPG signals to the deep learning model;
adjusting the flash or the exposure settings of the camera; and
initiating additional acquiring of images of the tissue using the adjusted flash or exposure settings.
15 . The method of claim 10 , further comprising:
measuring ambient light reaching the tissue using a photosensor onboard the mobile device;
measuring relative motion between the tissue and the camera using an accelerometer onboard the mobile device; and
grouping the measured ambient light and the measured relative motion as metadata.
16 . The method of claim 15 , wherein the quality of the pseudo PPG signals is based, at least in part, on the metadata.
17 . The method of claim 10 , wherein assessing the pseudo PPG signals to determine the quality of the pseudo PPG signals further comprises analyzing shapes and features of waveforms of the pseudo PPG signals, wherein the quality of the pseudo PPG signals is based, at least in part, on the shapes and features of the waveforms.
18 . The method of claim 10 , wherein the biomarker comprises measures of blood pressure and heart rate.
19 . The method of claim 10 , wherein the biomarker comprises measures SpO2 or other health-related quantity.
20 . The method of claim 10 , wherein the mobile device is a smartphone, wrist-worn wearable such as smartwatch or smart wristband, earbud, or tablet.