Detecting fever from video images and a baseline
Described herein are embodiments of systems and methods that utilize images of a user's face to detect fever and intoxication. One embodiment of a system to detect fever includes first and second inward-facing head-mounted cameras that are located less than 5 cm from a user's face, are sensitive to wavelengths below 1050 nanometer, and are configured to capture images of respective first and second regions on the user's face. The system also includes a computer that calculates, based on baseline images captured with the cameras while the user did not have a fever, a baseline pattern of hemoglobin concentrations at regions on the face. The computer also calculates, based on a current set of images captured with the cameras, a current pattern of hemoglobin concentrations at the regions, and detects whether the user has a fever based on a deviation of the current pattern from the baseline pattern.
1. A system configured to detect fever, comprising:
first and second inward-facing head-mounted cameras (Cam 1&2 ), located less than 5 cm from a user's face, sensitive to wavelengths below 1050 nanometer, and configured to capture images of respective first and second regions on the user's face; wherein middles of the first and second regions are at least 4 cm apart; and
a computer configured to:
calculate, based on baseline images captured with Cam 1&2 while the user did not have a fever, a baseline pattern comprising values indicative of first and second baseline hemoglobin concentrations at the first and second regions, respectively;
calculate, based on a current set of images captured with Cam 1&2 , a current pattern comprising values indicative of first and second current hemoglobin concentrations at the first and second regions, respectively; and
detect whether the user has a fever based on a deviation of the current pattern from the baseline pattern.
2. The system of claim 1 , wherein the computer is further configured to:
calculate, based on additional baseline images captured with Cam 1&2 while the user had a fever, a fever-baseline pattern comprising values indicative of first and second fever hemoglobin concentrations at the first and second regions, respectively; and base the detection of whether the user has the fever also on a deviation of the current pattern from the fever-baseline pattern.
3. The system of claim 1 , wherein the first region is located above the user's eyes, and the second region is located below the user's eyes.
4. The system of claim 1 , wherein the middle of the first region is located less than 4 cm from the vertical symmetric axis of the user's face, and the middle of the second region is located more than 4 cm from the vertical symmetric axis.
5. The system of claim 1 , wherein the baseline images and the current set of images comprise a first channel corresponding to wavelengths that are mostly below 580 nanometers and a second channel corresponding to wavelengths mostly above 580 nanometers; the baseline pattern comprises: (i) first values, derived based on the first channel in the baseline images, which are indicative of the first and second baseline hemoglobin concentrations at the first and second regions, respectively, and (ii) second values, derived based on the second channel in the baseline images, which are indicative of third and fourth baseline hemoglobin concentrations at the first and second regions, respectively; the current pattern comprises: (i) third values, derived based on the first channel in the current set of images, which are indicative of the first and second current hemoglobin concentrations at the first and second regions, respectively, and (ii) fourth values, derived based on the second channel in the current set of images, which are indicative of third and fourth current hemoglobin concentrations at the first and second regions, respectively; whereby having separate values for different wavelengths enables to account for thermal interference from the environment when detecting whether the user has the fever because thermal interference from the environment is expected to affect the third values more than the fourth values.
6. The system of claim 1 , wherein the computer is further configured to calculate, based on the current set of images, a current heart rate and/or a current respiration rate of the user, and to detect whether the user has hyperthermia or hypothermia based on the deviation of the current pattern from the baseline pattern and deviations of the current heart rate and/or the current respiration rate from a baseline heart rate and/or baseline respiration rate of the user, respectively.
7. The system of claim 1 , further comprising a short-wave infrared (SWIR) inward-facing head-mounted camera configured to detect wavelengths in at least a portion of the range of 700 nm to 2500 nm; wherein the computer is further configured to utilize a deviation of a current SWIR pattern from a baseline SWIR pattern taken while the user did not have a fever in the detection of whether the user has the fever.
8. The system of claim 1 , wherein the computer is further configured to detect blushing based on the deviation of the current pattern from the baseline pattern, and present an alert to the user about the blushing.
9. The system of claim 1 , wherein the computer is further configured to utilize one or more calibration measurements of the user's core body temperature, taken by a different device, prior to a certain time, to calculate the user's core body temperature based on a certain set of images that were taken by Cam 1&2 after the certain time.
10. The system of claim 1 , wherein the computer is further configured to calculate the user's core body temperature based on the deviation of the current pattern from the baseline pattern.
11. The system of claim 1 , wherein the computer is further configured to calculate the values indicative of the baseline and current hemoglobin concentrations based on detecting imaging photoplethysmogram signals in the baseline and current images.
12. A method for detecting fever, comprising:
receiving, from first and second inward-facing head-mounted cameras (Cam 1&2 ) sensitive to wavelengths below 1050 nanometer, images of respective first and second regions on a user's face; wherein middles of the first and second regions are at least 4 cm apart;
calculating, based on baseline images captured with Cam 1&2 while the user did not have a fever, a baseline pattern comprising values indicative of first and second baseline hemoglobin concentrations at the first and second regions, respectively;
calculating, based on a current set of images captured with Cam 1&2 , a current pattern comprising values indicative of first and second current hemoglobin concentrations at the first and second regions, respectively; and
detecting whether the user has a fever based on a deviation of the current pattern from the baseline pattern.
13. The method of claim 12 , further comprising calculating, based on additional baseline images captured with Cam 1&2 while the user had a fever, a fever-baseline pattern comprising values indicative of first and second fever hemoglobin concentrations at the first and second regions, respectively; and detecting whether the user has the fever also based on a deviation of the current pattern from the fever-baseline pattern.
14. The method of claim 12 , wherein the baseline images and the current set of images comprise a first channel corresponding to wavelengths that are mostly below 580 nanometers and a second channel corresponding to wavelengths mostly above 580 nanometers; the baseline pattern comprises: (i) first values, derived based on the first channel in the baseline images, which are indicative of the first and second baseline hemoglobin concentrations at the first and second regions, respectively, and (ii) second values, derived based on the second channel in the baseline images, which are indicative of third and fourth baseline hemoglobin concentrations at the first and second regions, respectively; the current pattern comprises: (i) third values, derived based on the first channel in the current set of images, which are indicative of the first and second current hemoglobin concentrations at the first and second regions, respectively, and (ii) fourth values, derived based on the second channel in the current set of images, which are indicative of third and fourth current hemoglobin concentrations at the first and second regions, respectively.
15. The method of claim 12 , further comprising: calculating, based on the current set of images, a current heart rate and/or a current respiration rate of the user, and detecting whether the user has the fever, hyperthermia, or hypothermia also based on deviations of the current heart rate and/or the current respiration rate from a baseline heart rate and/or baseline respiration rate of the user, respectively.
16. A non-transitory computer readable medium storing one or more computer programs configured to cause a processor based system to execute steps comprising:
receiving, from first and second inward-facing head-mounted cameras (Cam 1&2 ) sensitive to wavelengths below 1050 nanometer, images of respective first and second regions on a user's face; wherein middles of the first and second regions are at least 4 cm apart;
calculating, based on baseline images captured with Cam 1&2 while the user did not have a fever, a baseline pattern comprising values indicative of first and second baseline hemoglobin concentrations at the first and second regions, respectively;
calculating, based on a current set of images captured with Cam 1&2 , a current pattern comprising values indicative of first and second current hemoglobin concentrations at the first and second regions, respectively; and
detecting whether the user has a fever based on a deviation of the current pattern from the baseline pattern.
17. The non-transitory computer readable medium of claim 16 , further comprising instructions for execution of the following steps: calculating, based on additional baseline images captured with Cam 1&2 while the user had a fever, a fever-baseline pattern comprising values indicative of first and second fever hemoglobin concentrations at the first and second regions, respectively; and detecting whether the user has the fever also based on a deviation of the current pattern from the fever-baseline pattern.
18. The non-transitory computer readable medium of claim 16 , further comprising instructions for execution of the following steps: calculating, based on the current set of images, a current heart rate and/or a current respiration rate of the user, and detecting whether the user has the fever, hyperthermia, or hypothermia also based on deviations of the current heart rate and/or the current respiration rate from a baseline heart rate and/or baseline respiration rate of the user, respectively.
19. The non-transitory computer readable medium of claim 16 , further comprising instructions for execution of a step involving calculating the user's core body temperature based on the deviation of the current pattern from the baseline pattern.
20. The method of claim 12 , further comprising calculating the user's core body temperature based on the deviation of the current pattern from the baseline pattern.