Biometric data capturing and analysis using a hybrid sensing systems
Apparatuses and methods detect a health condition of a user that may be assessed from a thermal sensor signal and/or radar sensor signal. One or more resultant biometric vectors may be generated from biometric vectors based on the thermal and radar signals, where the resultant biometric vectors contain resultant information about one or more biometric features for a user. Hazard information about the user is obtained from the one or more resultant biometric vectors, where the hazard information is indicative of a health event for the user. Consequently, an appropriate action on behalf of the user may be performed to ameliorate the health condition. The one or more resultant biometric vectors may include additional biometric features and/or a time sequence of the resultant biometric vectors to enhance hazard prediction. Moreover, the apparatuses and methods may support the user in different settings including a home, business, or vehicle.
1. An apparatus supporting biometric data about a user, the apparatus comprising:
a thermal sensor interface configured to obtain a thermal signal from a thermal sensor;
a radar sensor (RF) interface configured to obtain a radar signal from a radar sensor;
a processor for executing computer-executable instructions;
a memory storing the computer-executable instructions that when executed by the processor cause the apparatus to perform:
generating a first biometric vector from the thermal signal, wherein the first biometric vector contains first information about a first biometric feature and wherein the first biometric vector contains third information about a second biometric feature;
generating a second biometric vector from the radar signal, wherein the second biometric vector contains second information about the first biometric feature and wherein the second biometric vector contains fourth information about the second biometric feature;
extracting first resultant information about the first biometric feature from the first and second information;
generating a first resultant biometric vector containing the first resultant information about the first biometric feature;
generating the first resultant biometric vector containing second resultant information about the second biometric feature;
based on a health record of the user, applying a first weight to the first resultant information about the first biometric feature and a second weight to the second resultant information about the second biometric feature, wherein the first weight and the second weight are different; and
determining hazard information from the first resultant biometric vector, wherein the hazard information includes a hazard level and a confidence level and wherein the hazard level is indicative of an occurrence of a health event for the user and the confidence level is indicative of a degree of certainty of the hazard level.
2. The apparatus of claim 1 , wherein the memory storing computer-executable instructions that when executed by the processor further cause the apparatus to perform:
when the thermal signal is not available, extracting the first resultant information about the first biometric feature only from the second information contained the second biometric vector from the radar signal.
3. The apparatus of claim 1 comprising:
a communications interface,
wherein the memory storing computer-executable instructions that when executed by the processor further cause the apparatus to perform:
extracting a thermal signature from the thermal signal, wherein the thermal signature identifies the user; and
when the health event is detected, sending a message indicative of the hazard level about the user through the communications interface.
4. The apparatus of claim 3 , wherein the hazard information includes a hazard type and wherein the memory storing computer-executable instructions that when executed by the processor further cause the apparatus to perform:
downloading and storing the health record of the user;
including at least a portion of the health record in the message;
identifying an action mapped to the hazard type; and
executing the identified action on behalf of the user.
5. The apparatus of claim 1 , wherein the memory storing computer-executable instructions that when executed by the processor further cause the apparatus to perform:
determining a hazard type from the first resultant biometric vector;
identifying an action mapped to the hazard type; and
performing the identified action on behalf of the user.
6. The apparatus of claim 5 , wherein the hazard type is one of a plurality of hazard types and wherein the memory storing computer-executable instructions that when executed by the processor further cause the apparatus to perform:
grouping the plurality of hazard types with different importance levels; and
generating an alert warning based on the different importance levels.
7. The apparatus of claim 5 , wherein the memory storing computer-executable instructions that when executed by the processor further cause the apparatus to perform:
generating a time sequence of resultant biometric vectors, wherein the time sequence comprises the first resultant biometric vector and a second resultant biometric vector corresponding to first and second time instances, respectively; and
determining the hazard level from the time sequence of resultant biometric vectors.
8. The apparatus of claim 1 , wherein the memory storing computer-executable instructions that when executed by the processor further cause the apparatus to perform:
training a first model based on a previously occurring thermal signal;
transforming the thermal signal in accordance with the first model;
obtaining the first biometric vector from the transformed thermal signal;
training a second model based on a previously occurring radar signal;
providing a thermal signature and a motion vector from the first model to the second model;
transforming the radar signal in accordance with the second model, the thermal signature, and the motion vector; and
obtaining the second biometric vector from the transformed radar signal.
9. The apparatus of claim 8 , wherein the first model comprises a first neural network model, wherein the second model comprises a second neural network model, and wherein the memory storing computer-executable instructions that when executed by the processor further cause the apparatus to perform:
pre-training the first neural network model using a pre-processed image and a first resulting feature vector; and
pre-training the second neural network model using a pre-processed RF signal and a second resulting feature vector.
10. The apparatus of claim 9 , wherein a combined neural network model comprises a first neural stage and a second neural stage and wherein the memory storing computer-executable instructions that when executed by the processor further cause the apparatus to perform:
applying the first neural network model and the second neural network model to the first neural stage; and
training the second neural stage with the resulting feature vectors as inputs and a resultant hazard and a confidence level as outputs.
11. The apparatus of claim 8 , wherein the memory storing computer-executable instructions that when executed by the processor further cause the apparatus to perform:
downloading the first and second models from a cloud server;
receiving updated model information from the cloud server; and
updating at least one of the first and second models based on updated model information.
12. The apparatus of claim 11 , wherein the memory storing computer-executable instructions that when executed by the processor further cause the apparatus to perform:
sending, to the cloud server, biometric information about the first and second biometric vectors.
13. A method for assessing a health condition of a user, the method comprising:
obtaining thermal signal from a thermal sensor;
obtaining a radar signal from a radar sensor;
generating a first biometric vector from the thermal signal, wherein the first biometric vector contains first information about a first biometric feature and wherein the first biometric vector contains third information about a second biometric feature;
generating a second biometric vector from the radar signal, wherein the second biometric vector contains second information about the first biometric feature and wherein the second biometric vector contains fourth information about the second biometric feature;
extracting first resultant information about the first biometric feature from the first and second information;
generating a first resultant biometric vector containing the first resultant information about the first biometric feature; and
generating the first resultant biometric vector containing second resultant information about the second biometric feature;
based on a health record of the user, applying a first weight to the first resultant information about the first biometric feature and a second weight to the second resultant information about the second biometric feature, wherein the first weight and the second weight are different; and
determining hazard information from the first resultant biometric vector, wherein the hazard information includes a hazard level and a confidence level and wherein the hazard level is indicative of an occurrence of a health event for the user and the confidence level is indicative of a degree of certainty of the hazard level.
14. The method of claim 13 comprising:
when the thermal signal is not available, extracting the first resultant information about the first biometric feature only from the second information contained the second biometric vector from the radar signal.
15. The method of claim 13 comprising:
extracting a thermal signature from the thermal signal, wherein the thermal signature identifies the user; and
when the health event is detected, sending a message indicative of the hazard level about the user.
16. The method of claim 15 , wherein the hazard information includes a hazard type, the method comprising:
downloading and storing the health record of the user;
including at least a portion of the health record in the message;
identifying an action mapped to the hazard type; and
executing the identified action on behalf of the user.
17. The method of claim 13 comprising:
sending the first and second biometric vectors to a cloud server; and
receiving the hazard information from the cloud server.
18. The method of claim 13 comprising:
determining a hazard type from the first resultant biometric vector;
identifying an action mapped to the hazard type; and
performing the identified action on behalf of the user.
19. The method of claim 13 , wherein the first biometric vector contains fifth information about a third biometric feature and the second biometric vector does not contain any information about the third biometric feature, the method comprising:
extracting second resultant information about the third biometric feature only from the fifth information.
20. An apparatus providing an assessment of a vehicle driver of a vehicle, the apparatus comprising:
a thermal sensor interface configured to obtain a thermal signal from a thermal sensor;
a radar sensor (RF) interface configured to obtain, from a radar sensor, a radar signal reflected by the vehicle driver;
a processor for executing computer-executable instructions;
a memory storing the computer-executable instructions that when executed by the processor cause the apparatus to perform:
generating a first biometric vector from the thermal signal, wherein the first biometric vector contains first information about a first biometric feature;
generating a second biometric vector from the radar signal, wherein the second biometric vector contains second information about the first biometric feature;
extracting first resultant information about the first biometric feature from the first and second information;
generating a first resultant biometric vector containing the first resultant information about the first biometric feature;
determining a hazard type from the first resultant biometric vector, wherein the hazard type is indicative of an occurrence of a health event for the vehicle driver;
identifying an action mapped to the hazard type;
performing the identified action on behalf of the vehicle driver;
downloading a chronicle health record and conduct record of the vehicle driver; and
identifying the identified action based on the chronicle health and conduct record.
21. The apparatus of claim 20 , wherein the identified action is indicative of switching to a self-driving function, the apparatus comprising:
a self-driving interface, wherein the memory storing computer-executable instructions that when executed by the processor further cause the apparatus to perform:
instructing, through the self-driving interface, the vehicle to be self-driven to an emergency health service.
22. The apparatus of claim 20 , wherein the memory storing computer-executable instructions that when executed by the processor further cause the apparatus to perform:
based on the chronicle health and conduct record of the vehicle driver, applying a first weight to the first resultant information about the first biometric feature and a second weight to a second resultant information about a second biometric feature, wherein the first weight and the second weight are different.
23. The apparatus of claim 20 , wherein the memory storing computer-executable instructions that when executed by the processor further cause the apparatus to perform:
when the health event is detected, sending the chronicle health and conduct record to an emergency service.
24. An apparatus providing an assessment of a vehicle driver of a vehicle, the apparatus comprising:
a radar sensor (RF) interface configured to obtain, from a radar sensor, a radar signal reflected by the vehicle driver;
a processor for executing computer-executable instructions;
a memory storing the computer-executable instructions that when executed by the processor cause the apparatus to perform:
generating a biometric vector from the radar signal, wherein the biometric vector contains information about a first biometric feature;
extracting first resultant information about the first biometric feature from the information;
generating a first resultant biometric vector containing the first resultant information about the first biometric feature;
determining a hazard type from the first resultant biometric vector, wherein the hazard type is indicative of an occurrence of a health event for the vehicle driver;
identifying an action mapped to the hazard type;
performing the identified action on behalf of the vehicle driver;
downloading a chronicle health record and conduct record of the vehicle driver; and
identifying the identified action based on the chronicle health and conduct record.
25. The apparatus of claim 24 , wherein the identified action is indicative of switching to a self-driving function, the apparatus comprising:
a self-driving interface, wherein the memory storing computer-executable instructions that when executed by the processor further cause the apparatus to perform:
instructing, through the self-driving interface, the vehicle to be self-driven to an emergency health service.
26. The apparatus of claim 24 , wherein the memory storing computer-executable instructions that when executed by the processor further cause the apparatus to perform:
based on the chronicle health and conduct record of the vehicle driver, applying a first weight to the first resultant information about the first biometric feature and a second weight to a second resultant information about a second biometric feature, wherein the first weight and the second weight are different.
27. The apparatus of claim 24 , wherein the memory storing computer-executable instructions that when executed by the processor further cause the apparatus to perform:
when the health event is detected, sending the chronicle health and conduct record to an emergency service.