Security system that detects atypical behavior
System and method that identify atypical behavior of a user. One embodiment of the system includes an eye tracker to perform tracking of a user's gaze while viewing items, an inward-facing head-mounted thermal camera to take thermal measurements of a region of interest on the face (TH ROI ) of the user, and a computer. The computer generates feature values based on TH ROI and the tracking, and utilizes a model to identify atypical behavior of the user based on the feature values. The model may be trained based on previous tracking and previous TH ROI of the user, taken while viewing other items.
1. A system configured to identify atypical behavior, comprising:
an eye tracker configured to perform tracking of a user's gaze while viewing items;
an inward-facing head-mounted thermal camera (CAM) configured to take thermal measurements of a region of interest on the face (TH ROI ) of the user; and
a computer configured to:
generate feature values based on TH ROI and the tracking; and
utilize a machine learning-based model to identify atypical behavior of the user based on the feature values; wherein the machine learning-based model was trained based on previous tracking and previous TH ROI of the user, taken while viewing other items.
2. The system of claim 1 , further comprising a frame configured to be worn on a user's head; wherein the eye tracker and CAM are coupled to the frame, the region of interest is on a periorbital area of the face, and CAM is located less than 15 cm from the face.
3. The system of claim 2 , further comprising a second inward-facing head-mounted thermal camera configured to take thermal measurements of a region on the forehead (TH ROI2 ); and wherein the computer is further configured to generate one or more of the feature values also based on TH ROI2 .
4. The system of claim 2 , further comprising a second inward-facing head-mounted thermal camera configured to take thermal measurements of a region on the nose (TH ROI2 ); and wherein the computer is further configured to generate one or more of the feature values also based on TH ROI2 .
5. The system of claim 1 , wherein the user's behavior during most of the viewing of the other items was typical, and the previous TH ROI were taken during different days and in different situations that include first TH ROI taken at most one hour before having a meal and second TH ROI taken at most one hour after having a meal.
6. The system of claim 1 , wherein when the user's behavior while viewing the items is atypical, a stress level of the user during the viewing, which is calculated based on TH ROI , reaches a threshold; and wherein stress levels of the user during most of the time spent viewing the other items, as calculated based on the previous TH ROI , do not reach the threshold.
7. The system of claim 1 , wherein the user's gaze is indicative of attention of the user in the items.
8. The system of claim 7 , wherein the computer is further configured to generate additional feature values indicative of expected attention of the user in the items and to utilize the additional feature values to identify the atypical behavior; wherein when the user's behavior while viewing the items is atypical, a divergence between the attention of the user in the items and the expected attention of the user in the items is above a certain value; and wherein during most of the time spent while viewing the other items, divergences between attention of the user in the other items and expected attention levels in the other items were not above the certain value.
9. The system of claim 8 , wherein the expected attention levels in the other items are calculated utilizing a certain model of the user; and wherein the certain model was trained based on previous tracking of the user's gaze.
10. The system of claim 8 , wherein the expected attention levels in the other items are calculated utilizing a certain model; and wherein the certain model was trained based on tracking of other users' gaze.
11. The system of claim 8 , wherein the expected attention levels in the other items are calculated using a saliency mapping algorithm.
12. The system of claim 1 , wherein the computer is further configured to generate one or more of the feature values based on a difference between TH ROI and a baseline value determined based on a set of previous measurements taken by CAM; and wherein most of the measurements belonging to the set were taken while the user's behavior was not considered atypical according to the machine learning-based model.
13. The system of claim 1 , further comprising a head-mounted display (HMD) configured to expose the user to video depicting the items; and wherein the computer is further configured to: (i) receive one or more values indicative of at least one of the following parameters of the user: heart rate, heart rate variability, galvanic skin response, respiratory rate, and respiratory rate variability, and (ii) to generate some of the feature values based on the one or more values.
14. The system of claim 1 , wherein the computer is further configured to: (i) receive one or more values measured by at least one of the following sensors coupled to the user: a photoplethysmogram (PPG) sensor, an electrocardiogram (ECG) sensor, an electroencephalography (EEG) sensor, a galvanic skin response (GSR) sensor, and a thermistor, and (ii) to generate some of the feature values based on the one or more values.
15. The system of claim 1 , wherein the computer is further configured to: (i) receive one or more values indicative of at least one of the following: whether the user touched at least one of the eyes, and whether the user is engaged in physical activity, and (ii) to generate some of the feature values based on the one or more values.
16. The system of claim 1 , wherein the computer is further configured to: (i) receive one or more values measured by at least one of the following sensors: an accelerometer, a pedometer, a humidity sensor, a miniature radar, a miniature active electro-optics distance measurement device, an anemometer, an acoustic sensor, and a light meter, and (ii) to generate some of the feature values based on the one or more values.
17. A method for identifying atypical behavior, comprising:
tracking a user's gaze while viewing items;
taking, using an inward-facing head-mounted thermal camera, thermal measurements of a region of interest on the face (TH ROI ) of a user;
generating feature values based on TH ROI and the tracking; and
utilizing a machine learning-based model to identify, based on the feature values, atypical behavior of the user; wherein the machine learning-based model was trained based on previous tracking and previous TH ROI of the user, taken while viewing other items.
18. The method of claim 17 , further comprising calculating expected attention of the user in the items and generating at least some of the feature values based on the expected attention of the user in the items; wherein the user's gaze is indicative of attention of the user in the items, and when the user's behavior while viewing the items is atypical, a divergence between the attention of the user in the items and the expected attention of the user in the items is above a certain value.
19. The method of claim 17 , wherein the region of interest is on a periorbital area of the face, and further comprising taking, using a second inward-facing head-mounted thermal camera, thermal measurements of a region on the forehead (TH ROI2 ); and generating one or more of the feature values also based on TH ROI2 .
20. The method of claim 17 , wherein the region of interest is on a periorbital area of the face, and further comprising taking, using a second inward-facing head-mounted thermal camera, thermal measurements of a region on the nose (TH ROI2 ); and generating one or more of the feature values also based on TH ROI2 .