Systems and methods for operator detection
Various embodiments of the invention provide for methods and systems for detecting an operator's use of a secure system. Images are acquired from image sensors that monitor a detection zone about the secure system. Faces within the images are identified and tracked among subsequently acquired images. Upon receipt of a valid login at the secure system, one of the tracked faces is selected as a primary operator and tracked as they operate the secure system.
1. A computer-implemented method for detecting an operator's use of a secure system, the method comprising:
acquiring one or more images from one or more image sensors, the image sensors being co-located with the secure system and monitoring a detection zone about the secure system;
identifying one or more faces within the images;
tracking the faces among subsequently acquired images; and
upon receipt of a valid login at the secure system, (i) selecting one of the tracked faces as a primary operator, (ii) tracking the head of the primary operator independent of the primary operator's face in images obtained by the one or more image sensors during operation of the secure system, and (iii) determining movements of the primary operator based on at least one of a location or a size of the tracked head regardless of whether the operator is facing the one or more image sensors,
wherein tracking the head of the primary operator independent of the primary operator's face comprises:
assigning tracking points to points on tracked head images;
periodically updating and re-assigning the tracking points;
detecting errant points based on a velocity threshold; and
applying a correction factor to the errant points, wherein the correction factor is based at least in part on an average change in position of neighboring points.
2. The method of claim 1 further comprising detecting the absence of the primary operator from the detection zone after tracking the head of the primary operator operating the system.
3. The method of claim 2 further comprising detecting the re-entry of the primary operator into the detection zone after detecting the absence of the primary operator from the detection zone.
4. The method of claim 3 wherein the detection of a re-entry occurs within a threshold time period.
5. The method of claim 4 wherein sets of facial exemplars for multiple operators are stored and a re-appearing operator may be re-authenticated on the secure system, thereby restoring or activating their session automatically.
6. The method of claim 2 wherein detecting the absence of the primary operator from the detection zone comprises determining if one or more walk-away conditions exist, the walk-away conditions comprising one or more of (i) a center point of a tracked head image nearing an edge of the image, (ii) a determination that the tracked head image will be outside a frame within the image in a subsequent frame, and (iii) the tracked head image appears smaller than a size threshold.
7. The method of claim 2 wherein the secure system is continuously notified while the operator remains in the detection zone.
8. The method of claim 7 wherein the notification causes the activation of a screen saver.
9. The method of claim 3 wherein detecting the re-entry of the primary operator into the detection zone comprises:
within a re-entry time threshold, detecting left and right eye positions of a newly-appearing face in an image;
computing a normalized image of the image;
comparing the normalized image to one or more stored facial images, and, upon finding a match, detecting the re-entry of the primary operator into the detection zone.
10. The method of claim 3 wherein information about the operator's presence within the detection zone is stored and reported to allow automatic time and attendance reporting.
11. The method of claim 1 wherein identifying one or more faces within the images comprises:
generating a list of candidate faces from the one or more faces;
removing invalid faces from the list of candidate faces based on one or more of a size parameter, a completeness parameter, a matching parameter and a motion parameter, thereby creating a filtered face list;
comparing the filtered face list with a previously generated face list using one or more of a size parameter, a completeness parameter, a matching parameter and a motion parameter; and
updating the previously generated face list with face information from the filtered face list.
12. The method of claim 1 wherein selecting one of the tracked operators as a primary operator comprises:
removing faces from a frame of one of the one or more images that are not present in a previous frame;
removing faces from the frame of one of the one or more images having been first detected less than a specified time period prior to the frame; and
selecting one of the remaining faces as the primary operator.
13. A system for detecting an operator's use of a secure system, the system comprising:
a detection and identification module configured to (i) acquire one or more images from one or more image sensors, the image sensors being co-located with the secure system and monitoring a detection zone about the secure system, (ii) identify one or more faces within the images, and (iii) track the faces among subsequently acquired images;
an authentication module configured to select one of the tracked faces as a primary operator in response to receiving a valid login at the secure system; and
a tracking module configured to (i) track the head of the primary operator independent of the primary operator's face in images obtained by the one or more image sensors during operation of the secure system and regardless of whether the operator is facing the one or more image sensors, and (ii) determine movements of the primary operator based on at least one of a location or a size of the tracked head,
wherein tracking the head of the primary operator independent of the primary operator's face comprises:
assigning tracking points to points on tracked head images;
periodically updating and re-assigning the tracking points;
detecting errant points based on a velocity threshold; and
applying a correction factor to the errant points, wherein the correction factor is based at least in part on an average change in position of neighboring points.
14. The system of claim 13 wherein the detection and identification module is further configured to detect re-entry of the primary operator into the detection zone after an absence of the primary operator from the detection zone.
15. The system of claim 13 wherein the tracking module is further configured to detect the absence of the primary operator from the detection zone after tracking the head of the primary operator operating the secure system.
16. The system of claim 15 wherein the tracking module is further configured to detect the absence of the primary operator from the detection zone by determining if one or more walk-away conditions exist, the walk-away conditions comprising one or more of (i) a center point of a tracked head image nearing an edge of the image, (ii) a determination that the tracked head image will be outside a frame within the image in a subsequent frame, and (iii) the tracked head image appears smaller than a size threshold.
17. The system of claim 15 wherein the tracking module is further configured to detect the re-entry of the primary operator into the detection zone by (i) detecting, within a re-entry time threshold, left and right eye positions of a newly-appearing face in an image, (ii) computing a normalized image of the image, and (iii) comparing the normalized image to one or more stored facial images, and, upon finding a match, detecting the re-entry of the primary operator into the detection zone.
18. The system of claim 15 further comprising a communications channel between multiple instances of the detection and identification module, authentication module and tracking module, allowing transmission of operator presence, location and biometric information between the instances to provide a single, aggregate measure of operator presence or absence.
19. The system of claim 13 wherein the detection and identification module is further configured to identify one or more faces by:
generating a list of candidate faces from the one or more faces;
removing invalid faces from the list of candidate faces based on one or more of a size parameter, a completeness parameter, a matching parameter and a motion parameter, thereby creating a filtered face list;
comparing the filtered face list with a previously generated face list using one or more of a size parameter, a completeness parameter, a matching parameter and a motion parameter; and
updating the previously generated face list with face information from the filtered face list.
20. The system of claim 13 wherein the detection and identification module is further configured to select one of the tracked operators as a primary operator by:
removing faces from a frame of one of the one or more images that are not present in a previous frame;
removing faces from the frame of one of the one or more images having been first detected less than a specified time period prior to the frame; and
selecting one of the remaining faces as the primary operator.