Object tracking and eye state identification method
View Patent ↗An object tracking method tracks a target object between successively generated infrared video images using a grey-scale hat filter to extract the target object from the background. The filtered image is binarized, and candidate binary blobs are extracted. The binary blob that minimizes the Euclidian spatial distance to the previous position of the object and satisfies a specified appearance model is selected, and its center of mass is taken as the current position of the object. Where the object is a person's eye, the eye state and decision confidence are determined by analyzing the shape and appearance of the binary blob along with changes in its size and the previous eye state, and applying corresponding parameters to an eye state decision matrix.
1. A method of tracking movement of a human eye between first and second successively generated video images after a position of the eye in said first video image has been identified, comprising the steps of:
defining a first state vector for the first video image corresponding to the identified position of said eye;
defining a search window in said second video image based on said first state vector;
bottom hat filtering said search window with a non-flat structuring element to form a filtered image;
binarizing said filtered image and identifying candidate binary blobs that possibly correspond to said eye;
computing a spatial Euclidian distance between each candidate binary blob and said first state vector, and selecting a candidate binary blob for which the computed spatial Euclidian distance is smallest;
determining a center of mass of the selected binary blob;
defining a second state vector based on said center of mass for identifying the location of said eye in said second video image;
selecting candidate binary blobs in successively generated video images based on minimum spatial Euclidian distance;
determining a size of the selected candidate binary blobs, and computing a change in size of successively selected candidate binary blobs; and
determining an eye state designating either that said eye is open or that said eye is closed and a confidence that the determined eye state is accurate by applying a sign of said change in size and a previously determined eye state to a decision matrix when said change in size has a magnitude that exceeds a threshold magnitude.
2. The method of claim 1 , wherein the step of bottom hat filtering said search window includes the steps of:
scanning said non-flat structuring element over said search window using a first convolution function to form a dilation image;
rotating said structuring element;
scanning the rotated structuring element over said dilation image using a second convolution function to form an erosion image; and
subtracting said erosion image from said search window to form said filtered image.
3. The method of claim 1 , including the steps of:
establishing an eye model defining image characteristics of human eyes and a non-eye model defining image characteristics of facial features other than human eyes;
extracting a patch of said search window based on said center of mass;
computing deviations of said patch from said eye model and said non-eye model; and
rejecting the selected candidate binary blob when the deviation of said patch from said eye model is greater than the deviation of said patch from said non-eye model.
4. The method of claim 3 , wherein said eye model includes an open-eye mode defining image characteristics of open eyes and a closed-eye model defining image characteristics of closed eyes, and the method includes the steps of:
computing deviations of said patch from said open-eye model and said closed-eye model; and
rejecting the selected candidate binary blob when the deviation of said patch from said non-eye model is less than both the deviation of said patch from said open-eye model and the deviation of said patch from said closed-eye model.
5. The method of claim 1 , including the steps of:
establishing an open-eye model defining image characteristics of open eyes and a closed-eye model defining image characteristics of closed eyes;
computing deviations of said patch from said open-eye model and said closed-eye model;
forming a model-based determination that said eye is open when the deviation of said patch from said open-eye model is less than the deviation of said patch from said closed-eye model, and closed when the deviation of said patch from said closed-eye model is less than the deviation of said patch from said open-eye model;
determining said eye state based on said model-based determination and said sign of said change in size.
6. The method of claim 1 , including the step of:
determining a shape parameter of said selected candidate binary blob according to a difference in standard deviation calculations along orthogonal axes;
forming an appearance-based determination that said eye is open if said shape parameter is less than a predefined threshold, and closed if said shape parameter is greater than said predefined threshold; and
determining said eye state based on said appearance-based determination and said sign of said change in size.