Method and device for calibration-free gaze estimation
View Patent ↗The invention relates to a method of gaze estimation. As to determine the position of the gaze without calibrating the system used for determining the gaze, the method comprises: detecting at least a location of the center of at least an eye on at least an eye image of a viewer watching at least a video image displayed on a screen; determining at least a first position of the gaze of the viewer on the screen by using the at least a detected location of the center of the at least an eye and a mapping function based on center-bias property of human gaze distribution. The invention also relates to a device configured for estimating the gaze.
1. A method of gaze estimation, the method comprising:
converting at least an eye image of a viewer watching at least a video image displayed on a screen into at least a heat map;
detecting at least a location of a centre of at least an eye on the at least a heat map;
converting the detected at least a location of the centre of the at least an eye into at least a first position of the gaze of the viewer on said screen by using a mapping function based on centre-bias property of human gaze distribution.
2. The method according to claim 1 , wherein the detecting step further comprises:
determining first pixels of the at least a heat map having an associated value greater than a first determined threshold value;
determining second pixels located in the neighbourhood of the first pixels having an associated value greater than a second determined threshold value;
computing the at least a location of the centre of the at least an eye as corresponding to a weighted average of the location of the first and second determined pixels.
3. The method according to claim 2 , wherein the at least a heat map is represented in color space YC b C r as output of the converting.
4. The method according to claim 2 , wherein said detecting step further comprises a Gaussian filtering of said at least a heat map, the first and second pixels being determined after the Gaussian filtering.
5. The method according to claim 1 , wherein the method further comprises:
determining at least a second position of the gaze of the viewer on said screen according to at least a saliency map associated with the at least a video image;
determining at least a third position of the gaze of the viewer on said screen, the at least a third position of the gaze corresponding to a fusion of the at least a first position of the gaze and the at least a second position of the gaze.
6. The method according to claim 5 , wherein said at least a third position of the gaze is determined by using a particle filtering method with at least another first position of the gaze and at least another second position of the gaze previously determined in a temporal point of view.
7. The method according to claim 1 , wherein the at least a first position of the gaze is determined by using a particle filtering method and at least another first position of the gaze previously determined in a temporal point of view.
8. The method according to claim 1 , wherein said at least a first position of the gaze of the viewer is determined by taking into account a movement of the head of the viewer.
9. A non-transitory processor readable medium having stored therein instructions for causing a processor to perform at least the method according to one of claims 1 to 8 .
10. A device configured for determining the gaze of a viewer, the device comprising at least one processor configured for:
converting at least an eye image of a viewer watching at least a video image displayed on a screen into at least a heat map;
detecting a location of a centre of at least an eye on the at least a heat map;
converting the detected location of the centre of the at least an eye into at least a first position of the gaze of the viewer on said screen by using a mapping function based on centre-bias property of human gaze distribution.
11. The device according to claim 10 , wherein the at least one processor is further configured for:
determining first pixels of the at least a heat map having an associated value greater than a first determined threshold value;
determining second pixels located in the neighbourhood of the first pixels having an associated value greater than a second determined threshold value;
computing the at least a location of the centre of the at least an eye as corresponding to a weighted average of the location of the first and second determined pixels.
12. The device according to claim 11 , wherein the at least a heat map is represented in color space YCbCr as output of the converting.
13. The device according to claim 10 , wherein the at least one processor is further configured for filtering said at least a heat map with a Gaussian filter.
14. The device according to claim 10 , wherein the at least one processor is further configured for:
determining at least a second position of the gaze of the viewer on said screen according to at least a saliency map associated with the at least a video image;
determining at least a third position of the gaze of the viewer on said screen, the at least a third position of the gaze corresponding to a fusion of the at least a first position of the gaze and the at least a second position of the gaze.
15. The device according to claim 14 , wherein said at least a third position of the gaze is determined by using a particle filtering method with at least another first position of the gaze and at least another second position of the gaze previously determined in a temporal point of view.
16. The device according to claim 10 , wherein said at least one processor is further configured for detecting a movement of the head of the viewer.
17. The device according to claim 10 , wherein the at least a first position of the gaze is determined by using a particle filtering method and at least another first position of the gaze previously determined in a temporal point of view.