Apparatus and method for estimating gazed position of person

1. An image processing apparatus comprising:

one or more processors; and

a memory coupled to the one or more processors, the memory having stored thereon instructions which, when executed by the one or more processors, cause the image processing apparatus to:

detect, respectively from plural images obtained by imaging with plural imaging units a space in which plural persons exist, positions of the persons on the images;

perform association of a same person in the plural images;

estimate positions of the persons in the space, based on the positions of the persons on the images detected from the plural images and a result of the association;

estimate eye directions of the persons, based on the plural images;

estimate a spatial distribution of positions gazed by the plural persons, based on the estimated positions of the plural persons in the space and the estimated eye directions of the plural persons; and

estimate a gazing level for each object, based on the estimated spatial distribution of the gazed positions.

2. The image processing apparatus according to claim 1 , wherein execution of the instructions further causes the image processing apparatus to obtain respective imaging parameters of the plural imaging units.

3. The image processing apparatus according to claim 2 , wherein execution of the instructions further causes the image processing apparatus to estimate the position of the person in the space, based on the positions of the persons on the image and the respective imaging parameters of the plural imaging units.

4. The image processing apparatus according to claim 2 , wherein execution of the instructions further causes the image processing apparatus to estimate the spatial distribution of the positions gazed by the persons, based on the positions of the persons in the space, the eye directions and the respective imaging parameters of the plural imaging units.

5. The image processing apparatus according to claim 2 , wherein execution of the instructions further causes the image processing apparatus to obtain internal parameters and external parameters of the plural imaging units calculated from the plural images respectively imaged by the plural imaging units, as the imaging parameters.

6. The image processing apparatus according to claim 5 , wherein the internal parameter includes at least one of information indicating a central position of the imaging unit in an image coordinate system, information indicating a magnification, and information indicating distortion of two axes of image.

7. The image processing apparatus according to claim 5 , wherein the external parameter includes information indicating a position or an orientation of the imaging unit.

8. The image processing apparatus according to claim 1 , wherein execution of the instructions further causes the image processing apparatus to estimate the eye direction of the person, based on a partial image representing a human body or a part thereof in each of the plural images.

9. The image processing apparatus according to claim 8 , wherein execution of the instructions further causes the image processing apparatus to use a partial image of at least one of the human body, a face and eyes, as the partial image representing the human body or the part thereof in each of the plural images.

10. The image processing apparatus according to claim 1 , wherein execution of the instructions further causes the image processing apparatus to decide whether or not the association is correct, based on the position of the person in the space and the eye direction; and

estimate the spatial distribution of the positions gazed by the persons, based on the positions of the persons in the space of which the association has been decided as being correct, and the eye directions.

11. The image processing apparatus according to claim 10 , wherein execution of the instructions further causes the image processing apparatus to calculate a value indicating consistency of the association, and decides that the association is correct, in a case where the calculated value is larger than a threshold.

12. The image processing apparatus according to claim 10 , wherein the instructions further cause the image processing apparatus to calculate a value indicating consistency of the association by integrating a value indicating consistency of the positions of the persons in the space and a value indicating consistency of the eye directions.

13. An image processing method comprising:

detecting, respectively from plural images obtained by imaging with plural imaging units a space in which plural persons exist, positions of the persons on the images;

performing association of a same person in the plural images;

estimating positions of the persons in the space, based on the positions of the persons on the images detected from the plural images and a result of the association;

estimating eye directions of the persons, based on the plural images;

estimating a spatial distribution of positions gazed by the plural persons, based on the estimated positions of the plural persons in the space and the estimated eye directions of the plural persons; and

estimating a gazing level for each object, based on the estimated spatial distribution of the gazed positions.

14. A non-transitory computer-readable storage medium which stores a program to cause a computer to perform:

a detecting step of detecting, respectively from plural images obtained by imaging with plural imaging units a space in which plural persons exist, positions of the persons on the images;

an associating step of performing association of a same person in the plural images;

a person position estimating step of estimating positions of the persons in the space, based on the positions of the persons on the images detected from the plural images and a result of the association in the associating step;

a direction estimating step of estimating eye directions of the persons, based on the plural images;

a distribution estimating step of estimating a spatial distribution of positions gazed by the plural persons, based on the estimated positions of the plural persons in the space and the estimated eye directions of the plural persons; and

a gazing level estimating step of estimating a gazing level for each object, based on the estimated spatial distribution of the gazed positions.