Information processing device, information processing method, and computer-readable recording medium
An information processing device includes an eyeball information acquisition section, a sightline estimation section, a display control section, a gesture information acquisition section, and a mode switching section. The eyeball information acquisition section acquires position and posture information of eyeballs of a user from a first sensor. The sightline estimation section estimates a sightline vector on the basis of the position and posture information of the eyeballs. The display control section controls a display device in such a manner that the display device displays a guide image for showing a gaze state based on the sightline vector. The gesture information acquisition section acquires gesture information, which is different from the position and posture information of the eyeballs, from a second sensor. The mode switching section switches between a first mode and a second mode according to mode switching input provided by the user with regard to the guide image, the first mode being a mode of reflecting the position and posture information of the eyeballs in the guide image, the second mode being a mode of correcting the gaze state on the basis of the gesture information.
1. An information processing device, comprising:
an eyeball information acquisition section configured to acquire position and posture information of eyeballs of a user from a first sensor;
a sightline estimation section configured to estimate a sightline vector of the user on a basis of the position and posture information of the eyeballs;
a display control section configured to control a display device in such a manner that the display device displays a guide image for showing a gaze state based on the estimated sightline vector;
a gesture information acquisition section configured to acquire gesture information of the user, which is different from the position and posture information of the eyeballs, from a second sensor; and
a mode switching section configured to switch between a first mode and a second mode according to mode switching input provided by the user with regard to the guide image, the first mode being a mode of reflecting the position and posture information of the eyeballs in the guide image, the second mode being a mode of correcting the gaze state on a basis of the gesture information, wherein
the gesture information includes information related to head gesture when the user rotates his/her head,
the display control section is further configured to set a rotation angle of the sightline vector according to a rotation angle of the head of the user, in such a manner that the rotation angle of the sightline vector falls within a first range, and
the eyeball information acquisition section, the sightline estimation section, the display control section, the gesture information acquisition section, and the mode switching section are each implemented via at least one processor.
2. The information processing device according to claim 1 , wherein
the guide image is an image that shows at least one of a gaze position based on the sightline vector or a gaze object including the gaze position as the gaze state, and
the display control section is further configured to correct the gaze position on a basis of the gesture information in a case where the second mode is performed.
3. The information processing device according to claim 2 , wherein
the display control section is further configured to control the display device in such a manner that the display device displays the guide image including a corrected gaze position in a case where the second mode is performed.
4. The information processing device according to claim 2 , wherein
the display control section is further configured to correct the gaze position by correcting the sightline vector on a basis of the gesture information.
5. The information processing device according to claim 2 , wherein
the display control section is further configured to control the display device in such a manner that the display device displays the guide image including the gaze position that is corrected according to rotation of the head of the user in a case where the second mode is performed.
6. The information processing device according to claim 5 , wherein
the display control section is further configured to rotate the sightline vector in a rotation direction of the head of the user.
7. The information processing device according to claim 5 , wherein
the display control section is further configured to set the rotation angle of the sightline vector to a maximum angle in the first range in a case where the rotation angle of the head of the user exceeds a second range.
8. The information processing device according to claim 7 , wherein
the second range is set on a basis of at least one of estimation accuracy of the sightline vector, calibration data of the sightline vector, density of candidate objects serving as candidates for the gaze object, or posture of the head of the user when the second mode is performed.
9. The information processing device according to claim 5 , wherein
the rotation angle of the head of the user is calculated on a basis of a rotation position of the head of the user when the second mode is performed.
10. The information processing device according to claim 5 , wherein
the display control section is further configured to continuously move the gaze position by continuously rotating the sightline vector in a rotation direction of the head of the user regardless of a rotation angle of the head of the user.
11. The information processing device according to claim 10 , wherein
the display control section is further configured to control the display device in such a manner that the display device displays the guide image, and
the gaze object is switched according to the gaze position, the gaze position being continuously moving.
12. The information processing device according to claim 5 , wherein
the display control section is further configured to calculate a relative rotation angle of the head of the user to a moving candidate object serving as a candidate for the gaze object, on a basis of a rotation speed of the head of the user, which moves in tandem with movement of the candidate object.
13. The information processing device according to claim 2 , wherein
the mode switching section is further configured to determine the mode switching input on a basis of an initiation condition and a termination condition of the second mode.
14. The information processing device according to claim 13 , wherein
the initiation condition includes at least one of a condition for a time period where the gaze object is gazed at, a condition for presence/absence of operation input provided by the user, or a condition for an amount of change in the sightline vector.
15. The information processing device according to claim 13 , wherein
the termination condition includes at least one of a condition for behavior of the eyeballs of the user, a condition for a rotation speed of the head of the user, a condition for presence/absence of operation input provided by the user, and a condition for an amount of change in the sightline vector.
16. The information processing device according to claim 1 , wherein
the gesture information includes at least one of information related to a body gesture when the user is tilting his/her upper body or information related to a hand gesture when the user is moving his/her hand.
17. The information processing device according to claim 1 , wherein
the display control section is further configured to calculate a correction amount for correcting the gaze state in the second mode, and perform display of the guide image in the first mode on a basis of the correction amount.
18. The information processing device according to claim 1 , wherein
the display control section is further configured to select an object having a smallest degree of separation from the sightline vector, as a gaze target of the user from among one or more objects serving as the gaze targets.
19. The information processing device according to claim 18 , wherein
the degree of separation from the sightline vector is an angular interval between the sightline vector and a position vector of the object.
20. The information processing device according to claim 1 , wherein
the guide image includes one or more objects serving as gaze targets of the user,
the display control section is further configured to control the display device in such a manner that the display device displays the guide image, and
display of respective objects changes according to degrees of separation between one or more objects and the sightline vector.
21. The information processing device according to claim 20 , wherein
the degree of separation from the sightline vector is an angular interval between the sightline vector and a position vector of the object, and
the display control section is further configured to change at least one of color, size, shape, or behavior of the one or more objects according to the angular interval.
22. The information processing device according to claim 20 , wherein
the display control section is further configured to perform one of a blur process and a hiding process on an object that is not included in a predetermined angular range based on the sightline vector, among the one or more objects.
23. An information processing method comprising:
acquiring position and posture information of eyeballs of a user from a first sensor;
estimating a sightline vector of the user on a basis of the position and posture information of the eyeballs;
controlling a display device in such a manner that the display device displays a guide image for showing a gaze state based on the estimated sightline vector;
acquiring gesture information of the user, which is different from the position and posture information of the eyeballs, from a second sensor;
switching between a first mode and a second mode according to mode switching input provided by the user with regard to the guide image, the first mode being a mode of reflecting the position and posture information of the eyeballs in the guide image, the second mode being a mode of correcting the gaze state on a basis of the gesture information, the gesture information including information related to head gesture when the user rotates his/her head; and
setting a rotation angle of the sightline vector according to a rotation angle of the head of the user, in such a manner that the rotation angle of the sightline vector falls within a first range.
24. A non-transitory computer-readable medium having embodied thereon a program, which when executed by a computer causes the computer to execute an information processing method, the method comprising:
acquiring position and posture information of eyeballs of a user from a first sensor;
estimating a sightline vector of the user on a basis of the position and posture information of the eyeballs;
controlling a display device in such a manner that the display device displays a guide image for showing a gaze state based on the estimated sightline vector;
acquiring gesture information of the user, which is different from the position and posture information of the eyeballs, from a second sensor;
switching between a first mode and a second mode according to mode switching input provided by the user with regard to the guide image, the first mode being a mode of reflecting the position and posture information of the eyeballs in the guide image, the second mode being a mode of correcting the gaze state on a basis of the gesture information, the gesture information including information related to head gesture when the user rotates his/her head; and
setting a rotation angle of the sightline vector according to a rotation angle of the head of the user, in such a manner that the rotation angle of the sightline vector falls within a first range.