IP Library Granted Patent US 11,490,808
Granted Patent B2
US 11,490,808 · App. 16/631,901 · Granted Nov 8, 2022

Information processing device and information processing method

Inventor: Takuro Noda (Tokyo, JP)
Assignee: SONY CORPORATION
A61B3/113A61B3/0008A61B3/0025A61B3/0041A61B3/0091
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,490,808
App. No.
16/631,901
Granted
Nov 8, 2022
Kind
B2
Abstract

Provided is an information processing device including an arithmetic processing unit that executes arithmetic processing related to calibration of detection of a sight line toward a display unit. The arithmetic processing unit determines calibration executability for each of both eyes based on acquired sight line data and uses only the sight line data of an eye for which calibration is determined to be executable when executing calibration for the eye.

Claims (50)

1. An information processing device, comprising:

a marker control unit configured to acquire sight line data;

an arithmetic processing unit configured to execute arithmetic processing related to calibration of detection of a sight line of a user toward a display unit, wherein the arithmetic processing unit is further configured to:

determine calibration executability for each eye of both eyes of the user based on the acquired sight line data,

execute calibration for a first eye of both the eyes of the user, and

use the acquired sight line data of a first eye of both the eyes for which calibration is determined to be executable based on the execution of the calibration of the first eye of both the eyes;

a transparent member including at least two light emission points, wherein

each light emission point of the at least two light emission points is configured to irradiate each eye of both the eyes of the user with light guided from a light source, and

the at least two light emission points are at positions where the at least two light emission points are configured to form at least two bright spots on a cornea of each eye of both the eyes.

2. The information processing device according to claim 1 , wherein the arithmetic processing unit is further configured to determine the calibration executability for each eye of both the eyes based on a usable calibration point.

3. The information processing device according to claim 2 , wherein the arithmetic processing unit is further configured to determine that the calibration is inexecutable for a second eye of both the eyes based on a number of usable calibration points smaller than a threshold.

4. The information processing device according to claim 2 , wherein the arithmetic processing unit is further configured to determine usability of a calibration point based on the sight line data accumulated at the calibration point.

5. The information processing device according to claim 2 , wherein the arithmetic processing unit is further configured to determine that a calibration point is unusable based on a number of pieces of the sight line data accumulated at the calibration point does not exceed a specific number in a specific time.

6. The information processing device according to claim 1 , wherein the arithmetic processing unit is further configured to determine the calibration executability for each eye of both the eyes based on variance of an optical axis vector at each usable calibration point of a plurality of usable calibration points.

7. The information processing device according to claim 6 , wherein

the arithmetic processing unit is further configured to determine that the calibration is inexecutable for a second eye of both the eyes when a correlation coefficient is smaller than a threshold,

the correlation coefficient indicates a correlation relation between the optical axis vector and a marker vector, and

the marker vector is from a pupil center of the user to a usable calibration point of the plurality of usable calibration points at which a gaze point marker is displayed.

8. The information processing device according to claim 1 , wherein

the arithmetic processing unit is further configured to calculate an optical axis vector by a pupil cornea reflection method,

the pupil cornea reflection method is based on a captured image including an eye of both the eyes of the user, and

the captured image is obtained by an image capturing operation when the eye of the user is irradiated with the light from the light source and a gaze point marker is displayed at a calibration point.

9. The information processing device according to claim 1 , further comprising a display control unit configured to control a display position of an object displayed on the display unit based on the first eye for which the calibration has been executed.

10. The information processing device according to claim 9 , wherein the display control unit is further configured to display the object in an area corresponding to an eye of both the eyes of the user for which the calibration has been executed.

11. The information processing device according to claim 10 , wherein the display control unit is further configured to display the object in a calibration area for an eye of both the eyes of the user for which the calibration has been executed.

12. The information processing device according to claim 1 , further comprising an evaluation unit configured to evaluate a variance of an optical axis vector, wherein the variance is calculated for a plurality of calibration points.

13. The information processing device according to claim 1 , wherein the marker control unit is further configured to change a display position of a gaze point marker displayed by the display unit.

14. The information processing device according to claim 1 , wherein the transparent member is in a plate shape and is between each eye of the user and the display unit.

15. The information processing device according to claim 1 , wherein the light emission point is based on cutout fabrication.

16. The information processing device according to claim 1 , wherein the at least two light emission points have light emission shapes different from each other.

17. The information processing device according to claim 1 , further comprising an image capturing unit configured to capture a first image including an eyeball of the user, wherein

the image capturing unit is at a position where the image capturing unit is further configured to capture a second image of the at least two bright spots on the cornea.

18. An information processing method, comprising:

executing, by a processor,

acquiring sight line data;

arithmetic processing related to calibration of detection of a sight line of a user toward a display unit, wherein

the execution of arithmetic processing further includes:

determining calibration executability for each eye of both eyes of the user based on the acquired sight line data,

executing calibration for an eye of both the eyes of the user, and

using the acquired sight line data of the eye of both the eyes for which calibration is determined to be executable when executing calibration for the eye;

irradiating, by at least two light emission points of a transparent member, each eye of both the eyes of the user with light guided from a light source; and

forming, by the at least two emission points, at least two bright spots on a cornea of each eye of both the eyes.

19. A non-transitory computer-readable medium having stored thereon computer-executable instructions, which when executed by a computer, cause the computer to execute operations, the operations comprising:

acquiring sight line data;

executing arithmetic processing related to calibration of detection of a sight line of a user toward a display unit, wherein the execution of arithmetic processing further includes:

determining calibration executability for each eye of both eyes of the user based on the acquired sight line data,

executing calibration for an eye of both eyes of the user, and

using the acquired sight line data of the eye of both the eyes for which calibration is determined to be executable when executing calibration for the eye;

irradiating, by at least two light emission points of a transparent member, each eye of both the eyes of the user with light guided from a light source; and

forming, by the at least two emission points, at least two bright spots on a cornea of each eye of both the eyes.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 30, 2020
From: NODA, TAKURO
To: SONY CORPORATION
Reel/Frame 052258/0996 →
Priority Claims (1)
JP JP2017-145346 · Jul 27, 2017 · national
Continuity (1)
Related Publication 20200154997A1 · May 21, 2020