IP Library › Granted Patent US 11,487,354
Granted Patent B2
US 11,487,354 · App. 16/981,055 · Granted Nov 1, 2022

Information processing apparatus, information processing method, and program

Inventor: Kenji Sugihara (Kanagawa, JP)
Assignee: SONY CORPORATION
G06F3/013G02B27/0093G02B2027/0178
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,487,354
App. No.
16/981,055
Granted
Nov 1, 2022
Kind
B2
Abstract

There is provided an information processing apparatus, an information processing method, and a program that make it possible to improve usability in information presentation. The information processing apparatus includes a recognition unit configured to recognize a shift in gaze and a move of a head of a user; and a display controller configured to control display of information on a subject according to the move of the head that is made after the gaze is turned to the subject.

Claims (49)

1. An information processing apparatus comprising:

a recognition unit configured to recognize a shift in gaze and a move of a head of a user; and

a display controller configured to

control display of information on a subject according to the move of the head that is made after the gaze is turned to the subject, and

make a simplified display of information on the subject in a head orientation peripheral region and a gaze direction peripheral region, based on the head orientation peripheral region overlapping with the gaze direction peripheral region,

wherein the head orientation peripheral region is peripheral to a region set based on an orientation of the head,

wherein the gaze direction peripheral region is peripheral to another region set based on a direction of the gaze,

wherein a real object is a subject of interest on which it is estimated that the user is interested in the subject of interest,

wherein the display controller is further configured to

estimate that a real object that is positioned in a gaze direction central region that is a central region of a region based on the direction of the gaze is the subject of interest, and

make a simplified display of information on the subject of interest in the head orientation peripheral region based on the orientation of the head and the gaze direction peripheral region based on the direction of the gaze, and

wherein the recognition unit and the display controller are each implemented via at least one processor.

2. The information processing apparatus according to claim 1 , wherein the display controller is further configured to perform display control stepwise on a display mode and information granularity of the information on the subject according to a change in the shift in gaze and the move of the head with respect to the subject.

3. The information processing apparatus according to claim 1 , wherein the subject is a real object or a virtual object, and

the display controller is further configured to, when the subject is the virtual object, control a position of display of the virtual object based on the direction of the gaze and the orientation of the head.

4. The information processing apparatus according to claim 3 , wherein the virtual object is notification information, and

the display controller is further configured to make a simplified display of the notification information in the gaze direction peripheral region based on the direction of the gaze and the head orientation peripheral region based on the orientation of the head.

5. The information processing apparatus according to claim 3 , wherein the real object is a subject of interest on which it is estimated that the user is interested in the subject of interest, and

the display controller is further configured to

estimate that a real object that is positioned in a gaze direction central region that is a central region of a region based on the direction of the gaze is the subject of interest, and

make a simplified display of information on the subject of interest in the head orientation peripheral region based on the orientation of the head and the gaze direction peripheral region based on the direction of the gaze.

6. The information processing apparatus according to claim 4 , wherein the display controller is further configured to determine whether a condition for ending the simplified display is met based on at least any one of the shift in the gaze and the move of the head and, when the condition for ending the simplified display is met, end the simplified display.

7. The information processing apparatus according to claim 6 , wherein the condition for ending the simplified display is based on whether the gaze is on the simplified display for a certain time.

8. The information processing apparatus according to claim 7 , wherein the display controller is further configured to end the simplified display when the gaze is turned to the simplified display but the gaze shifts before the certain time elapses.

9. The information processing apparatus according to claim 4 , wherein the display controller is further configured to perform display control to enhance the simplified display when the simplified display is positioned in a gaze direction central region that is a central region of a region based on the direction of the gaze and a certain time elapses.

10. The information processing apparatus according to claim 4 , wherein the display controller is further configured to switch the simplified display to a detailed display when the head is turned to the simplified display and the simplified display is positioned in a gaze direction central region that is a central region in the region based on the direction of the gaze and in a head orientation central region that is a central region of the region based on the orientation of the head.

11. The information processing apparatus according to claim 10 , wherein the display controller is further configured to determine whether a condition for ending the detailed display is met based on at least any one of the shift in gaze and the move of the head and, when the condition for ending the detailed display is met, end the detailed display.

12. The information processing apparatus according to claim 11 , wherein the condition for ending the detailed display is based on whether the gaze is on the detailed display for a certain time.

13. The information processing apparatus according to claim 12 , wherein the display controller is further configured to end the detailed display when the gaze shifts from the detailed display before the certain time elapses.

14. The information processing apparatus according to claim 1 , wherein the recognition unit is further configured to recognize the shift in gaze based on a move of an eye of the user.

15. An information processing method comprising:

by a processor,

recognizing a shift in gaze and a move of a head of a user;

controlling display of information on a subject according to the move of the head that is made after the gaze is turned to the subject;

making a simplified display of information on the subject in a head orientation peripheral region and a gaze direction peripheral region, based on the head orientation peripheral region overlapping with the gaze direction peripheral region,

wherein the head orientation peripheral region is peripheral to a region set based on an orientation of the head,

wherein the gaze direction peripheral region is peripheral to another region set based on a direction of the gaze, and

wherein a real object is a subject of interest on which it is estimated that the user is interested in the subject of interest;

estimating that a real object that is positioned in a gaze direction central region that is a central region of a region based on the direction of the gaze is the subject of interest; and

making a simplified display of information on the subject of interest in the head orientation peripheral region based on the orientation of the head and the gaze direction peripheral region based on the direction of the gaze.

16. A non-transitory computer-readable medium having embodied thereon a program, which when executed by a computer causes the computer to execute an information processing method, the method comprising:

recognizing a shift in gaze and a move of a head of a user;

controlling display of information on a subject according to the move of the head that is made after the gaze is turned to the subject;

making a simplified display of information on the subject in a head orientation peripheral region and a gaze direction peripheral region, based on the head orientation peripheral region overlapping with the gaze direction peripheral region,

wherein the head orientation peripheral region is peripheral to a region set based on an orientation of the head,

wherein the gaze direction peripheral region is peripheral to another region set based on a direction of the gaze, and

wherein a real object is a subject of interest on which it is estimated that the user is interested in the subject of interest;

estimating that a real object that is positioned in a gaze direction central region that is a central region of a region based on the direction of the gaze is the subject of interest; and

making a simplified display of information on the subject of interest in the head orientation peripheral region based on the orientation of the head and the gaze direction peripheral region based on the direction of the gaze.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 15, 2020
From: SUGIHARA, KENJI
To: SONY CORPORATION
Reel/Frame 053782/0589 →
Priority Claims (1)
JP JP2018-061172 · Mar 28, 2018 · national
Continuity (1)
Related Publication 20210004081A1 · Jan 7, 2021