IP Library Granted Patent US 9,946,338
Granted Patent B2
US 9,946,338 · App. 14/168,611 · Granted Apr 17, 2018

Information processing to vary screen display based on a gaze point of the user

Inventors: Kazunori Hayashi (Tokyo, JP); Yoichiro Sako (Tokyo, JP); Takayasu Kon (Tokyo, JP); Yasunori Kamada (Kanagawa, JP); Takatoshi Nakamura (Tokyo, JP)
Assignee: SONY CORPORATION
G06F3/013G06F3/048G06F3/0485G06F2203/04806
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,946,338
App. No.
14/168,611
Granted
Apr 17, 2018
Kind
B2
Abstract

There is provided an information processing apparatus including a detection unit configured to detect a gaze point of a user in a display image displayed on a display unit, an estimation unit configured to estimate an intention of the user based on the gaze point detected by the detection unit, an image generation unit configured to generates a varying image that subtly varies from the display image to a final display image according to the intention estimated by the estimation unit, and a display control unit configured to control the display unit in a manner that the varying image generated by the image generation unit is displayed.

Claims (52)

1. An information processing apparatus, comprising:

electrical circuitry configured to:

acquire, from an imaging unit, gaze information on a first gaze point of a first user in a display image displayed on a display unit; and

control the display unit to complete, before a far part area moves into a recognizable area of a field of view of the first user, an increase of a visibility of the far part area of the display image based on movement of the first gaze point towards the far part area,

wherein the far part area of the display image is defined, based on the movement of the first gaze point, as an area positioned outside of the recognizable area of the field of view, and forward of the first gaze point in a direction of the movement of the first gaze point toward the far part area, and

wherein the recognizable area contains the first gaze point.

2. The information processing apparatus according to claim 1 , wherein the electrical circuitry is further configured to:

generate a plurality of varying images; and

control the display unit to gradually change the far part area with the plurality of varying images at a determined interval.

3. The information processing apparatus according to claim 2 , wherein the determined interval is at least 0.1 seconds.

4. The information processing apparatus according to claim 2 ,

wherein the plurality of varying images are different from each other by 1%, and

wherein the electrical circuitry is further configured to control the display unit such that a first varying image of the plurality of varying images is switched to, at the determined interval, a second varying image that varies by 1% from the first varying image of the plurality of varying images that is currently displayed.

5. The information processing apparatus according to claim 1 ,

wherein the display image includes a text part, and

wherein the electrical circuitry is further configured to control the display unit to enhance the text part to increase the visibility of the far part area.

6. The information processing apparatus according to claim 5 , wherein the electrical circuitry is further configured to control the display unit to increase readability of the text part at the far part area through at least one of enhancement of resolution, enhancement of contrast with a background image, enlargement of a letter size, or edge enhancement.

7. The information processing apparatus according to claim 1 , wherein the electrical circuitry is further configured to:

increase a scrolling speed of the display image based on an increase in a speed of movement of the first gaze point;

control the display image to scroll based on the direction of the movement of the first gaze point at the increased scrolling speed;

decrease the scrolling speed of the display image based on a decrease in the speed of movement of the first gaze point; and

control the display image to scroll based on the direction of the movement of the first gaze point at the decreased scrolling speed.

8. The information processing apparatus according to claim 1 , wherein the electrical circuitry is further configured to:

acquire, from the imaging unit, gaze information on a second gaze point of a second user in the display image;

determine whether the second user is registered as an owner of the information processing apparatus; and

decrease the visibility of the display image based on the determination that the second user is different from the owner of the information processing apparatus,

wherein the visibility of the display image is decreased to prevent gaze of the display image by the second user.

9. The information processing apparatus according to claim 1 ,

wherein the display image includes an active window and a non-active window,

wherein the electrical circuitry is further configured to increase a viewability of an overlapped area of the non-active window as the far part area of the display image,

wherein the overlapped area is overlapped with a part of the active window.

10. The information processing apparatus according to claim 9 , wherein the electrical circuitry is further configured to activate the non-active window until a determined instruction is issued, based on the first gaze point that is positioned in a determined area of the non-active window for a determined time.

11. The information processing apparatus according to claim 1 ,

wherein the display image includes a plurality of windows having respective titles,

wherein the electrical circuitry is further configured to control the display unit to rearrange the plurality of windows such that each of the respective titles is visible based on the first gaze point that moves back and forth.

12. The information processing apparatus according to claim 1 ,

wherein the information processing apparatus is one of a smartphone, a head mounted display, a digital camera, a digital video camera, a personal digital assistant, a personal computer, a notebook, a tablet terminal, a mobile phone terminal, a portable music player, a portable video processing apparatus, or a portable game console.

13. The information processing apparatus according to claim 12 , wherein

the display unit includes a screen,

the imaging unit images the first user, and

the imaging unit is arranged at a position of the information processing apparatus to image an eye of the first user while the first user views the screen.

14. The information processing apparatus according to claim 13 , further comprising an input sensor configured to receive hand-operation input of the first user.

15. The information processing apparatus according to claim 14 , wherein the display unit is a touch panel display that comprises the input sensor.

16. The information processing apparatus according to claim 1 ,

wherein the display image includes a text part, and

wherein the electrical circuitry is further configured to control the display unit to increase the number of letters in the text part based on a speed of movement of the first gaze point.

17. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to execute operations, the operations comprising:

acquiring, from an imaging unit, gaze information on a gaze point of a user in a display image displayed on a display unit; and

controlling the display unit to complete, before a far part area moves into a recognizable area of a field of view of the user, an increase of a visibility of the far part area of the display image based on movement of the gaze point towards the far part area,

wherein the far part area of the display image is defined, based on the movement of the gaze point, as an area positioned outside of the recognizable area of the field of view, and forward of the gaze point in a direction of the movement of the gaze point toward the far part area, and

wherein the recognizable area contains the gaze point.

18. The information processing apparatus according to claim 1 , wherein the electrical circuitry is further configured to determine a gaze intention of the first user based on whether the first user stares the display image for a determined time.

Assignments (2)
CHANGE OF NAME Recorded Dec 17, 2025
From: SONY CORPORATION
To: SONY GROUP CORPORATION
Reel/Frame 073986/0571 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 30, 2014
From: HAYASHI, KAZUNORI; SAKO, YOICHIRO; KON, TAKAYASU; KAMADA, YASUNORI; NAKAMURA, TAKATOSHI
To: SONY CORPORATION
Reel/Frame 032141/0369 →
Priority Claims (1)
JP 2013-027673 · Feb 15, 2013 · national
Continuity (1)
Related Publication 20140232639A1 · Aug 21, 2014