IP Library Granted Patent US 9,618,759
Granted Patent B2
US 9,618,759 · App. 14/418,533 · Granted Apr 11, 2017

Image display apparatus and image display method

Inventors: Takafumi Morifuji (Tokyo, JP); Masami Ogata (Kanagawa, JP)
Assignee: Sony Corporation
G02B27/2264G02B27/0093G02B27/0172G06F3/013G06T19/006G09G3/003H04N13/004H04N13/0022H04N13/0033H04N13/0051H04N13/0239H04N13/044H04N13/0484H04N13/0497G02B2027/011G02B2027/0143G02B2027/0187H04N13/0434H04N13/0438
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,618,759
App. No.
14/418,533
Granted
Apr 11, 2017
Kind
B2
Abstract

A display image is superimposed and displayed on an outside image in a preferred manner. An optical system superimposes a display image displayed on a display device onto an outside image, and leads the display image to an eye of an observer. A display control unit controls the display size and the display position of the display image on the display device so that the display image is displayed in an image superimposition region (a flat region) detected from the outside image. For example, the display control unit controls the display state of the display image in accordance with the state of the image superimposition region in the outside image. Also, the display control unit performs control to selectively display the display image in a line-of-sight region or outside the line-of-sight region in accordance with the line of sight of the observer.

Claims (49)

1. An image display apparatus, comprising:

an optical system configured to superimpose a display image, displayed on a display device, onto a real-world image; and

a display control unit configured to temporal smooth, by a plurality or coordinate filters, and to control a size and a position of the display image displayed on the display device so that the display image is displayed in an image superimposition region in the real-world image.

2. The image display apparatus according to claim 1 , wherein the image superimposition region is detected based on captured image data obtained by the real-world image.

3. The image display apparatus according to claim 1 , wherein the display control unit is configured to control the size and the position of the display image based on image data to display the display image on the display device based on information about the image superimposition region.

4. The image display apparatus according to claim 1 , wherein the display control unit is further configured to change a display state of the display image based on a state of the image superimposition region in the real-world image.

5. The image display apparatus according to claim 4 , wherein the display control unit is further configured to correct image data to display the display image based on the state of the image superimposition region so that an element of the real-world image is removed from the display image to be observed by an observer.

6. The image display apparatus according to claim 1 , wherein, based on a determination that the image superimposition region is undetected from the real-world image, the display control unit changes manners of display of the display image.

7. The image display apparatus according to claim 1 , wherein the display control unit is configured to control the size and the position based on the temporal smoothing on sizes and positions on the display device, the sizes and positions are determined by the image superimposition region that is cyclically detected.

8. The image display apparatus according to claim 1 , wherein, in an event a change in the real-world image is detected, the display control unit changes manners of display of the display image.

9. The image display apparatus according to claim 1 , wherein

the optical system includes a first optical system configured to superimpose a left-eye image displayed on a first display device onto the real-world image, and a second optical system configured to superimpose a right-eye image displayed on a second display device onto the real-world image, and

wherein the display control unit is further configured to control disparities of the left-eye image and the right-eye image so that a depth position of a stereoscopic image to be perceived by an observer through the left-eye image and the right-eye image is located closer to a front side than a depth position of a region in which the stereoscopic image is displayed in a superimposed manner in the real world image.

10. The image display apparatus according to claim 1 , wherein the plurality of coordinate filters include average filter, majority filter or infinite impulse response (IIR) filter.

11. An image display method, comprising:

superimposing a display image displayed on a display device, onto a real-world image; and

temporal smoothing, by a plurality of coordinate filters, and controlling a size and a position of the display image displayed on the display device so that the display image is displayed in an image superimposition region in the real-world image.

12. An image display apparatus, comprising:

an optical system configured to superimpose a display image, displayed on a display device, onto a real-world image; and

a display control unit that includes a first control mode in which the display image is displayed in a region on which a line of sight of an observer concentrates in the real-world image, and a second control mode in which the display image is displayed in a region outside the region on which the line of sight of the observer concentrates in the real-world image,

wherein the display control unit is configured to temporal smooth, by a plurality of coordinate filters, and to control a size and a position of the display image display on the display device.

13. The image display apparatus according to claim 12 , wherein in the first control mode the observer is stable, and in the second control mode the observer is moving.

14. The image display apparatus according to claim 12 , wherein the display control unit is further configured to change a display state of the image based on a state of a region on which the display image is to be superimposed in the real-world image.

15. The image display apparatus according to claim 12 , wherein

the optical system includes a first optical system configured to superimpose a left-eye image displayed on a first display device onto the real-world image, and a second optical system configured to superimpose a right-eye image displayed on a second display device onto the real-world image, and

wherein the display control unit is further configured to control disparities of the left-eye image and the right-eye image so that a depth position of a stereoscopic image to be perceived by the observer through the left-eye image and the right-eye image is located closer to a front side than a depth position of a region in which the stereoscopic image is displayed in a superimposed manner in the real-world image.

16. An image display method, comprising:

superimposing a display image, displayed on a display device, onto a real-world image;

controlling display the display image in a region on which a line of sight of an observer concentrates in the real-world image, and display of the display image in a region outside the region on which the line of sight of the observer concentrates in the real-world image; and

temporal smoothing, by a plurality of coordinate filters, and controlling a size and a position of the display image displayed on the display device.

17. An image display apparatus, comprising:

an optical system configured to superimpose a display image, displayed on a display device onto a real-world image; and

a display control unit configured to change a display state of the display image based on a state of a region on which the display image is superimposed in the real-world image,

wherein the display control unit is further configured to temporal smooth, by a plurality of coordinate filters, and to control a size and a position of the display image displayed on the display device.

18. The image display apparatus according to claim 17 , wherein the display control unit is further configured to acquire the state of the region in the real-world image based on captured image data obtained by the real-world image.

19. An image display method, comprising:

superimposing a display image, displayed on a display device, onto a real-world image;

changing a display state of the display image based on a state of a region on which the display image is superimposed in the real-world image; and

temporal smoothing, by a plurality of coordinate filters, and controlling a size and a position of the display image displayed on the display device.

20. An image display apparatus, comprising:

a first optical system configured to superimpose a left-eye image, displayed on a first display device, onto a real-world image;

a second optical system configured to superimpose a right-eye image, displayed on a second display device, onto the real-world image; and

a display control unit configured to control disparities of the left-eye image and the right-eye image so that a depth position of a stereoscopic image to be perceived by an observer through the left-eye image and the right-eye image is located closer to a front side than a depth position of a region in which the stereoscopic image is displayed in a superimposed manner in the real-world image,

wherein the display control unit is further configured to temporal smooth, by a plurality of coordinate filters, and to control a size and a position of the left-eye image displayed on the first display device and the right-eye image displayed on the second display device.

21. An image display method, comprising:

superimposing a left-eye image, displayed on a first display device, onto a real-world image;

superimposing a right-eye image, displayed on a second display device, onto the real-world image;

controlling disparities of the left-eye image and the right-eye image so that a depth position of a stereoscopic image to be perceived by an observer through the left-eye image and the right-eye image is located closer to a front side than a depth position of a region in which the stereoscopic image is displayed in a superimposed manner in the real-world image; and

temporal smoothing, by a plurality of ordinate filters, and controlling a size and a position of the left-eye image displayed on the first device and the right-eye image displayed on the second display device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 20, 2015
From: MORIFUJI, TAKAFUMI; OGATA, MASAMI
To: SONY CORPORATION
Reel/Frame 035059/0144 →
Priority Claims (1)
JP 2012-173556 · Aug 6, 2012 · national
Continuity (1)
Related Publication 20150261003A1 · Sep 17, 2015