IP Library › Granted Patent US 11,126,001
Granted Patent B2
US 11,126,001 · App. 16/824,401 · Granted Sep 21, 2021

Image generating apparatus, head-mounted display, content processing system and image displaying method

Inventor: Katsushi Otsuka (Tokyo, JP)
Assignee: SONY INTERACTIVE ENTERTAINMENT INC.
G02B27/0172G02B27/0101G02B2027/014G02B2027/0123G02B2027/0178
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,126,001
App. No.
16/824,401
Granted
Sep 21, 2021
Kind
B2
Abstract

The present disclosure provides a head-mounted display that includes an image generation device generating a display image to be viewed through an eyepiece disposed in front of a display panel, the image generation device including a source image reception control section that receives a source image, a distorted image generation section that generates data on pixels in a display image obtained by giving the source image a distortion corresponding to the eyepiece, a partially distorted image storage section that stores the data on the pixels in an order of data generation, and an image display control section that, whenever data on a predetermined number of pixels smaller than a total number of pixels in the display image is stored in the partially distorted image storage section, outputs the stored data to the display panel; the display panel that sequentially displays data outputted from the image generation device; and a content processing device that generates the source image and transmits the generated source image to the head-mounted display.

Claims (123)

1. An image generation device that generates a display image to be viewed through an eyepiece disposed in front of a display panel, the image generation device comprising:

a source image reception control section that receives a source image;

a distorted image generation section that generates data on pixels in a display image obtained by giving the source image a distortion corresponding to the eyepiece;

a partially distorted image storage section that stores the data on the pixels in an order of data generation;

an image display control section that, whenever data on a predetermined number of pixels smaller than a total number of pixels in the display image is stored in the partially distorted image storage section, outputs the stored data to the display panel,

a user posture calculation section that acquires information regarding a posture of a head of a user wearing a head-mounted display having the display panel;

a user posture value storage section that stores the acquired information regarding the posture of the head of the user;

a view screen projection determination section that sets a view screen configured to define a plane of the display image in accordance with the posture of the head of the user,

wherein the distorted image generation section gives the distortion to an image projected on the view screen;

a user controller instruction reception control section that acquires a user instruction determining whether the plane of the source image is to be fixed in a virtual space of a display target or fixed to the display panel; and

a user controller input value storage section that stores the user instruction,

wherein, in a mode for fixing the plane of the source image in the virtual space of the display target, the distorted image generation section Lives the distortion to the image projected on the view screen.

2. The image generation device according to claim 1 ,

wherein the distorted image generation section generates the data on the pixels in the display image by referencing a map indicating, on an image plane, a positional relationship between pixels in a distorted display image stored in a storage section and pixels in the source image or by calculating the positional relationship.

3. The image generation device according to claim 1 , further comprising:

a data transfer control section that, whenever data on the predetermined number of pixels is stored in the partially distorted image storage section, exercises control so as to transmit the data.

4. The image generation device according to claim 2 ,

wherein the map indicates the positional relationship at discrete positions having fewer pixels than the display image, and

wherein, based on a positional relationship obtained by interpolating the positional relationship indicated by the map, the distorted image generation section generates data on all pixels in the display image.

5. The image generation device according to claim 4 , wherein the distorted image generation section generates the data on the pixels in the display image from the source image by combining a transformation based on the map determined by a structure of the eyepiece with a transformation based on a parameter not indicated by the map.

6. The image generation device according to claim 5 , wherein the distorted image generation section performs transformation by using at least either one of a pupillary distance of a user and the distance between the display panel and eyes of the user as the parameter not indicated by the map.

7. The image generation device according to claim 1 ,

wherein the partially distorted image storage section includes a plurality of storage areas each having a capacity for storing data on the predetermined number of pixels, and

wherein the distorted image generation section switches a storage location of the data on the pixels in the display image between the plurality of storage areas.

8. The image generation device according to claim 7 ,

wherein a plurality of the storage areas included in the partially distorted image storage section each have a capacity that is an integer multiple of the capacity of a unit area defined as a minimum unit of processing in the distorted image generation section.

9. The image generation device according to claim 1 ,

wherein the source image is an undistorted image generated for display on a flat-panel display.

10. A head-mounted display comprising:

an image generation device that generates a display image to be viewed through an eyepiece disposed in front of a display panel, the image generation device including

a source image reception control section that receives a source images

a distorted image generation section that generates data on pixels in a display image obtained by giving the source image a distortion corresponding to the eyepieces;

a partially distorted image storage section that stores the data on the pixels in an order of data generation;

an image display control section that, whenever data on a predetermined number of pixels smaller than a total number of pixels in the display image is stored in the partially distorted image storage section, outputs the stored data to the display panel;

the display panel that sequentially displays data outputted from the image generation device,

a user posture calculation section that acquires information regarding a posture of a head of a user wearing a head-mounted display having the display panel;

a user posture value storage section that stores the acquired information regarding the posture of the head of the user;

a view screen projection determination section that sets a view screen configured to define a plane of the display image in accordance with the posture of the head of the user,

wherein the distorted image generation section gives the distortion to an image projected on the view screen;

a user controller instruction reception control section that acquires a user instruction determining whether the plane of the source image is to be fixed in a virtual space of a display target or fixed to the display panel; and

a user controller input value storage section that stores the user instruction,

wherein, in a mode for fixing the plane of the source image in the virtual space of the display target, the distorted image generation section gives the distortion to the image projected on the view screen.

11. A content processing system comprising:

a head-mounted display that includes an image generation device generating a display image to be viewed through an eyepiece disposed in front of a display panel, the image generation device including

a source image reception control section that receives a source image,

a distorted image generation section that generates data on pixels in a display image obtained by giving the source image a distortion corresponding to the eyepiece;

a partially distorted image storage section that stores the data on the pixels in an order of data generation;

an image display control section that, whenever data on a predetermined number of pixels smaller than a total number of pixels in the display image is stored in the partially distorted image storage section, outputs the stored data to the display panel;

the display panel that sequentially displays data outputted from the image generation device;

a content processing device that generates the source image and transmits the generated source image to the head-mounted display;

a user posture calculation section that acquires information regarding a posture of a head of a user wearing a head-mounted display having the display panel;

a user posture value storage section that stores the acquired information regarding the posture of the head of the user;

a view screen projection determination section that sets a view screen configured to define a plane of the display image in accordance with the posture of the head of the user,

wherein the distorted image generation section gives the distortion to an image projected on the view screen;

a user controller instruction reception control section that acquires a user instruction determining whether the plane of the source image is to be fixed in a virtual space of a display target or fixed to the display panel, and

a user controller input value storage section that stores the user instruction,

wherein, in a mode for fixing the plane of the source image in the virtual space of the display target, the distorted image generation section Lives the distortion to the image projected on the view screen.

12. The content processing system according to claim 11 ,

wherein, when a source image transmitted from the content processing device is displayable as is, the image display control section outputs data on the source image to the display panel.

13. The content processing system according to claim 11 ,

wherein, at a time point when data on pixels in rows of a frame of the source image transmitted from the content processing device is acquired as necessary for determining one row of pixel values of the display image, the distorted image generation section starts a process of generating data on the one row of pixels.

14. An image display method used in an image generation device that generates a display image that is to be viewed through an eyepiece disposed in front of a display panel, the image display method comprising:

receiving a source image;

generating data on pixels in the display image that is obtained by giving the source image a distortion corresponding to the eyepiece, and sequentially storing the generated data in a memory;

whenever data on a predetermined number of pixels smaller than a total number of pixels in the display image is stored in the memory, outputting the stored data to the display panel;

acquiring information regarding a posture of a head of a user wearing a head-mounted display having the display panel;

storing the acquired information regarding the posture of the head of the user;

setting a view screen configured to define a plane of the display image in accordance with the posture of the head of the user;

applying the distortion to an image projected on the view screen;

acquiring a user instruction determining whether the plane of the source image is to be fixed in a virtual space of a display target or fixed to the display panel; and

storing the user instruction,

wherein in a mode for fixing the plane of the source image in the virtual space of the display target, applying the distortion to the image projected on the view screen.

15. A non-transitory computer readable medium having stored thereon a computer program for a computer generating a display image that is to be viewed through an eyepiece disposed in front of a display panel, the computer comprising:

by a source image reception control section, receiving a source image;

by a distorted image generation section, generating data on pixels in the display image that is obtained by giving the source image a distortion corresponding to the eyepiece;

by a partially distorted image storage section, sequentially storing the generated data in a memory;

by an image display control section, whenever data on a predetermined number of pixels smaller than a total number of pixels in the display image is stored in the memory, outputting the stored data to the display panel;

by a user posture calculation section, acquiring information regarding a posture of a head of a user wearing a head-mounted display having the display panel;

by a user posture value storage section, storing the acquired information regarding the posture of the head of the user;

by a view screen projection determination section, setting a view screen configured to define a plane of the display image in accordance with the posture of the head of the user,

wherein the distorted image generation section gives the distortion to an image projected on the view screen;

by a user controller instruction reception control section, acquiring a user instruction determining whether the plane of the source image is to be fixed in a virtual space of a display target or fixed to the display panel; and

by a user controller input value storage section, storing the user instruction,

wherein, in a mode for fixing the lane of the source image in the virtual space of the display target, the distorted image generation section gives the distortion to the image projected on the view screen.

16. An image generation device that generates a display image to be viewed through an eyepiece disposed in front of a display panel, the image generation device comprising:

a source image reception control section that receives a source image;

a distorted image generation section that generates data on pixels in a display image obtained by giving the source image a distortion corresponding to the eyepiece;

a partially distorted image storage section that stores the data on the pixels in an order of data generation; and

an image display control section that, whenever data on a predetermined number of pixels smaller than a total number of pixels in the display image is stored in the partially distorted image storage section, outputs the stored data to the display panel,

wherein the partially distorted image storage section includes a plurality of storage areas each having a capacity for storing data on the predetermined number of pixels, and

wherein the distorted image generation section switches a storage location of the data on the pixels in the display image between the plurality of storage areas.

17. A head-mounted display comprising:

an image generation device that generates a display image to be viewed through an eyepiece disposed in front of a display panel, the image generation device including

a source image reception control section that receives a source image,

a distorted image generation section that generates data on pixels in a display image obtained by giving the source image a distortion corresponding to the eyepiece,

a partially distorted image storage section that stores the data on the pixels in an order of data generation, and

an image display control section that, whenever data on a predetermined number of pixels smaller than a total number of pixels in the display image is stored in the partially distorted image storage section, outputs the stored data to the display panel; and

the display panel that sequentially displays data outputted from the image generation device,

wherein the partially distorted image storage section includes a plurality of storage areas each having a capacity for storing data on the predetermined number of pixels, and

wherein the distorted image generation section switches a storage location of the data on the pixels in the display image between the plurality of storage areas.

18. A content processing system comprising:

a head-mounted display that includes an image generation device generating a display image to be viewed through an eyepiece disposed in front of a display panel, the image generation device including

a source image reception control section that receives a source image,

a distorted image generation section that generates data on pixels in a display image obtained by giving the source image a distortion corresponding to the eyepiece,

a partially distorted image storage section that stores the data on the pixels in an order of data generation, and

an image display control section that, whenever data on a predetermined number of pixels smaller than a total number of pixels in the display image is stored in the partially distorted image storage section, outputs the stored data to the display panel;

the display panel that sequentially displays data outputted from the image generation device; and

a content processing device that generates the source image and transmits the generated source image to the head-mounted display,

wherein the partially distorted image storage section includes a plurality of storage areas each having a capacity for storing data on the predetermined number of pixels, and

wherein the distorted image generation section switches a storage location of the data on the pixels in the display image between the plurality of storage areas.

19. An image display method used in an image generation device that generates a display image that is to be viewed through an eyepiece disposed in front of a display panel, the image display method comprising:

receiving a source image;

generating data on pixels in the display image that is obtained by giving the source image a distortion corresponding to the eyepiece, and sequentially storing the generated data in a memory;

whenever data on a predetermined number of pixels smaller than a total number of pixels in the display image is stored in the memory, outputting the stored data to the display panel,

wherein the memory includes a plurality of storage areas each having a capacity for storing data on the predetermined number of pixels; and

switching a storage location of the data on the pixels in the display image between the plurality of storage areas.

20. A non-transitory computer readable medium having stored thereon a computer program for a computer generating a display image that is to be viewed through an eyepiece disposed in front of a display panel, the computer comprising:

by a source image reception control section, receiving a source image;

by a distorted image generation section, generating data on pixels in the display image that is obtained by giving the source image a distortion corresponding to the eyepiece;

by a partially distorted image storage section, sequentially storing the generated data in a memory; and

by an image display control section, whenever data on a predetermined number of pixels smaller than a total number of pixels in the display image is stored in the memory, outputting the stored data to the display panel,

wherein the partially distorted image storage section includes a plurality of storage areas each having a capacity for storing data on the predetermined number of pixels, and

wherein the distorted image generation section switches a storage location of the data on the pixels in the display image between the plurality of storage areas.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 29, 2020
From: OTSUKA, KATSUSHI
To: SONY INTERACTIVE ENTERTAINMENT INC.
Reel/Frame 052789/0148 →
Priority Claims (2)
JP JP2019-068187 · Mar 29, 2019 · national
JP JP2019-185339 · Oct 8, 2019 · national
Continuity (1)
Related Publication 20200310127A1 · Oct 1, 2020
Cited By (1)
US 12,332,439