IP Library Granted Patent US 9,904,056
Granted Patent B2
US 9,904,056 · App. 15/007,531 · Granted Feb 27, 2018

Display

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,904,056
App. No.
15/007,531
Granted
Feb 27, 2018
Kind
B2
Abstract

A display method using a display operable to display an image to a viewer is provided. The method includes detecting an initial position and/or orientation of an image view. Foreground portions of an image are rendered according to the detected initial position and/or orientation. Background portions of the image are rendered according to the detected initial position and/or orientation, at an image rendering rate. The following are performed at an image display rate higher than the image rendering rate: (i) detecting a current position and/or orientation of the image view; (ii) re-projecting the background portions of a most recent image according to any differences between the initial position and/or orientation and the current position and/or orientation of the image view; and (iii) combining the foreground portions and the re-projected background portions to generate an image for display.

Claims (44)

1. A display method using a display operable to display an image to a viewer, the method comprising:

detecting, by a detector device, at least one of an initial position and an initial orientation of an image view;

rendering, by an image renderer, foreground portions of an image according to the detected at least one of the initial position and the initial orientation;

rendering, by the image renderer, background portions of the image according to the detected at least one of the initial position and the initial orientation, at an image rendering rate;

at an image display rate higher than the image rendering rate:

(i) detecting, by an output image generator, at least one of a current position and a current orientation of the image view;

(ii) re-projecting, by the output image generator, background portions of a most recent image according to any differences between either the initial position and the current position, or between the initial orientation and the current orientation of the image view; and

(iii) combining, by the output image generator, the foreground portions and the re-projected background portions to generate an image for display;

in which:

rendering the background portions comprises providing depth data indicating an image depth of one or more image features;

re-projecting the background portions comprises re-projecting the depth data by mapping the re-projected depth data to a coordinate system of the most recent image and associating pixel data derived from the most recent image with the re-projected depth data; and

in which, for any pixel position at which re-projected depth data does not exist, the depth data and pixel data are derived from the most recent image are used.

2. A method according to claim 1 , in which the most recent image comprises a most recent image for display.

3. A method according to claim 1 , in which the most recent image comprises a most recently rendered image.

4. A method according to claim 1 , further comprising associating metadata with the generated image, the metadata indicating at least one of the initial position and the initial orientation of the image view.

5. A method according to claim 1 , in which the image view is an in-game camera view of a computer game.

6. A method according to claim 5 , in which the display comprises a display screen.

7. A method according to claim 1 , in which the display is a head-mountable display and at least one of a position and an orientation of the viewer's head is detected as at least one of a position and an orientation of a given image view by detecting at least one of a position and an orientation of the head-mountable display.

8. A method according to claim 1 , in which rendering the foreground portions comprises rendering the foreground portions of the image at the image rendering rate.

9. A method according to claim 1 , in which rendering the foreground portions comprises rendering the foreground portions of the image at the image display rate.

10. A non-transitory machine-readable storage medium which stores computer readable instruction thereon that, when executed by a computer, cause the computer to perform a display method using a display operable to display an image to a viewer, the method comprising:

detecting at least one of an initial position and an initial orientation of an image view;

rendering foreground portions of an image according to the detected at least one of the initial position and the initial orientation;

rendering background portions of the image according to the detected at least one of the initial position and the initial orientation, at an image rendering rate;

at an image display rate higher than the image rendering rate:

(i) detecting at least one of a current position and a current orientation of the image view;

(ii) re-projecting background portions of a most recent image according to any differences between either the initial position and the current position, or between the initial orientation and the current orientation of the image view; and

(iii) combining the foreground portions and the re-projected background portions to generate an image for display;

in which:

rendering the background portions comprises providing depth data indicating an image depth of one or more image features;

re-projecting the background portions comprises re-projecting the depth data by mapping the re-projected depth data to a coordinate system of the most recent image and associating pixel data derived from the most recent image with the re-projected depth data; and

in which, for any pixel position at which re-projected depth data does not exist, the depth data and pixel data are derived from the most recent image are used.

11. A display system connectable to a display operable to display an image to a viewer, the display system comprising:

a detector configured to detect at least one of an initial position and an initial orientation of an image view;

an image renderer configured (i) to generate foreground portions of an image according to at least one of the detected initial position and the detected initial orientation; and (ii) to generate background portions of the image at an image rendering rate according to at least one of the detected initial position and the detected initial orientation;

an output image generator operable at an image display rate higher than the image rendering rate and configured to:

(i) detect at least one of a current position and a current orientation of the image view;

(ii) re-project background portions of a most recent image according to any differences between either the initial position and the current position, or between the initial orientation and the current orientation of the image view; and

(iii) combine the foreground portions and the re-projected background portions to generate an image for display;

in which:

the image renderer is configured to provide depth data indicating an image depth of one or more image features;

the output image generator is configured to re-project the depth data by mapping the re-projected depth data to a coordinate system of the most recent image and associating pixel data derived from the most recent image with the re-projected depth data; and

in which, for any pixel position at which re-projected depth data does not exist, the depth data and pixel data are derived from the most recent image are used.

12. A head mountable display comprising a display system according to claim 11 and one or more display devices operatively coupled to the display system.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 19, 2020
From: SONY INTERACTIVE ENTERTAINMENT EUROPE LIMITED
To: SONY INTERACTIVE ENTERTAINMENT INC.
Reel/Frame 052167/0398 →
CHANGE OF NAME Recorded Jul 14, 2017
From: SONY COMPUTER ENTERTAINMENT EUROPE LIMITED
To: SONY INTERACTIVE ENTERTAINMENT EUROPE LIMITED
Reel/Frame 043198/0110 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 4, 2016
From: RAGHOEBARDAJAL, SHARWIN WINESH; BICKERSTAFF, IAN HENRY; RIBEIRO, BRUNO
To: SONY COMPUTER ENTERTAINMENT EUROPE LIMITED
Reel/Frame 038183/0567 →