IP Library › Granted Patent US 10,874,948
Granted Patent B2
US 10,874,948 · App. 16/459,737 · Granted Dec 29, 2020

Apparatus and method of mapping a virtual environment

Inventors: Nicholas Anthony Edward Ryan (London, GB); Hugh Alexander Dinsdale Spencer (London, GB); Andrew Swann (London, GB); Simon Andrew St John Brislin (Edinburgh, GB); Pritpal Singh Panesar (London, GB)
Assignee: Sony Interactive Entertainment Europe Limited
A63F13/86A63F13/497A63F13/525A63F13/5258A63F13/5378G06K9/00718G06K9/00744G06T7/543G06T7/75G11B20/10527A63F2300/203G06K2009/00738G11B2020/1062G11B2020/10592
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,874,948
App. No.
16/459,737
Granted
Dec 29, 2020
Kind
B2
Abstract

A method of mapping a virtual environment includes: obtaining a first sequence of video images output by a videogame title; obtaining a corresponding sequence of in-game virtual camera positions at which the video images were created; obtaining a corresponding sequence of depth buffer values for a depth buffer used by the videogame whilst creating the video images; and, for each of a plurality of video images and corresponding depth buffer values of the obtained sequences, obtain mapping points corresponding to a selected predetermined set of depth values corresponding to a predetermined set of positions within a respective video image; where for each pair of depth values and video image positions, a mapping point has a distance from the virtual camera position based upon the depth value, and a position based upon the relative positions of the virtual camera and the respective video image position, thereby obtaining a map dataset of mapping points corresponding to the first sequence of video images.

Claims (47)

1. A method of mapping a virtual environment, comprising the steps of:

obtaining a first sequence of video images output by a videogame title;

obtaining a corresponding sequence of in-game virtual camera positions at which the video images were created;

obtaining a corresponding sequence of depth buffer values for a depth buffer used by the videogame whilst creating the video images;

for each of a plurality of video images and corresponding depth buffer values of the obtained sequences, computing mapping points corresponding to a selected predetermined set of depth values corresponding to a predetermined set of positions within a respective video image; wherein for each pair of depth values and video image positions, a mapping point has a distance from the virtual camera position based upon the depth value, and a position based upon the relative positions of the virtual camera and the respective video image position, thereby obtaining a first map dataset of mapping points corresponding to the first sequence of video images; and

generating a graphical representation of some or all of the mapping points of at least the first map dataset, and outputting the graphical representation by the videogame for display.

2. The method according to claim 1 , wherein the predetermined set of positions within a respective video image comprises pixels sampled from one or more horizontal lines across the respective image.

3. The method according to claim 1 , wherein the predetermined set of positions within a respective video image comprises a sampling distribution of positions over the area of the video image.

4. The method according to claim 1 , comprising the step of

obtaining colour information at the predetermined set of positions within the respective video image; and

associating the colour information with the corresponding generated mapping points.

5. The method according to claim 1 , comprising the steps of:

recording the first sequence of video images output by the videogame;

recording the corresponding sequence of in-game virtual camera positions used to create the video images;

recording the corresponding sequence of depth buffer values for a depth buffer used by the videogame whilst creating the video images; and

recording the mapping points, and

associating the sequences of in game virtual camera positions, depth buffer values and the mapping points with the recording of the first sequence of video images.

6. The method according to claim 5 , comprising the steps of generating a second sequence of video images encoding the depth buffer value sequence.

7. The method according to claim 1 , comprising the step of: obtaining one or more additional map datasets generated using a sequence of video images, virtual camera positions and depth buffer values originating from a separate instance of the same videogame.

8. The method according to claim 1 , comprising the step of displaying the graphical representation on a display screen.

9. The method according to claim 1 , wherein the sequence of video images is obtained from a first video recording, with which the corresponding virtual camera positions and sequence of depth buffer values are associated.

10. The method according to claim 9 , wherein the corresponding sequence of depth buffer values is obtained from a second video recording that was generated by encoding the sequence of depth buffer values.

11. The method according to claim 9 , wherein the mapping points are obtained from data associated with a first video recording comprising the sequence of video images, or a second video recording comprising the sequence of depth buffer values.

12. The method according to claim 9 , comprising the steps of

generating the graphical representation of some or all of the mapping points of at least the first map dataset for display with the first video recording;

selecting a position on the graphical representation using a user interface; and

controlling a playback position of the first video recording by selecting a video frame whose corresponding camera position is closest to the selected position on the graphical representation.

13. The method according to claim 9 , comprising the steps of

generating at least part of a first map comprising a graphical representation of some or all of the mapping points of at least the first map dataset for display with the first video recording;

generating at least part of a second map comprising a graphical representation of some or all of the mapping points of at least a second map dataset associated with a different video recording of a different sequence of video images output by the same videogame title and sharing the same in-game co-ordinate system;

displaying at least part of the first map during playback of the first video recording, the displayed part encompassing at least the current virtual camera position associated with the displayed video image;

and displaying at least part of the second map during playback of the first video recording if the respective part of the second map is within a predetermined range of the current virtual camera position in the in-game co-ordinate system.

14. The method according to claim 13 , comprising the steps of:

detecting whether a user interacts with a displayed part of the second map, and if so,

switching to playback of the corresponding second video.

15. A non-transitory, computer readable medium having computer executable instructions stored thereon, which when executed by a computer system to map a virtual environment by carrying out actions, comprising:

obtaining a first sequence of video images output by a videogame title;

obtaining a corresponding sequence of in-game virtual camera positions at which the video images were created;

obtaining a corresponding sequence of depth buffer values for a depth buffer used by the videogame whilst creating the video images;

for each of a plurality of video images and corresponding depth buffer values of the obtained sequences, computing mapping points corresponding to a selected predetermined set of depth values corresponding to a predetermined set of positions within a respective video image; wherein for each pair of depth values and video image positions, a mapping point has a distance from the virtual camera position based upon the depth value, and a position based upon the relative positions of the virtual camera and the respective video image position, thereby obtaining a first map dataset of mapping points corresponding to the first sequence of video images; and

generating a graphical representation of some or all of the mapping points of at least the first map dataset, and outputting the graphical representation by the videogame for display.

16. An entertainment device, comprising:

a video data processor configured to obtain a first sequence of video images output by a videogame title;

a camera position data processor configured to obtain a corresponding sequence of in-game virtual camera positions at which the video images were created;

a depth data processor configured to obtain a corresponding sequence of depth buffer values for a depth buffer used by the videogame whilst creating the video images;

a mapping data processor is configured, for each of a plurality of video images and corresponding depth buffer values of the obtained sequences, to compute mapping points corresponding to a selected predetermined set of depth values corresponding to a predetermined set of positions within a respective video image; wherein for each pair of depth values and video image positions, a mapping point has a distance from the virtual camera position based upon the depth value, and a position based upon the relative positions of the virtual camera and the respective video image position, thereby obtaining a first map dataset of mapping points corresponding to the first sequence of video images; and

generating a graphical representation of some or all of the mapping points of at least the first map dataset, and outputting the graphical representation by the videogame for display.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 6, 2019
From: SONY INTERACTIVE ENTERTAINMENT INC.
To: SONY INTERACTIVE ENTERTAINMENT EUROPE LIMITED
Reel/Frame 049970/0803 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 26, 2019
From: RYAN, NICHOLAS ANTHONY EDWARD; SPENCER, HUGH ALEXANDER DINSDALE; SWANN, ANDREW; BRISLIN, SIMON ANDREW ST JOHN; PANESAR, PRITPAL SINGH
To: SONY INTERACTIVE ENTERTAINMENT EUROPE LIMITED
Reel/Frame 049868/0347 →
Priority Claims (1)
GB 1811555.0 · Jul 13, 2018 · national
Continuity (1)
Related Publication 20200016499A1 · Jan 16, 2020
Cited By (1)
US 12,226,692