IP Library › Granted Patent US 9,710,972
Granted Patent B2
US 9,710,972 · App. 14/483,865 · Granted Jul 18, 2017

Immersion photography with dynamic matte screen

Inventors: Mike Sanders (San Francisco, CA); Kim Libreri (Fairfax, CA); Nick Rasmussen (San Francisco, CA); John Gaeta (Ross, CA)
Assignee: LUCASFILM ENTERTAINMENT COMPANY LTD.
G06T19/006H04N5/265H04N5/275
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,710,972
App. No.
14/483,865
Granted
Jul 18, 2017
Kind
B2
Abstract

A method may include displaying, on one or more display devices in a virtual-reality environment, a visual representation of a 3-D virtual scene from the perspective of a subject location in the virtual-reality environment. The method may also include displaying, on the one or more display devices, a chroma-key background with the visual representation. The method may further include recording, using a camera, an image of the subject in the virtual-reality environment against the chroma-key background.

Claims (53)

1. A method comprising:

generating, for display on one or more display devices in a virtual-reality environment, an image comprising a visual representation of a 3-D virtual scene rendered from the perspective of a subject located in the virtual-reality environment;

modifying the image to further comprise a chroma-key background inserted into the visual representation of the 3-D virtual scene, wherein the chroma-key background is positioned in the image such that when the image is displayed on the one or more display devices, the subject is seen against the chroma-key background from the view of a camera recording the subject;

displaying, on the one or more display devices, the image comprising the visual representation of the 3-D virtual scene with the chroma-key background inserted, such that the chroma-key background is visible on the one or more display devices; and

recording, using the camera, an image of the subject in the virtual-reality environment against the chroma-key background along with at least a portion of the one or more display devices.

2. The method of claim 1 , wherein the visual representation comprises an interactive scene.

3. The method of claim 1 , further comprising:

tracking a location of the subject in the virtual-reality environment;

determining a location in the 3-D virtual scene based on the location of the subject in the virtual reality environment; and

rendering the 3-D virtual scene from the perspective of the location in the 3-D virtual scene to generate the visual representation of the 3-D virtual scene.

4. The method of claim 1 , wherein the image is recorded by the camera as the representation of the 3-D virtual scene is displayed on the one or more display devices.

5. The method of claim 1 , wherein the chroma-key background covers at least a portion of the 3-D virtual scene when displaying the chroma-key background with the visual representation of the 3-D virtual scene.

6. The method of claim 1 , further comprising:

detecting a change in a location of the subject in the virtual reality environment;

performing a real-time render of the 3-D virtual scene based on the change in the location of the subject in the virtual-reality environment; and

displaying, on the one or more display devices, an updated visual representation of the 3-D virtual scene based on the real-time render.

7. The method of claim 6 , further comprising changing the location of the chroma-key background on the one or more display devices based on the detected change in the location of the subject in the virtual-reality environment.

8. The method of claim 1 , further comprising:

extracting the image of the subject against the chroma-key background from the visual representation of the 3-D virtual scene;

inserting the extracted image of the subject into the 3-D virtual scene to create a frame for digital content.

9. The method of claim 8 , wherein creating the frame for the digital media production comprises rendering the 3-D virtual scene and the extracted image to generate a 2-D rendered image.

10. The method of claim 1 , wherein displaying the chroma-key background with the visual representation comprises interleaving the chroma-key background between displayed frames of the visual representation.

11. The method of claim 1 , wherein displaying the chroma key background with the visual representation comprises displaying the chroma-key background on at least a portion of the one or more display devices that is visible to the camera.

12. A system comprising:

a virtual-reality environment comprising one or more display devices;

a camera;

and a computer system comprising one or more processors and a memory device having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

generating, for display on the one or more display devices in the virtual-reality environment, an image comprising a visual representation of a 3-D virtual scene rendered from the perspective of a subject located in the virtual-reality environment;

modifying the image to further comprise a chroma-key background inserted into the visual representation of the 3-D virtual scene, wherein the chroma-key background is positioned in the image such that when the image is displayed on the one or more display devices, the subject is seen against the chroma-key background from the view of a camera recording the subject;

displaying, on the one or more display devices, the image comprising the visual representation of the 3-D virtual scene with the chroma-key background inserted, such that the chroma-key background is visible on the one or more display devices; and

recording, using the camera, an image of the subject in the virtual-reality environment against the chroma-key background along with at least a portion of the one or more display devices.

13. The system of claim 12 , wherein the visual representation comprises an interactive scene.

14. The system of claim 12 , wherein the instructions cause the one or more processors to perform further operations comprising:

receiving a location of the subject in the virtual-reality environment;

determining a location in the 3-D virtual scene based on the location of the subject in the virtual reality environment; and

rendering the 3-D virtual scene from the perspective of the location in the 3-D virtual scene to generate the visual representation of the 3-D virtual scene.

15. The system of claim 12 , further comprising using the image of the subject in the virtual-reality environment to perform a motion capture of the subject.

16. The system of claim 12 , wherein the instructions cause the one or more processors to perform further operations comprising:

detecting a change in a location of the subject in the virtual reality environment;

performing a real-time render of the 3-D virtual scene based on the change in the location of the subject in the virtual-reality environment; and

displaying, on the one or more display devices, an updated visual representation of the 3-D virtual scene based on the real-time render; and

changing the location of the chroma-key background on the one or more display devices based on the detected change in the location of the subject in the virtual-reality environment.

17. The system of claim 12 , wherein displaying the chroma-key background with the visual representation comprises interleaving the chroma-key background between displayed frames of the visual representation.

18. The system of claim 12 , wherein displaying the chroma key background with the visual representation comprises displaying the chroma-key background on at least a portion of the one or more display devices that is visible to the camera.

19. A non-transitory storage medium comprising instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:

generating, for display on one or more display devices in a virtual-reality environment, an image comprising a visual representation of a 3-D virtual scene rendered from the perspective of a subject located in the virtual-reality environment;

modifying the image to further comprise a chroma-key background inserted into the visual representation of the 3-D virtual scene, wherein the chroma-key background is positioned in the image such that when the image is displayed on the one or more display devices, the subject is seen against the chroma-key background from the view of a camera recording the subject;

displaying, on the one or more display devices, the image comprising the visual representation of the 3-D virtual scene with the chroma-key background inserted, such that the chroma-key background is visible on the one or more display devices; and

recording, using the camera, an image of the subject in the virtual-reality environment against the chroma-key background along with at least a portion of the one or more display devices.

20. The non-transitory storage medium of claim 19 , wherein the instructions cause the one or more processors to perform additional operations comprising:

receiving a location of the subject in the virtual-reality environment;

determining a location in the 3-D virtual scene based on the location of the subject in the virtual reality environment; and

rendering the 3-D virtual scene from the perspective of the location in the 3-D virtual scene to generate the visual representation of the 3-D virtual scene.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 11, 2014
From: SANDERS, MIKE; LIBRERI, KIM; RASMUSSEN, NICK; GAETA, JOHN
To: LUCASFILM ENTERTAINMENT COMPANY LTD.
Reel/Frame 033724/0001 →
Continuity (2)
Provisional Application 62005731 · May 30, 2014
Related Publication 20150348326A1 · Dec 3, 2015