IP Library Granted Patent US 9,143,721
Granted Patent B2
US 9,143,721 · App. 13/869,341 · Granted Sep 22, 2015

Content preparation systems and methods for interactive video systems

Inventors: Anthony Gentile (New York, NY); John Gentile (Montclair, NJ); Scott Wilker (Glen Rock, NJ); Kevin Clement (Crompond, NY); Mark Waldrep (Los Angeles, CA)
Assignee: Noo Inc.
H04N5/91G06T5/50G11B27/034G11B27/322G11B27/34H04N5/262H04N5/272G06F3/0484
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,143,721
App. No.
13/869,341
Granted
Sep 22, 2015
Kind
B2
Abstract

Content preparation systems and methods are disclosed that generate scenes used by an interactive role performance system for inserting a user image as a character in the scene. Original media content from a variety of sources, such as movies, television, and commercials, can provide participants with a wide variety of scenes and roles. In some examples, the content preparation system removes an original character from the selected media content and recreates the background to enable an image of a user to be inserted therein. By recreating the background after removing the character, the user is given greater freedom to perform, as the image of the user can perform anywhere within the scene. Moreover, systems and methods can generate and store metadata associated with the modified media content that facilitates the combining of the modified media content and the user image to replace the removed character image.

Claims (64)

1. A method for preparing media content, comprising modified video content and associated metadata, and a matte layer video file, for use with a video image combining system, the method comprising:

receiving original video content comprising N multiple frames having an original character associated therewith, wherein N is an integer greater than 2;

selecting a first plurality of M consecutive particular frames of the N multiple frames, wherein M is an integer greater than one but less than N;

for each of the M consecutive particular frames,

receiving the particular frame, wherein the particular frame displays a background image in which the original character occupies a position therein, and

modifying the particular frame to digitally remove the original character by extending the background image of the particular frame to fill the position of the original character to allow for subsequent insertion of a replacement character in approximately the same position;

combining the M modified particular frames with remaining N-M frames of the N multiple frames to create the modified video content;

generating metadata associated with the modified video content, the metadata being configured to direct the insertion of the replacement character upon playback of and into the modified video content, the metadata including at least,

a first frame number of the M consecutive particular frames and an indication of the number of modified frames, and

the position the original character occupied in the original video content; and

creating for at least a portion of the M consecutive particular frames the matte layer video file by:

generating a matte layer corresponding to each particular frame, wherein the matte layer delineates an element of the particular frame in a foreground of the original character; and

for each particular frame, associating the matte layer with the modified video content.

2. The method of claim 1 , wherein the metadata comprises data for generating a visual outline of the position of the original character.

3. The method of claim 1 , further comprising:

receiving original audio content comprising audio associated with the original character; and

modifying the original audio content associated with the M consecutive particular frames to remove audio associated with the original character to create modified audio content.

4. The method of claim 3 , further comprising generating subtitle data that corresponds to the original audio content.

5. The method of claim 4 , further comprising combining the subtitle data, the modified audio content, and the modified video content to create a bundled media content file.

6. The method of claim 1 , additionally comprising storing the modified video content, the associated metadata, and the matte layer video file on a computer-readable medium.

7. The method of claim 1 , wherein the original video content comprises at least one of a portion of a movie, a television show, and a commercial.

8. The method of claim 1 , additionally comprising:

selecting a second plurality of L consecutive particular frames of the N multiple frames displaying a second original character, wherein L is an integer greater than one but less than N; and

for each of the L consecutive particular frames, modifying the particular frame to erase the second original character, wherein said modifying comprises digitally removing the second original character by extending the background image to fill a position of the second original character to allow for subsequent insertion of a second replacement character in the position of the second original character.

9. The method of claim 1 , wherein the metadata is indicative of at least one of hue, color, lighting information, a camera location, a camera distance, a camera selection, and a camera angle of the modified video content.

10. The method of claim 1 , wherein the metadata comprises an eXtensible Markup Language (XML) file.

11. The method of claim 1 , wherein the metadata further comprises script data associated with a dialogue of the original character in the original video content.

12. The method of claim 1 , wherein the metadata further comprises user instruction data associated with movement of the original character within the original video content.

13. The method of claim 1 , wherein the original video content comprises a video game.

14. A computer readable medium storing modified video content and associated metadata and a matte layer video file, which have been prepared by a method comprising:

receiving original video content comprising N multiple frames having an original character associated therewith, wherein N is an integer greater than 2;

selecting a first plurality of M consecutive particular frames of the N multiple frames, wherein M is an integer greater than one but less than N;

for each of the M consecutive particular frames,

receiving the particular frame, wherein the particular frame displays a background image in which the original character occupies a position therein, and

modifying the particular frame to digitally remove the original character by extending the background image of the particular frame to fill the position of the original character to allow for subsequent insertion of a replacement character in approximately the same position;

combining the M modified particular frames with remaining N-M frames of the N multiple frames to create the modified video content;

generating metadata associated with the modified video content, the metadata being configured to direct the insertion of the replacement character upon playback of and into the modified video content, the metadata including at least,

a first frame number of the M consecutive particular frames and an indication of the number of modified frames, and

the position the original character occupied in the original video content; and

creating for at least a portion of the M consecutive particular frames the matte layer video file by:

generating a matte layer corresponding to each particular frame, wherein the matte layer delineates an element of the particular frame in a foreground of the original character; and

for each particular frame, associating the matte layer with the modified video content.

15. A method of compositing modified video content stored on a computer-readable medium,

wherein the modified video content, associated metadata, and a matte layer video file have been prepared by a method comprising:

receiving original video content comprising N multiple frames having an original character associated therewith, wherein N is an integer greater than 2;

selecting a first plurality of M consecutive particular frames of the N multiple frames, wherein M is an integer greater than one but less than N;

for each of the M consecutive particular frames,

receiving the particular frame, wherein the particular frame displays a background image in which the original character occupies a position therein, and

modifying the particular frame to digitally remove the original character by extending the background image of the particular frame to fill the position of the original character to allow for subsequent insertion of a replacement character in approximately the same position;

combining the M modified particular frames with remaining N-M frames of the N multiple frames to create the modified video content;

generating metadata associated with the modified video content, the metadata being configured to direct the insertion of the replacement character into the modified video content, the metadata including at least,

a first frame number of the M consecutive particular frames and an indication of the number of modified frames, and

the position the original character occupied in the original video content; and

creating for at least a portion of the M consecutive particular frames the matte layer video file by:

generating a matte layer corresponding to each particular frame, wherein the matte layer delineates an element of the particular frame in a foreground of the original character; and

for each particular frame, associating the matte layer with the modified video content; and

wherein the compositing the modified video content comprises:

capturing video content of a replacement character by a video camera;

removing a background from the video content;

upon playback of the modified video content, inserting the video content of the replacement character, sans background, into the modified video content at the approximate position of the original character, based on the metadata; and

for each particular frame a matte layer was generated, inserting the element delineated by the matte layer into the foreground of the inserted replacement character, based on the matte layer video file.

16. The method of claim 15 , wherein the replacement character steps away from the video camera to enable the video camera to record a first video signal of the background, and the replacement character then steps back in for generating a second video signal.

17. The method of claim 16 , wherein in the step of removing the background, a pixel of the first video signal is set as transparent or opaque, based on a color channel difference between the second video signal and the first video signal for the pixel.

18. The method of claim 17 , wherein the pixel of the first video signal is set as transparent if the absolute value of the color channel difference between the second video signal and the first video signal for the pixel is less than a threshold.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 7, 2014
From: YOOSTAR ENTERTAINMENT GROUP, INC.
To: NOO INC.
Reel/Frame 034129/0973 →
Continuity (4)
Continuation 12495548 · Jun 30, 2009
Provisional Application 61077363 · Jul 1, 2008
Provisional Application 61144383 · Jan 13, 2009
Related Publication 20130236160A1 · Sep 12, 2013