Live venue performance capture and visualization over game network
Systems, methods, and apparatuses disclosed herein can incorporate one or more real-world performers that are performing within a venue into interactive content. These systems, methods, and apparatuses can identify one or more joints or ligaments, for example, left shoulders, right knees, among others, of the one or more real-world performers from the image, or the series of images, of the one or more real-world performers. These joints or ligaments can be represented as one or more performer markers. These systems, methods, and apparatuses can generate one or more three-dimensional models of the one or more real-world performers in a three-dimensional space from the one or more performer markers. These systems, methods, and apparatuses can apply the one or more three-dimensional models of the one or more real-world performers to the one or more virtual characters in the three-dimensional space. These systems, methods, and apparatuses can render the one or more virtual characters from the three-dimensional space into a two-dimensional space of the interactive content to integrate the one or more real-world performers into the interactive content.
1 . A content server for incorporating a real-world performer within a venue into an interactive content, the content server comprising:
a memory that stores a video of the real-world performer; and
a processor configured to execute instructions stored in the memory, the instructions when executed by the processor, configuring the processor to:
capture movement of the real-world performer from the video,
generate a three-dimensional model of the real-world performer in a three-dimensional space that emulates the movement of the real-world performer,
apply the three-dimensional model of the real-world performer to a virtual character that is to be incorporated into the interactive content in the three-dimensional space to cause the virtual character to emulate the movement of the real-world performer in the interactive content,
render the virtual character from the three-dimensional space into a two-dimensional space of the interactive content to integrate the virtual character within the interactive content, and
provide the virtual character to a portable electronic device in response to the portable electronic device executing the interactive content to allow a user of the portable electronic device to interact with the real-world performer through the virtual character within the interactive content.
2 . The content server of claim 1 , wherein the instructions when executed by the processor, further configure the processor to receive the video from a camera as the real-world performer is performing within the venue.
3 . The content server of claim 1 , wherein the instructions when executed by the processor, configure the processor to capture the movement of the real-world performer with a markerless approach.
4 . The content server of claim 3 , wherein the markerless approach is configured to identify and parse the real-world performer from the video to identify the real-world performer from the video.
5 . The content server of claim 1 , wherein the instructions when executed by the processor, further configure the processor to:
identify a plurality of performer markers of the real-world performer from the video; and
generate the three-dimensional model of the real-world performer from the plurality of performer markers.
6 . The content server of claim 5 , wherein the instructions when executed by the processor, configure the processor to identify one or more joints or ligaments of the real-world performer from the video as the plurality of performer markers.
7 . The content server of claim 1 , wherein the interactive content is to solicit participation from an audience within the venue to perform actions.
8 . The content server of claim 1 , wherein the instructions when executed by the processor, configure the processor to provide the virtual character to the portable electronic device to allow the user of the portable electronic device to manipulate one or more buttons, one or more omnidirectional control sticks or buttons, or one or more touchscreen interfaces incorporated within the portable electronic device to interact with the real-world performer through the virtual character within the interactive content.
9 . The content server of claim 1 , wherein the instructions when executed by the processor, configure the processor to provide the virtual character to the portable electronic device to allow the user of the portable electronic device to engage with the virtual character within the interactive content to complete an action initiated by the real-world performer.
10 . The content server of claim 9 , wherein the instructions, when executed by the processor, configure the processor to apply the action initiated by the real-world performer to the three-dimensional model of the real-world performer.
11 . A method for incorporating a real-world performer into an interactive content, the method comprising:
accessing, by one or more computer systems, a video of the real-world performer;
capturing, by the one or more computer systems, movement of the real-world performer within the venue from the video;
generating, by the one or more computer systems, a three-dimensional model of the real-world performer in a three-dimensional space that emulates the movement of the real-world performer;
applying, by the one or more computer systems, the three-dimensional model of the real-world performer to a virtual character that is to be incorporated into the interactive content in the three-dimensional space to cause the virtual character to emulate the movement of the real-world performer in the interactive content;
rendering the virtual character from the three-dimensional space to a two-dimensional space of the interactive content to integrate the virtual character within the interactive content; and
providing, by the one or more computer systems, the virtual character to a portable electronic device in response to the portable electronic device executing the interactive content to allow a user of the portable electronic device to interact with the real-world performer through the virtual character within the interactive content.
12 . The method of claim 11 , wherein the accessing comprises receiving the video from a camera as the real-world performer is performing within the venue.
13 . The method of claim 11 , wherein the capturing comprises capturing the movement of the real-world performer with a markerless approach.
14 . The method of claim 13 , wherein the markerless approach identifies and parses the real-world performer from the video to identify the real-world performer from the video.
15 . The method of claim 11 , wherein the generating comprises:
identifying a plurality of performer markers of the real-world performer from the video; and
generating the three-dimensional model of the real-world performer from the plurality of performer markers.
16 . The method of claim 15 , wherein the identifying comprises identifying one or more joints or ligaments of the real-world performer from the video as the plurality of performer markers.
17 . The method of claim 15 , wherein the providing comprises providing the virtual character to the portable electronic device to allow the user of the portable electronic device to engage with the virtual character within the interactive content to complete an action initiated by the real-world performer.
18 . The method of claim 15 , wherein the generating comprises applying the action initiated by the real-world performer to the three-dimensional model of the real-world performer.
19 . The method of claim 11 , wherein the interactive content is to solicit participation from an audience within the venue to perform actions.
20 . A venue for incorporating a real-world performer into an interactive content, the interactive content is to solicit participation from an audience within the venue to perform actions, the venue comprising:
a content server configured to:
capture movement of the real-world performer within the venue from a video of the real-world performer,
apply a three-dimensional model of the real-world performer to a virtual character that is to be incorporated into the interactive content in a three-dimensional space to cause the virtual character to emulate the movement of the real-world performer in the interactive content, and
render the virtual character from the three-dimensional space into a two-dimensional space of the interactive content to integrate the virtual character within the interactive content; and
a plurality of electronic devices configured to:
receive the virtual character in the two-dimensional space from the content server,
execute a software application having the interactive content, and
interact with the real-world performer through the virtual character within the interactive content.
21 . The venue of claim 20 , further comprising a camera to capture the video as the real-world performer is performing within the venue, and
wherein the content server is further configured to receive the video from the camera.
22 . The venue of claim 20 , wherein the content server is configured to capture the movement of the real-world performer with a markerless approach.
23 . The venue of claim 22 , wherein the markerless approach identifies and parses the real-world performer from the video to identify the real-world performer from the video.
24 . The venue of claim 20 , wherein the content server is further configured to:
identify a plurality of performer markers of the real-world performer from the video; and
generate the three-dimensional model of the real-world performer from the plurality of performer markers.
25 . The venue of claim 24 , wherein the identifying comprises identifying one or more joints or ligaments of the real-world performer from the video as the plurality of performer markers.
26 . The venue of claim 20 , wherein the plurality of electronic devices comprise one or more buttons, one or more omnidirectional control sticks or buttons, or one or more touchscreen interfaces incorporated that are manipulated to interact with the real-world performer through the virtual character within the interactive content.
27 . The venue of claim 20 , wherein the plurality of electronic devices are configured to engage with the virtual character within the interactive content to complete an action initiated by the real-world performer.
28 . The venue of claim 27 , wherein the content server is configured to apply the action initiated by the real-world performer to the three-dimensional model of the real-world performer.