Interactive virtual reality broadcast systems and methods
A method for generating a virtual reality (VR) broadcast of a VR performance is provided. The VR broadcast is received by audience devices that receive audience feedback data. A VR system controller synchronizes the audience feedback data with the VR broadcast to generate synchronized audience feedback data in real time and outputs the VR broadcast with the synchronized audience feedback data to the audience devices. The VR system controller receives performer feedback data and transmits performer feedback data to the audience devices.
1. A method comprising:
receiving, with a virtual reality (VR) system controller, audio and visual performance data of a performance by a performer from at least one performer device;
generating, with the VR system controller, a VR broadcast based on the audio and visual performance data of the performance;
receiving, with each audience device of a plurality of audience devices, the VR broadcast, each audience device being configured to receive audience feedback data from an associated audience member,
transmitting, with each audience device of the plurality of audience devices, the audience feedback data to the VR system controller, the VR system controller being configured to synchronize the audience feedback data from each audience device with the VR broadcast to generate synchronized audience feedback data in real time such that the audience feedback data is synchronized in time with the audio and visual performance data of the performance of the VR broadcast;
transmitting, with the VR system controller, the VR broadcast with the synchronized audience feedback data to the plurality of audience devices;
receiving, with each audience device of the plurality of audience devices, the VR broadcast with the synchronized audience feedback data from each audience device;
outputting, with each audience device of the plurality of audience devices, the VR broadcast including the audio and visual performance data with the synchronized audience feedback data to the associated audience member;
receiving, with VR system controller, performer feedback data captured by the at least one performer device, the performer feedback data including audio and visual feedback data of the performer responsive to the audience feedback data;
transmitting, with the VR system controller, the performer feedback data to at least one audience device of the plurality of audience devices;
receiving, with the at least one audience device, the performer feedback data; and
outputting, with the at least one audience device, the performer feedback data to the associated audience member while the associated audience member is watching and listening to the VR broadcast.
2. The method of claim 1 , wherein the audience feedback data includes at least one of audience video data that includes facial expressions of the associated audience member and audience audio data that includes audible sounds generated by the associated audience member while watching and listening to the VR broadcast.
3. A method comprising:
receiving, with a virtual reality (VR) system controller, audio data associated with at least one performer performing on a performance area of a performance space;
receiving, with the VR system controller, facial expression data from at least one facial expression sensor and body movement data from at least one body movement sensor, each facial expression sensor and each body movement sensor being associated with a performer of the at least one performer performing on the performance area of the performance space, the facial expression data from each facial expression sensor capturing facial expressions of the associated performer and the body movement data from each body movement sensor capturing body movements of the associated performer;
retrieving from a data storage, with the VR system controller, at least one previously stored three-dimensional (3D) VR avatar associated with the at least one performer and a previously stored VR performance area representing the performance area of the performance space;
generating, with the VR system controller, a 3D VR performance environment showing the at least one 3D VR avatar performing within the VR performance area, with each 3D VR avatar being manipulated to include the facial expressions and the body movements of the associated performer based on the facial expression data from the associated facial expression sensor for the associated performer and the body movement data from the associated body movement sensor for the associated performer;
generating, with the VR system controller, a VR broadcast of the 3D VR performance environment including the at least one 3D VR avatar displayed on the VR performance area and including the audio data such that the facial expressions and the body movements of the at least one 3D VR avatar are synchronized with the audio data;
transmitting, with the VR system controller, the VR broadcast to at least one audience device, each audience device outputting the VR broadcast to an associated audience member, each audience device being configured to receive visual audience feedback data from the associated audience member;
receiving, with the VR system controller, the visual audience feedback data from the at least one audience device; and
outputting, with the VR system controller, the visual audience feedback data to the at least one performer in real time while the at least one performer is performing at the performance space by displaying the visual audience feedback data on a display viewable by the at least one performer performing at the performance space.
4. The method of claim 3 , wherein each audience device includes a camera and the visual audience feedback data includes facial expression data of the associated audience member captured by the camera.
5. The method of claim 3 , wherein each audience device includes a microphone and is further configured to receive audio audience feedback data from the associated audience member captured by the microphone, the method further comprising:
receiving, with the VR system controller, the audio audience feedback data from the at least one audience device; and
outputting, with the VR system controller, the audio audience feedback data to the at least one performer in real time while the at least one performer is performing at the performance space by outputting the audio audience feedback data with at least one speaker to the at least one performer performing at the performance space.
6. The method of claim 3 , wherein the display is included in a set of AR glasses worn by the at least one performer.
7. A method comprising:
receiving, with each audience device of a plurality of audience devices, a virtual reality (VR) broadcast transmitted from a VR system controller, each audience device having being configured to capture audience feedback data from an associated audience member, the VR broadcast including audio and visual performance data of a performance;
transmitting, with each audience device of the plurality of audience devices, the audience feedback data to the VR system controller, the VR system controller being configured to synchronize the audience feedback data from each audience device with the VR broadcast to generate synchronized audience feedback data in real time such that the audience feedback data is synchronized in time with the audio and visual performance data of the performance of the VR broadcast;
receiving, with each audience device of the plurality of audience devices, the VR broadcast with the synchronized audience feedback data from each audience device; and
outputting, with each audience device of the plurality of audience devices, the VR broadcast including the audio and visual performance data with the synchronized audience feedback data to the associated audience member, such that each audience member receives reactions of other audience members while watching and listening to the VR broadcast;
wherein each audience device includes a camera, the audience feedback data includes visual data captured by the camera, and the visual data includes facial expression data of the associated audience member captured by the camera.
8. The method of claim 7 , wherein each audience device includes a microphone and the audience feedback data includes audio data captured by the microphone.
9. The method of claim 7 , wherein the VR system controller is configured to generate and transmit the VR broadcast based on a real-time performance by at least one performer performing while each audience member is watching and listening to the VR broadcast.
10. The method of claim 9 , further comprising outputting, with the VR system controller, the synchronized audience feedback data to the at least one performer during the real-time performance by the at least one performer.
11. A method comprising:
receiving, with a virtual reality (VR) system controller, audio data associated with at least one performer performing a performance within a performance area;
receiving, with the virtual reality (VR) system controller, facial expression data from at least one expression sensor and body movement data from a at least one body movement sensor, each facial expression sensor and each body movement sensor being associated with a performer of the at least one performer, the facial expression data from each facial expression sensor capturing facial expressions of the associated performer and the body movement data from each body movement sensor capturing body movements of the associated performer;
retrieving, with the VR system controller, at least one previously stored three-dimensional (3D) VR avatar associated with the at least one performer and a previously stored VR performance area representing the performance area from a data storage;
generating, with the VR system controller, a 3D VR performance environment showing the at least one 3D VR avatar performing within the VR performance area and with each 3D VR avatar being manipulated to include the facial expressions and the body movements of the associated performer based on the facial expression data from the associated facial expression sensor for the associated performer and the body movement data from the associated body movement sensor for the associated performer;
generating, with the VR system controller, a VR broadcast of the 3D VR performance environment including the at least one 3D VR avatar displayed on the VR performance area and including the audio data such that the facial expressions and the body movements of the at least one 3D VR avatar are synchronized with the audio data;
transmitting, with the VR system controller, the VR broadcast to at least one audience device, each audience device outputting the VR broadcast to an associated audience member;
generating, with the VR system controller, the VR broadcast for a particular audience device of the at least one audience device with audio data of the VR broadcast for the particular audience device corresponding to a particular audio stream of a plurality of audio streams, each audio stream of the plurality of audio streams corresponding to a different location of the 3D VR performance environment;
receiving, with the VR system controller, user input from the particular audience device indicating a different location within the 3D VR performance environment; and
generating the VR broadcast for the particular audience device with audio data corresponding to a different audio stream of the plurality of audio streams in response to receiving the user input indicating the different location, the different audio stream being corresponding to different location within the 3D VR performance environment.
12. The method of claim 11 , further comprising:
receiving, with the VR system controller, visual audience feedback data from the at least one audience device; and
outputting, with the VR system controller, the visual audience feedback data to the at least one performer in real time while the at least one performer is performing within the performance area by displaying the visual audience feedback data on a display viewable by the at least one performer performing within the performance area.
13. The method of claim 12 , wherein each audience device includes a camera and the visual data includes facial expression data of the associated audience member captured by the camera.
14. The method of claim 12 , further comprising:
receiving, with the VR system controller, audio audience feedback data from the at least one audience device; and
outputting, with the VR system controller, the audio audience feedback data to the at least one performer in real time while the at least one performer is performing within the performance area by outputting the audio audience feedback data with at least one speaker to the at least one performer performing within the performance area.
15. A method comprising:
receiving, with each audience device of a plurality of audience devices, a virtual reality (VR) broadcast transmitted from a VR system controller, each audience device being configured to capture audience feedback data from an associated audience member, the VR broadcast including audio and visual performance data of a performance, the VR system controller being configured to generate and transmit the VR broadcast based on a real-time performance by at least one performer performing while each audience member is watching and listening to the VR broadcast;
transmitting, with each audience device of the plurality of audience devices, the audience feedback data to the VR system controller, the VR system controller being configured to synchronize the audience feedback data from each audience device with the VR broadcast to generate synchronized audience feedback data in real time such that the audience feedback data is synchronized in time with the audio and visual performance data of the performance of the VR broadcast;
receiving, with each audience device of the plurality of audience devices, the VR broadcast with the synchronized audience feedback data from each audience device;
outputting, with each audience device of the plurality of audience devices, the VR broadcast including the audio and visual performance data with the synchronized audience feedback data to the associated audience member, such that each audience member receives reactions of other audience members while watching and listening to the VR broadcast; and
outputting, with the VR system controller, the synchronized audience feedback data to the at least one performer during the real-time performance by the at least one performer.
16. The method of claim 15 , wherein each audience device includes a camera and the audience feedback data includes visual data captured by the camera.
17. The method of claim 15 , wherein each audience device includes a camera and a microphone and the audience feedback data includes visual data captured by the camera and audio data captured by the microphone.
18. The method of claim 17 , wherein the visual data includes facial expression data of the associated audience member captured by the camera.