Systems and methods for real time control of a remote video production with multiple streams
The present invention relates to a cloud-based video production system and corresponding methods that deliver real time or near real time video with broadcast quality. Video from the same source is encoded into at least two separate streams, one with low latency and another with higher latency and higher quality. Corresponding frames of each stream have identical timestamps. The two streams are sent over separate signal paths. A first signal path is a real time control (RTC) path having low latency, and used to communicate the lower quality video content from video sources to a producer user interface. The second signal path provides the same video content at a higher video quality than that provided over the first signal path, but with longer latency. Commands from the user interface based on the lower latency stream can then be carried out on the higher latency stream.
1. A method for real time video production using multiple streams, comprising:
encoding video content from each of one or more video sources into at least two separate streams, a first video stream from each of the one or more video sources comprising a low quality version of the video content having a low latency and a second video stream from each of the one or more video sources comprising a higher quality version of the video content having a higher latency;
providing corresponding video frames of the first video stream and the second video stream with identical video timestamps;
providing a first signal path for communicating the first video stream from the one or more video sources to a user interface at the low latency;
providing a second signal path for communicating the second video stream from the one or more video sources to the user interface at the higher latency;
sending control signals comprising commands for selecting and manipulating the video content from the one or more video sources, the commands being based on one or more of the first video streams received on the first signal path; and
carrying out the commands on video content from one or more of the second video streams received on the second signal path to produce a video program;
wherein:
a user device comprising the user interface is adapted to receive the first video streams over the first signal path;
a cloud production server is adapted to receive the second video streams over the second signal path; and
the commands are communicated from the user device to a control unit of the cloud production server.
2. A method in accordance with claim 1 , further comprising buffering the second video streams at one or more buffers to account for latencies in carrying out the commands.
3. A method in accordance with claim 2 , wherein the buffering enables rewind and instant replay features.
4. A method in accordance with claim 1 , wherein the user device comprises a display for displaying the first video streams from the one or more video sources.
5. A method in accordance with claim 2 , wherein:
the control signal further comprises the video timestamp for each of the video frames subject to the command; and
the commands are executed at the time when the video timestamp at an output of the buffer matches the video timestamp received in the control signal.
6. A method in accordance with claim 5 , wherein the video timestamp received in the control signal comprises the video timestamp of the video frame displayed at the time the control signal is generated.
7. A method in accordance with claim 1 , wherein the second video streams comprise production quality video.
8. A method in accordance with claim 1 , wherein the video content comprises live video content.
9. A method for real time video production using multiple streams, comprising:
encoding video content from each of one or more video sources into at least two separate streams, a first video stream from each of the one or more video sources comprising a low quality version of the video content having a low latency and a second video stream from each of the one or more video sources comprising a higher quality version of the video content having a higher latency;
providing corresponding video frames of the first video stream and the second video stream with identical video timestamps;
providing a first signal path for communicating the first video stream from the one or more video sources to a user interface at the low latency;
providing a second signal path for communicating the second video stream from the one or more video sources to the user interface at the higher latency;
sending control signals comprising commands for selecting and manipulating the video content from the one or more video sources, the commands being based on one or more of the first video streams received on the first signal path; and
carrying out the commands on video content from one or more of the second video streams received on the second signal path to produce a video program;
wherein the commands comprise commands for: switching between the video content provided by the one or more video sources; selecting the video content or portions of the video content provided by one or more of the one or more video sources; combining the video content or portions of the video content from the one or more video sources; manipulating the video content or the portions of the video content; adjusting video brightness; providing a graphics overlay; turning on and off a video graphics overlay; adjusting an audio level; and adding special effects to the video content or portions of the video content.
10. A system for real time video production using multiple streams, comprising:
one or more video sources each providing respective video content, each of the one or more video sources encoding the respective video content into at least two separate streams, a first video stream from each of the one or more video sources comprising a low quality version of the video content having a low latency and a second video stream from each of the one or more video sources comprising a higher quality version of the video content having a higher latency;
a first signal path for communicating the first video stream from the one or more video sources to a user interface at the low latency;
a second signal path for communicating the second video stream from the one or more video sources to the user interface at the higher latency;
a user interface for sending control signals comprising commands for selecting and manipulating the video content from the one or more video sources, the commands being based on one or more of the first video streams received on the first signal path; and
a cloud production server for carrying out the commands on video content from one or more of the second video streams received on the second signal path to produce a video program;
wherein:
corresponding video frames of the first video stream and the second video stream are provided with identical video timestamps;
a user device comprises the user interface which is adapted to receive the first video streams over the first signal path;
the cloud production server is adapted to receive the second video streams over the second signal path; and
the commands are communicated from the user device to a control unit of the cloud production server.
11. A system in accordance with claim 10 , further comprising one or more buffers for buffering the second video streams to account for latencies in carrying out the commands.
12. A system in accordance with claim 11 , wherein the buffering enables rewind and instant replay features.
13. A system in accordance with claim 10 , wherein the user device comprises a display for displaying the first video streams from the one or more video sources.
14. A system in accordance with claim 11 , wherein:
the control signal further comprises the video timestamp for each of the video frames subject to the command; and
the commands are executed at the time when the video timestamp at an output of the buffer matches the video timestamp received in the control signal.
15. A system in accordance with claim 14 , wherein the video timestamp received in the control signal comprises the video timestamp of the video frame displayed at the time the control signal is generated.
16. A system in accordance with claim 10 , wherein the second video streams comprise production quality video.
17. A system in accordance with claim 10 , wherein the video content comprises live video content.
18. A system for real time video production using multiple streams, comprising:
one or more video sources each providing respective video content, each of the one or more video sources encoding the respective video content into at least two separate streams, a first video stream from each of the one or more video sources comprising a low quality version of the video content having a low latency and a second video stream from each of the one or more video sources comprising a higher quality version of the video content having a higher latency;
a first signal path for communicating the first video stream from the one or more video sources to a user interface at the low latency;
a second signal path for communicating the second video stream from the one or more video sources to the user interface at the higher latency;
a user interface for sending control signals comprising commands for selecting and manipulating the video content from the one or more video sources, the commands being based on one or more of the first video streams received on the first signal path; and
a cloud production server for carrying out the commands on video content from one or more of the second video streams received on the second signal path to produce a video program;
wherein:
corresponding video frames of the first video stream and the second video stream are provided with identical video timestamps; and
the commands comprise commands for: switching between the video content provided by the one or more video sources; selecting the video content or portions of the video content provided by one or more of the one or more video sources; combining the video content or portions of the video content from the one or more video sources; manipulating the video content or the portions of the video content; adjusting video brightness; providing a graphics overlay; turning on and off a video graphics overlay; adjusting an audio level; and adding special effects to the video content or portions of the video content.