Method of synchronizing audio-visual assets
View Patent ↗A system and method of synchronizing the processing and distribution of audio-visual assets in a distributed production studio are presented. The method addresses the processing of broadcast television, including match moving computer graphics technology, in such an environment. Audio-visual sources provide the assets while processing nodes interact with them to produce a composite output. Synchronization of the communication between, and the processing by, the sources and nodes is managed by a master controller and delay buffers located on the nodes. Assets are tagged with updatable reference-to-zero values indicative of a current transmission delay between their last source and the controller. These reference-to-zero values are used by processing nodes to determine where in their local buffer to place each asset so they emerge from the buffer to a local digital signal processor in synchronicity and that assets originally produced at a common time are executed locally at a common time.
1. A method of synchronizing audio-visual assets, comprising:
providing one or more remotely located end-viewer stations comprising a digital video display and a digital audio loudspeaker;
providing one or more remotely located audio-visual sources at least one of which produces an audio-visual asset representative of a live event;
providing one or more remotely located processing nodes each of which comprises a digital signal processor and a delay buffer;
providing a master controller comprising a master digital signal processor, a master clock and a master parameter list, said master parameter list comprising, for each processing node and source, parameters indicative of an identifier, a reference-to-zero value, available bandwidth, a frame size, a frame drop rate and a bandwidth setting, and additionally for each processing node, parameters indicative of a maximum buffer delay capacity and a processing power;
sending, by a first one of said audio-visual sources, via a digital communications network, a first one of said audio-visual assets, and by a second one of said audio-visual sources, a second one of said audio-visual assets, each of said audio-visual assets comprising a data packet and an ID packet, said ID packet comprising said source identifier and said source-reference-to zero value, said source reference-to-zero value being indicative of a transmission time delay of said audio-visual asset between said source and said master;
receiving, by one of said processing nodes, said first and second audio-visual assets;
placing, by said receiving processing node, each of said audio-visual assets in said delay buffer in delay buffer locations such that said first and second audio-visual assets are processed by said digital signal processor at common time with respect to said master clock to produce an output audio-visual asset;
transmitting by said receiving processing node, said output audio-visual asset synchronized to a current reference to zero value of said receiving processing node;
receiving by said one or more end viewer stations said output audio-visual asset;
displaying, by said receiving end viewer stations, a video component of said output audio-visual asset on said end viewer digital video display; and
producing, by said receiving end viewer stations, an audible sound corresponding to an audio component of said output audio-visual asset produced by said digital audio loudspeaker.
2. The method of claim 1 further comprising:
sending, by a second one of said processing nodes, via said digital communications network, a third one of said audio-visual assets;
receiving by said receiving node, said third audio-visual asset; and
placing, by said receiving processing node, said third audio-visual asset in said delay buffer at a delay buffer location such that said first, second and third audio-visual assets are processed by said digital signal processor at a common time with respect to said master clock to produce said output audio-visual asset.
3. The method of claim 2 further comprising:
calculating, by said master digital signal processor, a least common buffer delay of said maximum buffer delays for a set of node parameters contained in said master parameter list; and
altering a required transmission bandwidth of one or more of said audio-visual assets such that said output audio-visual asset may be produced within said least common buffer delay, said altering comprising changing a video frame size, or a frame drop rate, or both, of said audio-visual source generating said audio-visual asset or said processing node generating said audio-visual asset.
4. The method of claim 3 wherein first audio-visual source is a broadcast quality television camera.
5. The method of claim 4 wherein said third audio-visual asset comprises instructions for placing a graphic generated by said second audio-visual source at a predetermined location within a video frame of a video component of one of said first audio-visual assets produced by said first audio-visual source.
6. The method of claim 5 further wherein said digital signal processor on said receiving processing node performs functions comprising real-time match moving computer graphics insertion in said output audio-visual asset.
7. The method of claim 2 further comprising an alteration in a processing requirement of said one or more assets, a detection of a requirement for a break in a chain of processing and a re-buffering of one or more of said audio-visual assets to facilitate said break in said chain of processing.
8. A system for synchronizing audio-visual assets, comprising:
one or more audio-visual sources at least one of which produces an audio-visual asset representative of a live event;
one or more processing nodes each of which comprises a digital signal processor and a delay buffer;
a master controller comprising a master digital signal processor, a master clock and a master parameter list, said master parameter list comprising, for each processing node and source, parameters indicative of an identifier, a reference-to-zero value, a bandwidth selection, a frame resolution, and additionally for each processing node, parameters indicative of a maximum buffer delay and a processing power;
a first one of said audio-visual assets sent by a first one of said audio-visual sources via a digital communications network and received by one of said processing nodes, and a second one of said audio-visual assets sent by a second one of said audio-visual sources via said digital communications network to said same receiving processing node, each of said audio-visual assets comprising a data packet and an ID packet, said ID packet comprising said source identifier and said source-reference-to zero value, said source reference-to-zero value being indicative of a time delay between transmission of said audio-visual asset by said source and a current time of said master clock;
delay buffer locations in said receiving processing node wherein each of said first and second audio-visual assets are placed such that said first and second audio-visual assets are processed by said digital signal processor at common time with respect to said master clock to produce an output audio-visual asset synchronized to a current reference to zero value of said receiving processing node; and
one or more end-viewer stations comprising a digital video display and a digital audio loudspeaker, and wherein one or more of said end-viewer stations receive, via said digital communications network, said output audio-visual asset transmitted from said receiving processing node and display a video component of said output audio-visual asset on said end viewer digital video display and produce, using said digital audio loudspeaker, an audible sound corresponding to an audio component of said output audio-visual asset.