Adaptive streaming transcoder synchronization
Techniques for synchronizing streams in an adaptive streaming environment include cross-chassis and intra-chassis synchronization of video and/or audio. In an adaptive streaming system that streams content in segments or chunks at varying levels of quality corresponding to different coding rates, implementations enable seamless switching between transcoded audio output streams. The streams synchronized may be those that have been ingested by the transcoder in a compressed format and transcoded into a different compressed format, as part of the transcoding process. The transcoded audio output streams may be synchronized such that audio frame boundaries contain identical pulse-code modulation (PCM) samples, packetized elementary stream (PES) location, and presentation time stamp (PTS) value.
1. A system for inter-chassis synchronization between audio output streams, the system comprising:
a first adaptive transcoder in a first chassis for generating multiple transcoded audio output streams from an input to the first adaptive transcoder, and synchronizing the multiple transcoded audio output streams based on corresponding segments;
a second adaptive transcoder in a second chassis, the second adaptive transcoder including:
at least one input for receiving an input having audio content matching audio content in the input to the first adaptive transcoder;
one or more transcoding processors generating multiple transcoded audio output streams from the input stream and synchronizing the multiple transcoded audio output streams representations based on corresponding segments, wherein each transcoding processor has an audio encoder for inserting presentation time stamps in the segments of the transcoded audio output streams;
the second chassis synchronizing a second group of transcoded audio output streams output from the second transcoder with a first group of transcoded audio output streams output from the first transcoder by:
receiving, from the first chassis, video segment presentation time stamp and corresponding audio aligned presentation time stamp pairs for the input to the first adaptive transcoder;
storing the pairs of video segment presentation time stamp and audio aligned presentation time stamp data from the first chassis to a memory location in the second chassis designated for first transcoder sync information;
identifying and storing video segment presentation time stamp and corresponding audio aligned presentation time stamp pairs for the input to the second adaptive transcoder to a memory location in the second chassis designated for second transcoder sync information;
for a video segment presentation time stamp received at the input to the second transcoder, searching for a first transcoder video segment presentation time stamp in the memory location designated for first transcoder sync information that matches the video segment presentation time stamp received at the input at the second transcoder
compare an audio aligned presentation time stamp paired to a matching first transcoder video segment presentation time stamp to a second transcoder audio aligned presentation time stamp paired with the received video segment presentation time stamp;
determine an audio sync presentation time stamp offset based on the difference between the first transcoder's audio aligned presentation time stamp to the second transcoder's audio aligned presentation time stamp; and
based on the audio sync presentation time stamp offset, adjusting the presentation time stamp inserted by the audio encoder in segments of the transcoded audio output streams to synchronize the transcoded audio output streams output from the second transcoder with the transcoded audio output streams output from the first transcoder.
2. The system of claim 1 , wherein the audio aligned presentation time stamp is associated with an audio packetized elementary stream (PES).
3. The system of claim 1 , wherein the video segment presentation time stamp is associated with a video instantaneous decoder refresh (IDR) frame.
4. The system of claim 1 , wherein the first transcoder and second transcoder synchronize their respective transcoded audio output streams based on group of picture (GOP)/ instantaneous decoder refresh (IDR) frames.
5. The system of claim 4 , wherein the first group of transcoded audio output streams output from the first transcoder to be synchronized with the second group of transcoded audio output streams output from the second transcoder is based on a pairing of the first group to the second group based on corresponding GOP/IDR synchronizations.
6. The system of claim 1 , wherein the audio aligned PTS at the input of the Audio Encoder is adjusted at an Audio PCM Sample Adjuster by dropping PCM samples at the input of the Audio Encoder.
7. The system of claim 1 , wherein the audio aligned presentation time stamps paired to matching first transcoder video segment presentation time stamps are synchronization points.
8. The system of claim 7 , wherein at the synchronization points, the secondary transcoder adjusts audio frame boundaries so the synchronization points between transcoded output streams contain audio transport packets that start on identical pulse-code modulation (PCM) sample, packetized elementary stream (PES) locations, and presentation time stamp (PTS) values.
9. The system of claim 1 , wherein the one or more transcoding processors comprise:
a demultiplexer for demuxing the input stream;
an audio decoder for decoding an audio portion of the input stream, wherein the audio encoders encode the audio portion of the input stream and generate the multiple transcoded audio output streams;
the transcoder further comprising a multiplexer for multiplexing the transcoded audio outputs with video to output transport streams.
10. The system of claim 9 , wherein the search for the first transcoder video segment presentation time stamp that matches the video segment presentation time stamp received at the input at the second transcoder is performed at the multiplexing output of the second transcoder.
11. The system of claim 1 , wherein when second transcoder audio aligned presentation time stamp paired with the received video segment presentation time stamp is behind the audio aligned presentation time stamp paired to a matching first transcoder video segment presentation time stamp, the secondary transcoder will use the audio sync presentation time stamp offset to adjust the audio aligned PTS at the input of the audio encoder by dropping pulse-code modulation (PCM) samples at the input of the audio encoder.
12. The system of claim 1 , wherein when second transcoder audio aligned presentation time stamp paired with the received video segment presentation time stamp is ahead of the audio aligned presentation time stamp paired to a matching first transcoder video segment presentation time stamp, the secondary transcoder waits for the presentation time stamp of the first group audio to sync to the presentation time stamp of the second group.
13. The system of claim 1 , wherein the audio transcoded output streams in the first and second groups have a same audio type but vary in audio output rates.
14. The system of claim 1 , wherein one or more additional second transcoders receive sync information from the first chassis for synchronizing transcoded audio output streams output from the one or more additional second transcoder to the first group of transcoded audio output streams output from the first transcoder.
15. A method for inter-chassis synchronization of a group of transcoded audio output transport streams at a second chassis with a group of transcoded audio output transport streams at a first chassis, the method comprising:
receiving at the second chassis, from the first chassis, video segment presentation time stamp and corresponding audio aligned presentation time stamp pairs for the input to the first adaptive transcoder;
storing the pairs of video segment presentation time stamp and audio aligned presentation time stamp data from the first chassis to a memory location in the second chassis designated for first transcoder sync information;
identifying and storing video segment presentation time stamp and corresponding audio aligned presentation time stamp pairs for the input to the second adaptive transcoder to a memory location in the second chassis designated for second transcoder sync information;
for a video segment presentation time stamp received at the input to the second transcoder, searching for a first transcoder video segment presentation time stamp in the memory location designated for first transcoder sync information that matches the video segment presentation time stamp received at the input at the second transcoder
comparing an audio aligned presentation time stamp paired to the matching first transcoder video segment presentation time stamp to a second transcoder audio aligned presentation time stamp paired with the received video segment presentation time stamp;
determining an audio sync presentation time stamp offset based on the difference between the first transcoder's audio aligned presentation time stamp to the second transcoder's audio aligned presentation time stamp; and
based on the audio sync presentation time stamp offset, adjusting the presentation time stamp included by an audio encoder in segments of the transcoded audio output streams, wherein each transcoding processor has an audio encoder for including a presentation time stamps in the segments of the transcoded audio output streams, thereby synchronizing the second group of transcoded audio output streams output from the second transcoder with the first group of transcoded audio output streams output from the first transcoder.
16. The method of claim 15 , wherein an audio source at the first transcoder system is ahead or behind an audio source at one or more secondary transcoders.
17. The method of claim 15 , wherein the audio aligned presentation time stamp is associated with an audio packetized elementary stream (PES), and the video segment presentation time stamp is associated with a video instantaneous decoder refresh (IDR) frame.
18. The method of claim 15 , wherein the transcoded audio output streams across the first and second group are synchronized based on group of picture (GOP)/ instantaneous decoder refresh (IDR) frames.
19. The method of claim 15 , wherein the audio aligned presentation time stamp (PTS) at the input of the audio encoder is adjusted at an audio pulse-code modulation (PCM) sample adjuster by dropping pulse-code modulation (PCM) samples at the input of the Audio Encoder.
20. The method of claim 15 , wherein when second transcoder audio aligned presentation time stamp paired with the received video segment presentation time stamp is behind the audio aligned presentation time stamp paired to a matching first transcoder video segment presentation time stamp, the secondary transcoder will use the audio sync presentation time stamp offset to adjust the audio aligned PTS at the input of the audio encoder by dropping pulse-code modulation (PCM) samples at the input of the audio encoder.