Multiplexing, synchronizing, and assembling multiple audio/video (A/V) streams in a media gateway
View Patent ↗A media gateway that services a plurality of client devices may be used to combine a plurality of encoded multimedia transport streams, which may comprise a buffered copy of received encoded broadcast multimedia transport stream and an encoded local multimedia transport stream that pertains to the broadcast transport stream. The media gateway may decode the encoded broadcast multimedia transport stream to extract broadcast multimedia content carried therein based on determination of capabilities of one or more client devices used in playback of the content. The encoded local multimedia transport stream may be generated based on local multimedia content generated and/or captured via one or more client devices. The buffering duration of the received encoded broadcast multimedia transport stream may be adaptively determined, to ensure that the encoded local multimedia transport stream and the encoded broadcast multimedia transport stream are synchronized when they are combined.
1. A method, comprising:
receiving, by a media gateway, an encoded broadcast multimedia transport stream from a service provider over a delivery network, the encoded broadcast multimedia transport stream comprising broadcast multimedia content for rendering via at least one of a plurality of client devices;
decoding, by the media gateway during a first time period, the encoded broadcast multimedia transport stream to extract said broadcast multimedia content;
receiving, by the media gateway, a local multimedia transport stream generated by at least one of the plurality of client devices;
encoding, by the media gateway during a second time period subsequent to the first time period, the received local multimedia transport stream;
buffering, by the media gateway, the encoded broadcast multimedia transport stream for a third time period comprising a sum of the first time period and the second time period; and
multiplexing, by the media gateway subsequent to the third time period, the buffered encoded broadcast multimedia transport stream and the encoded local multimedia transport stream into an output multimedia transport stream.
2. The method according to claim 1 , wherein encoding the local multimedia transport stream further comprises encoding video information and/or audio information related to rendering of the broadcast multimedia content.
3. The method according to claim 1 , further comprising determining, by the media gateway, a duration of the third time period.
4. The method according to claim 3 , wherein determining the duration of the third time period further comprises determining a duration of the first time period and determining a duration of the second time period; and wherein multiplexing the buffered encoded broadcast multimedia transport stream and the encoded local multimedia transport stream further comprises synchronizing the encoded broadcast multimedia transport stream and the encoded local multimedia transport stream.
5. The method according to claim 4 , further comprising displaying the decoded broadcast multimedia transport stream subsequent to the first time period, and wherein receiving the local multimedia transport stream further comprises receiving the local multimedia transport stream subsequent to the first time period.
6. The method according to claim 5 , wherein receiving the local multimedia transport stream subsequent to the first time period further comprises receiving the local multimedia transport stream during display of the decoded broadcast multimedia transport stream.
7. The method according to claim 1 , wherein the third time period further comprises a sum of the first time period; the second time period; and a fourth time period, subsequent to the first time period and prior to the second time period, during display of the decoded broadcast multimedia transport stream and generation of the local multimedia transport stream.
8. The method according to claim 7 , further comprising determining, by the media gateway, the durations of the first time period, second time period, and fourth time period based on frame rates, audio and video resolutions, and/or CODEC types of the encoded broadcast multimedia transport stream and the encoded local multimedia transport stream.
9. A system, comprising:
a media gateway comprising one or more circuits configured to:
receive an encoded broadcast multimedia transport stream from a service provider over a delivery network, the encoded broadcast multimedia transport stream comprising broadcast multimedia content for rendering via at least one of a plurality of client devices;
decode said encoded broadcast multimedia transport stream, during a first time period, to extract said broadcast multimedia content;
receive a local multimedia transport stream generated by at least one of the plurality of client devices;
encode, during a second time period subsequent to the first time period, the received local multimedia transport stream;
buffer the encoded broadcast multimedia transport stream for a third time period comprising a sum of the first time period and the second time period; and
multiplexing, subsequent to the third time period, the buffered encoded broadcast multimedia transport stream and the encoded local multimedia transport stream into an output multimedia transport stream.
10. The system according to claim 9 , wherein the one or more circuits are further configured to encode video information and/or audio information related to rendering of the broadcast multimedia content.
11. The system according to claim 9 , wherein the one or more circuits are further configured to adaptively determine a duration of the third time period.
12. The system according to claim 11 , wherein the one or more circuits are further configured to determine a duration of the first time period; determine a duration of the second time period; and synchronize the encoded broadcast multimedia transport stream and the encoded local multimedia transport stream.
13. The system according to claim 12 , wherein the one or more circuits are further configured to provide the decoded broadcast multimedia transport stream for display subsequent to the first time period, and receive the local multimedia transport stream subsequent to the first time period.
14. The system according to claim 13 , wherein the one or more circuits are further configured to receive the local multimedia transport stream during display of the decoded broadcast multimedia transport stream.
15. The system according to claim 9 , wherein the third time period further comprises a sum of the first time period; the second time period; and a fourth time period, subsequent to the first time period and prior to the second time period, during display of the decoded broadcast multimedia transport stream and generation of the local multimedia transport stream.
16. The system according to claim 15 , wherein the one or more circuits are further configured to determine the durations of the first time period, second time period, and fourth time period based on frame rates, audio and video resolutions, and/or CODEC types of the encoded broadcast multimedia transport stream and the encoded local multimedia transport stream.
17. The system according to claim 9 , wherein the one or more circuits are further configured to decode the encoded broadcast multimedia transport stream according to an audio resolution capability or a video resolution capability of at least one of the plurality of client devices.
18. A method, comprising:
receiving, by a media gateway, an encoded broadcast multimedia transport stream from a service provider over a delivery network, the encoded broadcast multimedia transport stream comprising broadcast multimedia content for rendering via a client device;
decoding, by the media gateway during a first time period, the encoded broadcast multimedia transport stream to extract said broadcast multimedia content, the first time period comprising a time to decode the broadcast multimedia transport stream in accordance with a resolution capability of at least one of said plurality of client devices;
determining, by the media gateway, a third time period for buffering the encoded broadcast multimedia transport stream based on a sum of the first time period and a second time period for encoding a local multimedia transport stream generated by the client device;
buffering, by the media gateway, the encoded broadcast multimedia transport stream for the third time period; and
multiplexing, by the media gateway subsequent to the third time period, the buffered encoded broadcast multimedia transport stream and the encoded local multimedia transport stream.
19. The method according to claim 18 , further comprising receiving, by the media gateway, the local multimedia transport stream and encoding the local multimedia transport stream during the second time period.
20. The method according to claim 18 , further comprising determining, by the media gateway, the durations of the first time period and second time period delay according to frame rates, audio and video resolutions, and/or CODEC types of the encoded broadcast multimedia transport stream and the encoded local multimedia transport stream.
21. The method according to claim 1 , wherein decoding the encoded broadcast multimedia transport stream to extract said broadcast multimedia content further comprises decoding the encoded broadcast multimedia transport stream according to an audio resolution capability or a video resolution capability of at least one of the plurality of client devices.