IP Library Granted Patent US 8,532,171
Granted Patent B1
US 8,532,171 · App. 12/978,384 · Granted Sep 10, 2013

Multiple stream adaptive bit rate system

Inventors: Karthik Narayanan (Chennai, IN); Zubair Visharam (Santa Clara, CA); Sunil Mukundan (Chennai, IN)
Assignee: Juniper Networks, Inc.
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,532,171
App. No.
12/978,384
Granted
Sep 10, 2013
Kind
B1
Abstract

A method and apparatus for segmenting multiple live media streams of differing qualities into a single output stream. The live media streams may have constant or unpredictable key frame intervals.

Claims (104)

1. A method comprising:

receiving, by a device, a plurality of media streams,

each media stream being associated with a different bit rate for a particular content;

determining, by the device, key frames associated with the plurality of received media streams,

each key frame being associated with a particular received media stream of the plurality of received media streams, and

each key frame being associated with a respective arrival time;

determining, by the device and based on the respective arrival time associated with each key frame, a lock-in time based on a last received key frame of the determined key frames;

determining, by the device and when each of the plurality of media streams reaches the lock-in time, a first occurrence of a key frame of the determined key frames; and

inserting, by the device and based on the first occurrence of the key frame, segments of each of the plurality of received media streams into an output stream at specific time intervals.

2. The method of claim 1 , further comprising:

revising the lock-in time; and

using the revised lock-in time to determine the first occurrence of the key frame.

3. The method of claim 2 , further comprising:

revising the lock-in time when one or more key frame intervals are different than an initial offset.

4. The method of claim 1 , further comprising:

time-aligning at least one of the key frames across segments of each of the plurality of received media streams prior to inserting the segments into the output stream.

5. The method of claim 1 , where the plurality of media streams are live media streams.

6. The method of claim 1 , where the plurality of media streams are media streams from a plurality of cameras.

7. The method of claim 1 , where the plurality of media streams include related audio and video streams.

8. The method of claim 1 , where the respective arrival times of the key frames are constant.

9. The method of claim 1 , where each segment contains at least one key frame.

10. The method of claim 1 , where the plurality of media streams include at least one of the following types of streams: H.264, AAC, MPEG-2 SPTS, or MPEG-2 MPTS.

11. A method comprising:

receiving, by a device, a plurality of multimedia streams,

the plurality of multimedia streams including related multimedia streams;

determining, by the device, a key frame associated with each multimedia stream of the plurality of received multimedia streams,

each key frame being associated with a particular received media stream of the plurality of received media streams, and

each key frame being associated with a respective arrival time;

determining, by the device and based on the respective arrival time associated with each key frame, a lock-in time based on a last received key frame of the determined key frames;

determining, by the device and when each of the plurality of media streams reaches the lock-in time, a first occurrence of a key frame of the determined key frames;

inserting, by the device and based on the first occurrence of the key frame, segments of a first multimedia stream, of the plurality of received multimedia streams, into an output stream; and

inserting, by the device and based on the first occurrence of the key frame, segments of a second multimedia stream, of the plurality of received multimedia streams, into the output stream to align with the inserted segments of the first multimedia stream.

12. The method of claim 11 , further comprising:

adjusting a playback length of the second multimedia stream to match a playback length of the first multimedia stream by creating smaller segments of the second multimedia stream for insertion into the output stream.

13. A non-transitory computer readable storage medium storing instructions, the instructions comprising:

one or more instructions which, when executed by at least one processor, cause the at least one processor to:

receive a plurality of media streams,

each media stream being associated with a different bit rate for particular content;

determine key frames associated with the plurality of received media streams,

each key frame being associated with a particular received media stream of the plurality of received media streams, and

each key frame being associated with a respective arrival time;

determine, based on the respective arrival time associated with each key frame, a lock-in time based on a last received key frame of the determined key frames;

determine, when each of the plurality of media streams reaches the lock-in time, a first occurrence of a key frame of the determined key frames; and

insert, based on the first occurrence of the key frame, segments of each of the plurality of received media streams into an output stream at specific time intervals.

14. The non-transitory computer readable storage medium of claim 13 , where the instructions further comprise:

one or more instructions to revise the lock-in time; and

one or more instructions to use the revised lock-in time to determine the first occurrence of the key frame.

15. The non-transitory computer readable storage medium of claim 14 , where the instructions further comprise:

one or more instructions to revise the lock-in time when one or more key frame intervals are different than an initial offset.

16. The non-transitory computer readable storage medium of claim 13 , where the instructions further comprise:

one or more instructions to time-align at least one of the key frames across segments of each of the plurality of received media streams prior to inserting the segments into the output stream.

17. The non-transitory computer readable storage medium of claim 13 , where the plurality of media streams are live media streams.

18. The non-transitory computer readable storage medium of claim 13 , where the plurality of media streams are media streams from a plurality of cameras.

19. The non-transitory computer readable storage medium of claim 13 , where the plurality of media streams include related audio and video streams.

20. The non-transitory computer readable storage medium of claim 13 , where the respective arrival times of the key frames are constant.

21. The non-transitory computer readable storage medium of claim 13 , where each segment contains at least one key frame.

22. The non-transitory computer readable storage medium of claim 13 , where the plurality of media streams include at least one of the following types of streams: H.264, AAC, MPEG-2 SPTS, or MPEG-2 MPTS.

23. A non-transitory computer readable storage medium storing instructions, the instructions comprising:

one or more instructions which, when executed by at least one processor, cause the at least one processor to:

receive a plurality of multimedia streams,

the plurality of multimedia streams including related multimedia streams;

determine a key frame associated with each multimedia stream of the plurality of received multimedia streams,

each key frame being associated with a particular received media stream of the plurality of received media streams, and

each key frame being associated with a respective arrival time;

determine, based on the respective arrival time associated with each key frame, a lock-in time based on a last received key frame of the determined key frames;

determine, when each of the plurality of media streams reaches the lock-in time, a first occurrence of a key frame of the determined key frames;

insert, based on the first occurrence of the key frame, segments of a first multimedia stream, of the plurality of received multimedia streams, into an output stream; and

insert, based on the first occurrence of the key frame, segments of a second multimedia stream, of the plurality of received multimedia streams, into the output stream to align with the inserted segments of the first multimedia stream.

24. The non-transitory computer readable storage medium of claim 23 , where the instructions further comprise:

one or more instructions to adjust a playback length of the second multimedia stream to match a playback length of the first multimedia stream by creating smaller segments of the second multimedia stream for insertion into the output stream.

25. A system comprising:

one or more devices to:

receive a plurality of media streams,

each media stream being associated with a different bit rate for particular content;

determine key frames associated with the plurality of received media streams,

each key frame being associated with a particular received media stream of the plurality of received media streams, and

each key frame being associated with a respective arrival time;

determine, based on the respective arrival time associated with each key frame, a lock-in time based on a last received key frame of the determined key frames;

determine, when each of the plurality of media streams reaches the lock-in time, a first occurrence of a key frame of the determined key frames; and insert, based on the first occurrence of the key frame, segments of each of the plurality of received media streams into an output stream at specific time intervals.

26. The system of claim 25 , where the one or more devices are further to:

revise the lock-in time; and

use the revised lock-in time to determine the first occurrence of the key frame.

27. The system of claim 26 , the one or more devices are further to:

revise the lock-in time when one or more key frame intervals are different than an initial offset.

28. The system of claim 25 , where the one or more devices are further to:

time-align at least one of the key frames across segments of each of the plurality of received media streams prior to inserting the segments into the output stream.

29. The system of claim 25 , where the plurality of media streams are live media streams.

30. The system of claim 25 , where the plurality of media streams are media streams from a plurality of cameras.

31. The system of claim 25 , where the plurality of media streams include related audio and video streams.

32. The system of claim 25 , where the respective arrival times of the key frames are constant.

33. The system of claim 25 , where each segment contains at least one key frame.

34. The system of claim 25 , where the plurality of media streams include at least one of the following types of streams: H.264, AAC, MPEG-2 SPTS, or MPEG-2 MPTS.

35. A system comprising:

one or more devices to:

receive a plurality of multimedia streams, the plurality of multimedia streams including related multimedia streams;

determine a key frame associated with each multimedia stream of the plurality of received multimedia streams,

each key frame being associated with a particular received media stream of the plurality of received media streams, and

each key frame being associated with a respective arrival time;

determine, based on the respective arrival time associated with each key frame, a lock-in time based on a last received key frame of the determined key frames;

determine, when each of the plurality of media streams reaches the lock-in time, a first occurrence of a key frame of the determined key frames

insert, based on the first occurrence of the key frame, segments of a first multimedia stream, of the plurality of received multimedia streams, into an output stream; and

insert, based on the first occurrence of the key frame, segments of a second multimedia stream, of the plurality of received multimedia streams, into the output stream to align with the inserted segments of the first multimedia stream.

36. The system of claim 35 , where the one or more devices are further to:

adjust a playback length of the second multimedia stream to match a playback length of the first multimedia stream by creating smaller segments of the second multimedia stream for insertion into the output stream.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 19, 2011
From: ANKEENA NETWORKS, INC.
To: JUNIPER NETWORKS, INC.
Reel/Frame 026925/0618 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 6, 2011
From: NARAYANAN, KARTHIK; VISHARAM, ZUBAIR; MUKUNDAN, SUNIL
To: JUNIPER NETWORKS, INC.
Reel/Frame 025598/0819 →