IP Library Granted Patent US 9,118,744
Granted Patent B2
US 9,118,744 · App. 13/561,069 · Granted Aug 25, 2015

Replacing lost media data for network streaming

Inventors: Thadi M. Nagaraj (San Diego, CA); Hariharan Sukumar (San Diego, CA); Pratik Kotkar (San Diego, CA); Shailesh Maheshwari (San Diego, CA); Daniel Amerga (San Diego, CA)
Assignee: QUALCOMM Incorporated
H04L65/604H04L65/608H04L65/80H04N19/00H04N21/44209H04N21/64322H04N21/8456H04N21/85406
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,118,744
App. No.
13/561,069
Granted
Aug 25, 2015
Kind
B2
Abstract

In one example, a device includes one or more processors configured to determine that data for at least a portion of a segment of media data has been lost after a remaining portion of the segment has been received via a network transmission in accordance with dynamic adaptive streaming over HTTP (DASH), based on the determination, prior to decoding the media data, add default data to the segment to replace the data that has been determined to be lost to form a replacement segment, and output media data of the replacement segment.

Claims (58)

1. A method of processing media data, the method comprising, by a processing unit separate from a decoder and that processes the media data before frames of the media data are decoded by the decoder:

determining that data for at least a portion of a segment of media data has been lost after a remaining portion of the segment has been received via a network transmission in accordance with dynamic adaptive streaming over HTTP (DASH), wherein the media data of the segment includes a first plurality of frames, wherein the first plurality of frames are coded without reference to frames external to the segment, and wherein the data that has been lost includes a second plurality of frames that is a subset of the first plurality of frames;

based on the determination, prior to the decoder decoding any of the frames of the media data of the segment, adding default data to the segment to replace the data that has been determined to be lost to form a replacement segment such that a duration of the added default data is equal to a duration of the number of frames in the second plurality of frames; and

outputting media data of the replacement segment to the decoder.

2. The method of claim 1 , wherein determining that the data for the at least portion of the segment has been lost comprises determining that at least one packet of the segment has not arrived or has been corrupted.

3. The method of claim 1 , wherein adding default data comprises adding at least one of default audio data and default video data to the segment.

4. The method of claim 1 , wherein adding default data comprises adding default video data to the segment, and wherein the default video data comprises one of a logo screen of a network from which the remaining portion of the segment was received, a seasonally relevant display, and a content-relevant display.

5. The method of claim 1 , wherein adding default data comprises adding content-relevant audio data to the segment.

6. The method of claim 1 , further comprising modifying a header of the segment based on the addition of the default data.

7. The method of claim 1 , further comprising receiving information defining the default data.

8. The method of claim 7 , wherein receiving the information defining the default data comprises receiving the information defining the default data as part of the network transmission.

9. The method of claim 7 , wherein receiving the information defining the default data comprises receiving the information defining the default data as side information.

10. The method of claim 1 , further comprising retrieving configuration data indicative of the default data.

11. The method of claim 1 , wherein the media data comprises video data, wherein the decoder comprises a video decoder, and wherein outputting comprises sending the video data of the replacement segment to the video decoder.

12. The method of claim 11 , wherein adding the default data comprises assigning values for at least one of a frame number and a picture order count (POC) value to pictures of the default data.

13. The method of claim 1 , wherein the media data comprises audio data, wherein the decoder comprises an audio decoder, and wherein outputting comprises sending the audio data of the replacement segment to the audio decoder.

14. The method of claim 1 , wherein the default data comprises data that is separate and independent from the media data and does not result from loss correction or corruption correction techniques.

15. The method of claim 1 , further comprising receiving the default data prior to receiving any data of the segment.

16. A device for processing media data, the device comprising:

a decoder; and

one or more processors separate from the decoder and that the media data before frames of the media data are decoded by the decoder, wherein the one or more processors are configured to:

determine that data for at least a portion of a segment of media data has been lost after a remaining portion of the segment has been received via a network transmission in accordance with dynamic adaptive streaming over HTTP (DASH), wherein the media data of the segment includes a first plurality of frames, wherein the first plurality of frames are coded without reference to frames external to the segment, and wherein the data that has been lost includes a second plurality of frames that is a subset of the first plurality of frames;

based on the determination, prior to the decoder decoding any of the frames of the media data of the segment, add default data to the segment to replace the data that has been determined to be lost to form a replacement segment such that a duration of the added default data is equal to a duration of the number of frames in the second plurality of frames; and

output media data of the replacement segment to the decoder.

17. The device of claim 16 , wherein the one or more processors are configured to determine that the data for the at least portion of the segment has been lost when at least one packet of the segment has not arrived or has been corrupted.

18. The device of claim 16 , wherein the one or more processors are configured to add at least one of default audio data and default video data to the segment.

19. The device of claim 16 , wherein the one or more processors are further configured to receive information defining the default data.

20. The device of claim 16 , wherein the decoder comprises a video decoder, wherein the media data comprises video data, and wherein the one or more processors are configured to send the video data of the replacement segment to the video decoder.

21. The device of claim 20 , wherein the one or more processors are configured to assign values for at least one of a frame number and a picture order count (POC) value to pictures of the default data.

22. The device of claim 16 , wherein the default data comprises data that is separate and independent from the media data and does not result from loss correction or corruption correction techniques.

23. The device of claim 16 , wherein the one or more processors are configured to receive the default data prior to receiving any data of the segment.

24. A device for processing media data, the device comprising:

a decoder;

means, separate from the decoder and that processes the media data before frames of the media data are decoded by the decoder, for determining that data for at least a portion of a segment of media data has been lost after a remaining portion of the segment has been received via a network transmission in accordance with dynamic adaptive streaming over HTTP (DASH), wherein the media data of the segment includes a first plurality of frames, wherein the first plurality of frames are coded without reference to frames external to the segment, and wherein the data that has been lost includes a second plurality of frames that is a subset of the first plurality of frames;

means, separate from the decoder and that processes the media data before frames of the media data are decoded by the decoder, for adding, based on the determination, prior to the decoder decoding any of the frames of the media data of the segment, default data to the segment to replace the data that has been determined to be lost to form a replacement segment such that a duration of the added default data is equal to a duration of the number of frames in the second plurality of frames; and

means, separate from the decoder and that processes the media data before frames of the media data are decoded by the decoder, for outputting media data of the replacement segment to the decoder.

25. The device of claim 24 , wherein the means for determining that the data for the at least portion of the segment has been lost comprises means for determining that at least one packet of the segment has not arrived or has been corrupted.

26. The device of claim 24 , wherein the means for adding default data comprises means for adding at least one of default audio data and default video data to the segment.

27. The device of claim 24 , further comprising means for receiving information defining the default data.

28. The device of claim 27 , wherein the means for receiving the information defining the default data comprises means for receiving the information defining the default data as part of the network transmission.

29. The device of claim 27 , wherein the means for receiving the information defining the default data comprises means for receiving the information defining the default data as side information.

30. The device of claim 24 , wherein the decoder comprises a video decoder, wherein the media data comprises video data, and wherein the means for outputting comprises means for sending the video data of the replacement segment to the video decoder.

31. The device of claim 30 , wherein the means for adding the default data comprises means for assigning values for at least one of a frame number and a picture order count (POC) value to pictures of the default data.

32. The device of claim 24 , wherein the default data comprises data that is separate and independent from the media data and does not result from loss correction or corruption correction techniques.

33. The device of claim 24 , further comprising means for receiving the default data prior to receiving any data of the segment.

34. A computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors, separate from a decoder and that process the media data before frames of the media data are decoded by the decoder, to:

determine that data for at least a portion of a segment of media data has been lost after a remaining portion of the segment has been received via a network transmission in accordance with dynamic adaptive streaming over HTTP (DASH), wherein the media data of the segment includes a first plurality of frames, wherein the first plurality of frames are coded without reference to frames external to the segment, and wherein the data that has been lost includes a second plurality of frames that is a subset of the first plurality of frames;

based on the determination, prior to the decoder decoding any of the frames of the media data of the segment, add default data to the segment to replace the data that has been determined to be lost to form a replacement segment such that a duration of the added default data is equal to a duration of the number of frames in the second plurality of frames; and

output media data of the replacement segment to the decoder.

35. The computer-readable storage medium of claim 34 , wherein the instructions that cause the one or more processors to determine that the data for the at least portion of the segment has been lost comprise instructions that cause the one or more processors to determine that at least one packet of the segment has not arrived or has been corrupted.

36. The computer-readable storage medium of claim 34 , wherein the instructions that cause the one or more processors to add default data comprise instructions that cause the one or more processors to add at least one of default audio data and default video data to the segment.

37. The computer-readable storage medium of claim 34 , further comprising instructions that cause the one or more processors to receive information defining the default data.

38. The computer-readable storage medium of claim 37 , wherein the instructions that cause the one or more processors to receive the information defining the default data comprise instructions that cause the one or more processors to receive the information defining the default data as part of the network transmission.

39. The computer-readable storage medium of claim 37 , wherein the instructions that cause the one or more processors to receive the information defining the default data comprise instructions that cause the one or more processors to receive the information defining the default data as side information.

40. The computer-readable storage medium of claim 34 , wherein the media data comprises video data, wherein the decoder comprises a video decoder, and wherein the instructions that cause the one or more processors to output comprise instructions that cause the one or more processors to send the video data of the replacement segment to the video decoder.

41. The computer-readable storage medium of claim 40 , wherein the instructions that cause the one or more processors to add the default data comprise instructions that cause the one or more processors to assign values for at least one of a frame number and a picture order count (POC) value to pictures of the default data.

42. The computer-readable storage medium of claim 34 , wherein the default data comprises data that is separate and independent from the media data and does not result from loss correction or corruption correction techniques.

43. The computer-readable storage medium of claim 34 , further comprising instructions that cause the processor to receive the default data prior to receiving any data of the segment.

Assignments (2)
CORRECTIVE ASSIGNMENT TO CORRECT MISSING APPLICATION NUMBER AND DATE INSIDE THE ASSIGNMENT DOCUMENT PREVIOUSLY RECORDED AT REEL: 028753 FRAME: 0070. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Apr 21, 2015
From: NAGARAJ, THADI M; SUKUMAR, HARIHARAN; KOTKAR, PRATIK; MAHESHWARI, SHAILESH; AMERGA, DANIEL
To: QUALCOMM INCORPORATED
Reel/Frame 035475/0683 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 9, 2012
From: NAGARAJ, THADI M.; SUKUMAR, HARIHARAN; KOTKAR, PRATIK; MAHESHWARI, SHAILESH; AMERGA, DANIEL
To: QUALCOMM INCORPORATED
Reel/Frame 028753/0070 →
Continuity (1)
Related Publication 20140032987A1 · Jan 30, 2014