Low latency video streaming
A client device includes one or more processors configured to determine, from a manifest file, a plurality of types of segments included in a representation of media content, one or more functions provided by each of the types of segments, and positions of segments conforming to each of the types of segments in the representation, wherein at least one of the types of segments provides a point at which to begin retrieving data from the representation, determine, from the manifest file, a segment of the representation conforming to the type that provides the point at which to begin retrieving data from the representation, and retrieve the determined segment from the representation.
1. A method of retrieving media data, the method comprising:
determining, from a manifest file, a plurality of types of segments included in a representation of media content, one or more functions provided by each of the types of segments, and positions of segments conforming to each of the types of segments in the representation, wherein at least one of the types of segments provides a point at which to begin retrieving data from the representation, and wherein the plurality of types of segments include:
a delivery unit media segment format that indicates that corresponding segments include media data;
a random access media segment format that indicates that corresponding segments provide points at which retrieving data from the representation can begin;
a no overlap segment format that indicates that corresponding segments do not overlap starting and ending times of other segments in the representation and other segments in other representations in an adaptation set including the representation; and
a switching media segment format that indicates that corresponding segments provide points at which retrieval can switch to the representation from the other representations in the adaptation set without reinitialization;
determining, from the manifest file, a segment of the representation conforming to the type that provides the point at which to begin retrieving data from the representation; and
retrieving the determined segment from the representation.
2. The method of claim 1 , wherein the delivery unit media segment format indicates that corresponding segments:
contain a value of “dums” in a segment type box of the segment;
include one or more self-contained movie fragments, wherein each self-contained movie fragment comprises a movie fragment (“moof”) box and a media data (“mdat”) box that contains media samples that do not use external data references referenced by a track in the movie fragment box;
each of the moof boxes contains at least one track fragment;
each of the moof boxes does not use external references;
a “default-base-is-moof” flag of the media segment is set to true; and
a “base-data-offset-present” flag of the media segment is set to false.
3. The method of claim 1 , wherein the random access media segment format indicates that corresponding segments:
comprise random access segments, an ordinal first access unit in each movie fragment of the segments corresponds to an I SAU of a stream access point (SAP) of type 1, 2, or 3; and
include all necessary information to access media data in a bitstream following the segments.
4. The method of claim 3 , wherein the random access media segment format indicates that corresponding segments comprise at least one of an instantaneous decoder refresh (IDR) picture, a broken link access (BLA) picture, or a clean random access (CRA) picture.
5. The method of claim 1 , wherein the random access media segment format indicates that corresponding segments include one or more segment index (“sidx”) boxes, and wherein an ordinal first sidx box precedes all moof boxes of the media segment and describes the entire media segment.
6. The method of claim 1 , wherein the switching media segment format indicates that an ordinal first sample in an ordinal first movie fragment of the corresponding segments corresponds to an I SAU of a stream access point (SAP) of type 1 or 2.
7. A client device for retrieving media data, the client device comprising one or more processors configured to:
determine, from a manifest file, a plurality of types of segments included in a representation of media content, one or more functions provided by each of the types of segments, and positions of segments conforming to each of the types of segments in the representation, wherein at least one of the types of segments provides a point at which to begin retrieving data from the representation, and wherein the plurality of types of segments include:
a delivery unit media segment format that indicates that corresponding segments include media data;
a random access media segment format that indicates that corresponding segments provide points at which retrieving data from the representation can begin;
a no overlap segment format that indicates that corresponding segments do not overlap starting and ending times of other segments in the representation and other segments in other representations in an adaptation set including the representation; and
a switching media segment format that indicates that corresponding segments provide points at which retrieval can switch to the representation from the other representations in the adaptation set without reinitialization;
determine, from the manifest file, a segment of the representation conforming to the type that provides the point at which to begin retrieving data from the representation; and
retrieve the determined segment from the representation.
8. The client device of claim 7 , wherein the delivery unit media segment format indicates that corresponding segments:
contain a value of “dums” in a segment type box of the segment;
include one or more self-contained movie fragments, wherein each self-contained movie fragment comprises a movie fragment (“moof”) box and a media data (“mdat”) box that contains media samples that do not use external data references referenced by a track in the movie fragment box;
each of the moof boxes contains at least one track fragment;
each of the moof boxes does not use external references;
a “default-base-is-moof” flag of the media segment is set to true; and
a “base-data-offset-present” flag of the media segment is set to false.
9. The client device of claim 7 , wherein the random access media segment format indicates that corresponding segments:
comprise random access segments, an ordinal first access unit in each movie fragment of the segments corresponds to an I SAU of a stream access point (SAP) of type 1, 2, or 3; and
include all necessary information to access media data in a bitstream following the segments.
10. The client device of claim 9 , wherein the random access media segment format indicates that corresponding segments comprise at least one of an instantaneous decoder refresh (IDR) picture, a broken link access (BLA) picture, or a clean random access (CRA) picture.
11. The client device of claim 7 , wherein the random access media segment format indicates that corresponding segments include one or more segment index (“sidx”) boxes, and wherein an ordinal first sidx box precedes all moof boxes of the media segment and describes the entire media segment.
12. The client device of claim 7 , wherein the switching media segment format indicates that an ordinal first sample in an ordinal first movie fragment of the corresponding segments corresponds to an I SAU of a stream access point (SAP) of type 1 or 2.
13. A client device for retrieving media data, the client device comprising:
means for determining, from a manifest file, a plurality of types of segments included in a representation of media content, one or more functions provided by each of the types of segments, and positions of segments conforming to each of the types of segments in the representation, wherein at least one of the types of segments provides a point at which to begin retrieving data from the representation, and wherein the plurality of types of segments include:
a delivery unit media segment format that indicates that corresponding segments include media data;
a random access media segment format that indicates that corresponding segments provide points at which retrieving data from the representation can begin;
a no overlap segment format that indicates that corresponding segments do not overlap starting and ending times of other segments in the representation and other segments in other representations in an adaptation set including the representation; and
a switching media segment format that indicates that corresponding segments provide points at which retrieval can switch to the representation from the other representations in the adaptation set without reinitialization;
means for determining, from the manifest file, a segment of the representation conforming to the type that provides the point at which to begin retrieving data from the representation; and
means for retrieving the determined segment from the representation.
14. The client device of claim 13 , wherein the delivery unit media segment format indicates that corresponding segments:
contain a value of “dums” in a segment type box of the segment;
include one or more self-contained movie fragments, wherein each self-contained movie fragment comprises a movie fragment (“moof”) box and a media data (“mdat”) box that contains media samples that do not use external data references referenced by a track in the movie fragment box;
each of the moof boxes contains at least one track fragment;
each of the moof boxes does not use external references;
a “default-base-is-moof” flag of the media segment is set to true; and
a “base-data-offset-present” flag of the media segment is set to false.
15. The client device of claim 13 , wherein the random access media segment format indicates that corresponding segments:
comprise random access segments, an ordinal first access unit in each movie fragment of the segments corresponds to an I SAU of a stream access point (SAP) of type 1, 2, or 3; and
include all necessary information to access media data in a bitstream following the segments.
16. The client device of claim 15 , wherein the random access media segment format indicates that corresponding segments comprise at least one of an instantaneous decoder refresh (IDR) picture, a broken link access (BLA) picture, or a clean random access (CRA) picture.
17. The client device of claim 13 , wherein the random access media segment format indicates that corresponding segments include one or more segment index (“sidx”) boxes, and wherein an ordinal first sidx box precedes all moof boxes of the media segment and describes the entire media segment.
18. The client device of claim 13 , wherein the switching media segment format indicates that an ordinal first sample in an ordinal first movie fragment of the corresponding segments corresponds to an I SAU of a stream access point (SAP) of type 1 or 2.
19. A computer-readable storage medium having stored thereon instructions that, when executed, cause a processor to:
determine, from a manifest file, a plurality of types of segments included in a representation of media content, one or more functions provided by each of the types of segments, and positions of segments conforming to each of the types of segments in the representation, wherein at least one of the types of segments provides a point at which to begin retrieving data from the representation, and wherein the plurality of types of segments include:
a delivery unit media segment format that indicates that corresponding segments include media data;
a random access media segment format that indicates that corresponding segments provide points at which retrieving data from the representation can begin;
a no overlap segment format that indicates that corresponding segments do not overlap starting and ending times of other segments in the representation and other segments in other representations in an adaptation set including the representation; and
a switching media segment format that indicates that corresponding segments provide points at which retrieval can switch to the representation from the other representations in the adaptation set without reinitialization;
determine, from the manifest file, a segment of the representation conforming to the type that provides the point at which to begin retrieving data from the representation; and
retrieve the determined segment from the representation.
20. The computer-readable storage medium of claim 19 , wherein the delivery unit media segment format indicates that corresponding segments:
contain a value of “dums” in a segment type box of the segment;
include one or more self-contained movie fragments, wherein each self-contained movie fragment comprises a movie fragment (“moof”) box and a media data (“mdat”) box that contains media samples that do not use external data references referenced by a track in the movie fragment box;
each of the moof boxes contains at least one track fragment;
each of the moof boxes does not use external references;
a “default-base-is-moof” flag of the media segment is set to true; and
a “base-data-offset-present” flag of the media segment is set to false.
21. The computer-readable storage medium of claim 19 , wherein the random access media segment format indicates that corresponding segments:
comprise random access segments, an ordinal first access unit in each movie fragment of the segments corresponds to an I SAU of a stream access point (SAP) of type 1, 2, or 3; and
include all necessary information to access media data in a bitstream following the segments.
22. The computer-readable storage medium of claim 21 , wherein the random access media segment format indicates that corresponding segments comprise at least one of an instantaneous decoder refresh (IDR) picture, a broken link access (BLA) picture, or a clean random access (CRA) picture.
23. The computer-readable storage medium of claim 19 , wherein the random access media segment format indicates that corresponding segments include one or more segment index (“sidx”) boxes, and wherein an ordinal first sidx box precedes all moof boxes of the media segment and describes the entire media segment.
24. The computer-readable storage medium of claim 19 , wherein the switching media segment format indicates that an ordinal first sample in an ordinal first movie fragment of the corresponding segments corresponds to an I SAU of a stream access point (SAP) of type 1 or 2.
25. A method of signaling media information, the method comprising:
constructing a manifest file indicating a plurality of types of segments included in a representation of media content, one or more functions provided by each of the types of segments, positions of segments conforming to each of the types of segments in the representation, wherein at least one of the types of segments provides a point at which to begin retrieving data from the representation, and a segment of the representation conforming to the type that provides the point at which to begin retrieving data from the representation, and wherein the plurality of types of segments include:
a delivery unit media segment format that indicates that corresponding segments include media data;
a random access media segment format that indicates that corresponding segments provide points at which retrieving data from the representation can begin;
a no overlap segment format that indicates that corresponding segments do not overlap starting and ending times of other segments in the representation and other segments in other representations in an adaptation set including the representation; and
a switching media segment format that indicates that corresponding segments provide points at which retrieval can switch to the representation from the other representations in the adaptation set without reinitialization;
sending the manifest file to a client device; and
in response to a request from the client device for the segment conforming to the type that provides the point at which to begin retrieving data from the representation, sending the segment that provides the point at which to begin retrieving data from the representation to the client device.
26. The method of claim 25 , wherein the delivery unit media segment format indicates that corresponding segments:
contain a value of “dums” in a segment type box of the segment;
include one or more self-contained movie fragments, wherein each self-contained movie fragment comprises a movie fragment (“moof”) box and a media data (“mdat”) box that contains media samples that do not use external data references referenced by a track in the movie fragment box;
each of the moof boxes contains at least one track fragment;
each of the moof boxes does not use external references;
a “default-base-is-moof” flag of the media segment is set to true; and
a “base-data-offset-present” flag of the media segment is set to false.
27. The method of claim 25 , wherein the random access media segment format indicates that corresponding segments:
comprise random access segments, an ordinal first access unit in each movie fragment of the segments corresponds to an I SAU of a stream access point (SAP) of type 1, 2, or 3; and
include all necessary information to access media data in a bitstream following the segments.
28. The method of claim 25 , wherein the random access media segment format indicates that corresponding segments include one or more segment index (“sidx”) boxes, and wherein an ordinal first sidx box precedes all moof boxes of the media segment and describes the entire media segment.
29. The method of claim 25 , wherein the switching media segment format indicates that an ordinal first sample in an ordinal first movie fragment of the corresponding segments corresponds to an I SAU of a stream access point (SAP) of type 1 or 2.
30. A server device for signaling media information, the server device comprising one or more processors configured to:
construct a manifest file indicating a plurality of types of segments included in a representation of media content, one or more functions provided by each of the types of segments, positions of segments conforming to each of the types of segments in the representation, wherein at least one of the types of segments provides a point at which to begin retrieving data from the representation, and a segment of the representation conforming to the type that provides the point at which to begin retrieving data from the representation, and wherein the plurality of types of segments include:
a delivery unit media segment format that indicates that corresponding segments include media data;
a random access media segment format that indicates that corresponding segments provide points at which retrieving data from the representation can begin;
a no overlap segment format that indicates that corresponding segments do not overlap starting and ending times of other segments in the representation and other segments in other representations in an adaptation set including the representation; and
a switching media segment format that indicates that corresponding segments provide points at which retrieval can switch to the representation from the other representations in the adaptation set without reinitialization;
send the manifest file to a client device; and
in response to a request from the client device for the segment conforming to the type that provides the point at which to begin retrieving data from the representation, send the segment that provides the point at which to begin retrieving data from the representation to the client device.
31. The server device of claim 30 , wherein the delivery unit media segment format indicates that corresponding segments:
contain a value of “dums” in a segment type box of the segment;
include one or more self-contained movie fragments, wherein each self-contained movie fragment comprises a movie fragment (“moof”) box and a media data (“mdat”) box that contains media samples that do not use external data references referenced by a track in the movie fragment box;
each of the moof boxes contains at least one track fragment;
each of the moof boxes does not use external references;
a “default-base-is-moof” flag of the media segment is set to true; and
a “base-data-offset-present” flag of the media segment is set to false.
32. The server device of claim 30 , wherein the random access media segment format indicates that corresponding segments:
comprise random access segments, an ordinal first access unit in each movie fragment of the segments corresponds to an I SAU of a stream access point (SAP) of type 1, 2, or 3; and
include all necessary information to access media data in a bitstream following the segments.
33. The server device of claim 30 , wherein the random access media segment format indicates that corresponding segments include one or more segment index (“sidx”) boxes, and wherein an ordinal first sidx box precedes all moof boxes of the media segment and describes the entire media segment.
34. The server device of claim 30 , wherein the switching media segment format indicates that an ordinal first sample in an ordinal first movie fragment of the corresponding segments corresponds to an I SAU of a stream access point (SAP) of type 1 or 2.
35. A server device for signaling media information, the server device comprising:
means for constructing a manifest file indicating a plurality of types of segments included in a representation of media content, one or more functions provided by each of the types of segments, positions of segments conforming to each of the types of segments in the representation, wherein at least one of the types of segments provides a point at which to begin retrieving data from the representation, and a segment of the representation conforming to the type that provides the point at which to begin retrieving data from the representation, and wherein the plurality of types of segments include:
a delivery unit media segment format that indicates that corresponding segments include media data;
a random access media segment format that indicates that corresponding segments provide points at which retrieving data from the representation can begin;
a no overlap segment format that indicates that corresponding segments do not overlap starting and ending times of other segments in the representation and other segments in other representations in an adaptation set including the representation; and
a switching media segment format that indicates that corresponding segments provide points at which retrieval can switch to the representation from the other representations in the adaptation set without reinitialization;
means for sending the manifest file to a client device; and
means for sending the segment that provides the point at which to begin retrieving data from the representation to the client device in response to a request from the client device for the segment conforming to the type that provides the point at which to begin retrieving data from the representation.
36. A non-transitory computer-readable storage medium having stored thereon instructions that, when executed, cause a processor of a server device to:
construct a manifest file indicating a plurality of types of segments included in a representation of media content, one or more functions provided by each of the types of segments, positions of segments conforming to each of the types of segments in the representation, wherein at least one of the types of segments provides a point at which to begin retrieving data from the representation, and a segment of the representation conforming to the type that provides the point at which to begin retrieving data from the representation, and wherein the plurality of types of segments include:
a delivery unit media segment format that indicates that corresponding segments include media data;
a random access media segment format that indicates that corresponding segments provide points at which retrieving data from the representation can begin;
a no overlap segment format that indicates that corresponding segments do not overlap starting and ending times of other segments in the representation and other segments in other representations in an adaptation set including the representation; and
a switching media segment format that indicates that corresponding segments provide points at which retrieval can switch to the representation from the other representations in the adaptation set without reinitialization;
send the manifest file to a client device; and
send the segment that provides the point at which to begin retrieving data from the representation to the client device in response to a request from the client device for the segment conforming to the type that provides the point at which to begin retrieving data from the representation.
37. The non-transitory computer-readable storage medium of claim 36 , wherein the delivery unit media segment format indicates that corresponding segments:
contain a value of “dums” in a segment type box of the segment;
include one or more self-contained movie fragments, wherein each self-contained movie fragment comprises a movie fragment (“moof”) box and a media data (“mdat”) box that contains media samples that do not use external data references referenced by a track in the movie fragment box;
each of the moof boxes contains at least one track fragment;
each of the moof boxes does not use external references;
a “default-base-is-moof” flag of the media segment is set to true; and
a “base-data-offset-present” flag of the media segment is set to false.
38. The non-transitory computer-readable storage medium of claim 36 , wherein the random access media segment format indicates that corresponding segments:
comprise random access segments, an ordinal first access unit in each movie fragment of the segments corresponds to an I SAU of a stream access point (SAP) of type 1, 2, or 3; and
include all necessary information to access media data in a bitstream following the segments.
39. The non-transitory computer-readable storage medium of claim 36 , wherein the random access media segment format indicates that corresponding segments include one or more segment index (“sidx”) boxes, and wherein an ordinal first sidx box precedes all moof boxes of the media segment and describes the entire media segment.
40. The non-transitory computer-readable storage medium of claim 36 , wherein the switching media segment format indicates that an odrinal first sample in an ordinal first movie fragment of the corresponding segments corresponds to an I SAU of a stream access point (SAP) or type 1 or 2.