Transporting HEIF-formatted images over real-time transport protocol
An example device for retrieving media data, the device comprising: a memory configured to store scene data and image data; and one or more processors implemented in circuitry and configured to: receive data for a bitstream indicating that the bitstream includes one or more overlay images, the overlay images comprising either a single static image to be presented with scene data that remains constant while being presented frame-by-frame with frames of the scene data or an image sequence that repeats with a regular periodicity while being presented frame-by-frame with the frames of the scene data; receive the one or more overlay images from the bitstream; receive the scene data including one or more scene images; and present the scene images and the overlay images.
1 . A method of retrieving media data, the method comprising:
receiving, for a Real-time Transport Protocol (RTP) bitstream, session description protocol (SDP) data including an SDP image attribute indicating that the RTP bitstream includes one or more sets of one or more images to be overlayed over scene data of the RTP bitstream, the SDP image attribute including:
a payload type syntax element that corresponds to a set of one or more images of the one or more sets of one or more images to be overlayed over the scene data and carried within the RTP bitstream; and
an item count syntax element corresponding to the payload type syntax element, the item count syntax element representing a number of images included in the set of one or more images to be overlayed over the scene data;
receiving the one or more images from the RTP bitstream;
receiving the scene data including one or more frames; and
presenting the one or more frames and the one or more images, wherein the one or more images are overlayed over the one or more frames of the scene data.
2 . The method of claim 1 , wherein the one or more images are received via a High Efficiency Video Coding (HEVC) payload format.
3 . The method of claim 2 , wherein the one or more frames of scene data are received via the HEVC payload format.
4 . The method of claim 1 , further comprising storing the one or more images in local memory, and wherein presenting the one or more frames and the one or more images comprises:
retrieving the one or more images from the local memory; and
presenting the retrieved one or more images.
5 . The method of claim 1 , wherein the one or more sets of one or more images include one or more image sequences each associated with a corresponding item count syntax element of the SDP image attribute, and wherein each of the respective item count syntax elements indicates the number of images included in the corresponding one or more image sequences.
6 . The method of claim 1 , wherein the one or more sets of one or more images include a collection of one or more still images, and wherein the item count syntax element indicates the number of images included in the collection of one or more still images.
7 . The method of claim 1 , further comprising, when the one or more images include two or more images, receiving data representing a display order of the two or more images.
8 . The method of claim 1 , further comprising receiving image metadata for each of the one or more images from the RTP bitstream, the metadata indicating one or more of image dimensions associated with the one or more images or an image role associated with the one or more images.
9 . The method of claim 8 , wherein the metadata includes one or more of an order element indicating an order of presentation of the one or more images or a duration element indicating an amount of time a corresponding one of the one or more images is to be displayed.
10 . The method of claim 1 , wherein the one or more sets of one or more images includes a collection of still images and an image sequence, and wherein the SDP image attribute includes a first payload type syntax element having a first value corresponding to the collection of still images and a second payload type syntax element having a second value, different than the first value, corresponding to the image sequence.
11 . A device for retrieving media data, the device comprising:
a memory configured to store scene data and image data; and
one or more processors implemented in circuitry and configured to:
receive, for a Real-time Transport Protocol (RTP) bitstream, session description protocol (SDP) data including an SDP image attribute indicating that the RTP bitstream includes one or more sets of one or more images to be overlayed over scene data of the RTP bitstream, the SDP image attribute including:
a payload type syntax element that corresponds to a set of one or more images of the one or more sets of one or more images to be overlayed over the scene data and carried within the RTP bitstream; and
an item count syntax element corresponding to the payload type syntax element, the item count syntax element representing a number of images included in the set of one or more images to be overlayed over the scene data;
receive the one or more images from the RTP bitstream;
receive the scene data including one or more frames; and
present the one or more frames and the one or more images, wherein the one or more images are overlayed over the one or more frames of the scene data.
12 . The device of claim 11 , wherein the one or more processors are further configured to receive the one or more images via a High Efficiency Video Coding (HEVC) payload format.
13 . The device of claim 12 , wherein the one or more processors are further configured to receive the one or more frames of scene data via the HEVC payload format.
14 . The device of claim 11 , wherein the one or more processors are further configured to:
store the one or more images in local memory;
retrieve the one or more images from the local memory; and
present the retrieved one or more images.
15 . The device of claim 11 , wherein the one or more sets of one or more images include one or more image sequences each associated with a corresponding item count syntax element of the SDP image attribute, and wherein each of the respective item count syntax elements indicates the number of images included in the corresponding one or more image sequences.
16 . The device of claim 11 , wherein the one or more sets of one or more images include a collection of one or more still images, and wherein the item count syntax element indicates the number of images included in the collection of one or more still images.
17 . The device of claim 11 , wherein the one or more processors are further configured to receive image metadata for each of the one or more images from the RTP bitstream, the metadata indicating one or more of image dimensions associated with the one or more images or an image role associated with the one or more images.
18 . The device of claim 11 , wherein the one or more sets of one or more images includes a collection of still images and an image sequence, and wherein the SDP image attribute includes a first payload type syntax element having a first value corresponding to the collection of still images and a second payload type syntax element having a second value, different than the first value, corresponding to the image sequence.
19 . A device for retrieving media data, the device comprising:
a memory configured to store scene data and image data; and
one or more processors implemented in circuitry and configured to:
receive, for a Real-time Transport Protocol (RTP) bitstream, session description protocol (SDP) data including an SDP image attribute indicating that the RTP bitstream includes one or more sets of one or more images, the SDP image attribute including:
a payload type syntax element that corresponds to a set of one or more images of the one or more sets of one or more images carried within the RTP bitstream;
receive the one or more images from the RTP bitstream; and
present the one or more images.
20 . The device of claim 19 , wherein the one or more processors are further configured to receive the one or more images via a High Efficiency Video Coding (HEVC) payload format.
21 . The device of claim 19 , wherein the attribute includes an item count syntax element, the item count syntax element indicating a number of images included in the one or more images.
22 . The device of claim 21 , wherein the one or more sets of one or more images include one or more image sequences each associated with a corresponding item count syntax element of the SDP image attribute, and wherein each of the respective item count syntax elements indicates the number of images included in the corresponding one or more image sequences.
23 . The device of claim 21 , wherein the one or more sets of one or more images include a collection of one or more still images, and wherein the item count syntax element indicates the number of images included in the collection of one or more still images.