Scene tracks for representing media assets
Improved media processing techniques includes media data formats with a “convenience” sample having both a snapshot and a delta description of a scene at the same media time. In other examples, improved media processing techniques include creating a temporary media sample having a merged snapshot or a merged delta in a context of a pre-render thread to improve performance of rendering performed in a context of a separate rendering thread.
1 . A non-transitory computer readable medium for representing a media asset comprising:
a sequence of entries for representing a model of objects in the asset, each entry describing the model over an interval of media time, the entries comprising:
model snapshots each providing a self-contained definition of the model over its respective interval, and
model deltas each providing a definition of the model over its respective interval with reference to a prior model defined in a prior entry,
wherein the sequence of entries includes at least one convenience entry having a convenience model delta of the model deltas and a corresponding model snapshot of the model snapshots, and the model delta and the corresponding model snapshot describe a common interval of time.
2 . The medium of claim 1 , wherein a first model snapshot of the model snapshots for a first interval identifies objects in a scene, and a first model delta of the model deltas for a second interval identifies changes to the objects in the scene, new objects added to the scene, and/or removal of objects in the scene.
3 . The medium of claim 1 , wherein a first model delta of the model deltas for a first interval is defined with reference to another prior entry of a second model delta of the model deltas for a second interval.
4 . The medium of claim 1 , wherein the asset is a three-dimensional (3D) scene, the objects are 3D objects, and the model is a 3D model.
5 . A method for seeking within a media stream of media samples, comprising:
seeking within the media stream from a current media time to a desired media time including identifying a first media sample corresponding to the desired media time;
when the first media sample contains a model delta at the desired media time, the model delta referring to a second media sample corresponding to a second media time:
creating a temporary snapshot at the desired media time by merging the second media sample with the model delta at the desired media time; and
rendering the temporary snapshot; and
when the first media sample contains a snapshot at the desired media time, rendering the snapshot from the first media sample.
6 . The method of claim 5 , wherein the second media sample includes a snapshot at the second media time, the creating of the temporary snapshot includes:
merging the snapshot at the second media time with the model delta at the desired media time.
7 . The method of claim 5 , wherein the second media sample includes a model delta at the second media time that refers to a third media sample corresponding to a third media time, the creating of the temporary snapshot includes:
merging the model delta at the second media time with the model delta at the desired media time.
8 . The method of claim 7 , wherein the creating the temporary snapshot occurs on a computing device within a context of a first computer thread, and the rendering of the temporary snapshots occurs on the computing device within a context of a second computer thread, separate from the first computer thread.
9 . The method of claim 5 , wherein the media stream includes a sequence of entries for representing a model of objects in a media asset, each entry describing the model over an interval of media time, and the second media sample is a convivence media sample including:
a model snapshot at the desired media time providing a self-contained definition of the model over its respective interval including the desired media time, and
a model deltas providing a definition of the model over the respective interval with reference to a model defined in a second media sample for a corresponding second media time.
10 . A method for seeking within a media stream of media samples, comprising:
rendering a current state of a scene by rendering a current media sample of the media stream corresponding to a current media time;
seeking within the media stream from the current media time to a desired media time;
identifying a sought media sample corresponding to the desired media time;
when the sought media sample contains a model delta, creating a merged model delta by merging model deltas in media samples, wherein the model deltas corresponds to media times between the current media time and the desired media time;
rendering the scene at the desired media time by applying the merged model delta to the current state.
11 . A non-transitory computer readable storage including instructions that, when executed by a processor, the processor causes:
seeking within a media stream from a current media time to a desired media time including identifying a first media sample corresponding to the desired media time;
when the first media sample contains a model delta at the desired media time that refers to a second media sample corresponding to a second media time, creating a temporary snapshot at the desired media time by merging the second media sample with the model delta at the desired media time, and rendering the temporary snapshot; and
when the first media sample contains a snapshot at the desired media time, rendering the snapshot from the first media sample.
12 . The computer readable storage of claim 11 , wherein the second media sample includes a snapshot at the second media time, and the creating of the temporary snapshot includes:
merging the snapshot at the second media time with the model delta at the desired media time.
13 . The computer readable storage of claim 11 , wherein the second media sample includes a model delta at the second media time that refers to a third media sample corresponding to a third media time, the creating of the temporary snapshot includes:
merging the model delta at the second media time with the model delta at the desired media time.
14 . The computer readable storage of claim 11 , wherein the creating the temporary snapshot occurs on a computing device within a context of a first computer thread, and the rendering of the temporary snapshots occurs on the computing device within a context of a second computer thread, separate from the first computer thread.
15 . The computer readable storage of claim 11 wherein the media stream includes a sequence of entries for representing a model of objects in a media asset, each entry describing the model over an interval of media time, and the second media sample is a convivence media sample including:
a model snapshot at the desired media time providing a self-contained definition of the model over its respective interval including the desired media time, and
a model deltas providing a definition of the model over the respective interval with reference to a model defined in a second media sample for a corresponding second media time.
16 . A rendering device, comprising:
a processor, and
a computer readable storage including instructions that, when executed by the processor, the device causes:
seeking within a media stream from a current media time to a desired media time including identifying a first media sample corresponding to the desired media time;
when the first media sample contains a model delta at the desired media time that refers to a second media sample corresponding to a second media time, creating a temporary snapshot at the desired media time by merging the second media sample with the model delta at the desired media time, and rendering the temporary snapshot; and
when the first media sample contains a snapshot at the desired media time, rendering the snapshot from the first media sample.
17 . The rendering device of claim 16 , wherein the second media sample includes a snapshot at the second media time, the creating of the temporary snapshot includes:
merging the snapshot at the second media time with the model delta at the desired media time.
18 . The rendering device of claim 16 , wherein the second media sample includes a model delta at the second media time that refers to a third media sample corresponding to a third media time, the creating of the temporary snapshot includes:
merging the model delta at the second media time with the model delta at the desired media time.
19 . The rendering device of claim 16 , wherein the creating the temporary snapshot occurs on a computing device within a context of a first computer thread, and the rendering of the temporary snapshots occurs on the computing device within a context of a second computer thread, separate from the first computer thread.
20 . The rendering device of claim 16 , wherein the media stream includes a sequence of entries for representing a model of objects in a media asset, each entry describing the model over an interval of media time, and the second media sample is a convivence media sample including:
a model snapshot at the desired media time providing a self-contained definition of the model over its respective interval including the desired media time, and
a model deltas providing a definition of the model over the respective interval with reference to a model defined in a second media sample for a corresponding second media time.