System and method of streaming 3-D wireframe animations
Optimal resilience to errors in packetized streaming 3-D wireframe animation is achieved by partitioning the stream into layers and applying unequal error correction coding to each layer independently to maintain the same overall bitrate. The unequal error protection scheme for each of the layers combined with error concealment at the receiver achieves graceful degradation of streamed animation at higher packet loss rates than approaches that do not account for subjective parameters such as visual smoothness.
1. A method comprising:
computing, via a processor, visual smoothness values for each node in a wireframe mesh, each node comprising an indexed set of vertices in the wireframe mesh;
partitioning the indexed set of vertices to produce a partitioned mesh having more nodes than the wireframe mesh;
computing a visual smoothness for each node in the partitioned mesh;
layering data describing the wireframe mesh into a plurality of layers by ordering nodes into a respective layer according to whether the nodes have a similar visual smoothness; and
allocating first nodes associated with visible parts of the wireframe mesh to a first layer of the plurality of layers and allocating second nodes associated with non-visible parts to a second layer of the plurality of layers, wherein the second layer is only transmitted to a decoder where a user at the decoder is able to rotate the wireframe mesh to view the nodes associated with the non-visible parts, wherein average visual smoothness values associated with each of the plurality of layers reflect a respective layer's importance in the animation sequence;
applying different parameters n, FR, Sp, and F BOP to a channel bitrate algorithm that applies to a block-of-packets structure for each layer of the plurality of layers, the channel bitrate algorithm comprising:
R =( N*FR*Sp )/ F BOP
wherein R is a bitrate, n is a number of lines, and Sp is a number of columns in the block-of-packets structure, FR is a sequence frame rate, and F BOP is a number of data frames in the block-of-packets structure for a respective layer; and
applying unequal error protection to each respective layer of the plurality of layers according to a result of the channel bitrate algorithm.
2. The method of claim 1 , wherein each layer of the plurality of layers comprises one of a node and a group of nodes within the wireframe mesh.
3. The method of claim 1 , wherein a most important portion of the plurality of layers comprises the average visual smoothness value with a highest value and is error coded to be most resilient to packet errors.
4. The method of claim 1 , wherein a number of nodes within a particular portion of the plurality of layers is associated with an output bit rate of the particular portion of the plurality of layers.
5. The method of claim 1 , further comprising producing a 3-D packetized streaming signal representative of a scene comprising animation associated with the wireframe mesh.
6. The method of claim 1 , wherein partitioning further comprises partitioning the wireframe mesh into arbitrary sized sub-meshes that will be allocated to a same layer.
7. A method comprising:
receiving streamed data from a source;
generating, from the streamed data, a representation of a wireframe mesh partitioned to produce a partitioned mesh having more nodes than the wireframe mesh, wherein the wireframe mesh is partitioned to yield partitions corresponding to different objects within a scene represented by the nodes of the partitioned mesh and a corresponding motion of the different objects in an animation sequence, the partitions being balanced based on a number of nodes in each partition, encoding parameters, and the corresponding motion associated with each partition, the streamed data based on an unequal error protection scheme, wherein:
the streamed data comprises a portion of a dataset at the source describing the wireframe mesh, the dataset comprising a plurality of layers associated with different average visual smoothness values that reflect a respective layer's importance in an animation sequence;
a layer of the plurality of layers in the dataset comprises a plurality of partitions that identifies data in the layer that is visible and invisible when an animation in the animation sequence is in a static mode;
the portion of the dataset comprises a portion of a plurality of layers and a portion of the plurality of partitions unassociated with the portion of the plurality of layers; and
the portion of the dataset is selected to be supportable by the current bandwidth and to provide largest ones of the average visual smoothness values; and
applying, via a processor, applying different parameters n, FR, Sp, and F BOP to a channel bitrate algorithm that applies to a block-of-packets structure for each layer of the plurality of layers, the channel bitrate algorithm comprising:
R =( N*FR*Sp )/ F BOP
wherein R is a bitrate, n is a number of lines, and Sp is a number of columns in the block-of-packets structure, FR is a sequence frame rate, and F BOP is a number of data frames in the block-of-packets structure for a respective layer; and
applying an unequal error concealment scheme to each respective layer in the pluarilty of layers according to a result of the channel bitrate algorithm.
8. The method of claim 7 , wherein the unequal error protection scheme comprises optimizing a distribution of a bit budget allocation amongst the plurality of layers.
9. The method of claim 7 , wherein the plurality of layers are partitioned according to visual importance.
10. The method of claim 7 , wherein interpolation-based concealment is applied to residual packets not recovered from error-concealment.