Systems and methods for a smooth temporally segmented encoding of dynamic unstructured spatial data
A system and associated methods perform smooth temporally segmented encoding of dynamic unstructured spatial data or dynamic three-dimensional (3D) content. The system receives the primitives that define visual changes to the dynamic 3D content across multiple frames. The system determines a first set of primitives that remain unchanged for at least N frames and a second set of primitives that remain unchanged for less than the N frames. The system generates a first data stream that encodes the first set of primitives with temporal values at a reduced frame rate, and generates a second data stream that encodes the second set of primitives without temporal values at the desired frame rate. The system streams the first data stream and the second data stream in response to a request for the dynamic 3D content.
1 . A method comprising:
requesting three-dimensional (3D) content that changes across a plurality of frames;
receiving a first encoding of the 3D content comprising a first set of primitives with each primitive of the first set of primitives being defined with a position in a 3D space and visual characteristic attributes;
receiving a second encoding of the 3D content comprising a second set of primitives with each primitive of the second set of primitives being defined with a position in the 3D space, visual characteristic attributes, and a temporal value; and
presenting the 3D content by rendering each primitive of the first set of primitives for a fixed first duration and by rendering each particular primitive of the second set of primitives for a second duration that is defined by the temporal value of the particular primitive.
2 . The method of claim 1 , wherein the fixed first duration corresponds to a single frame or all frames of the plurality of frames, and wherein the second duration is longer than a single frame and less than all frames of the plurality of frames.
3 . The method of claim 1 further comprising:
receiving a third encoding of the 3D content comprising different sets of primitives at a frame rate of the 3D content; and
wherein presenting the 3D content comprises:
rendering a single set of primitives from the different sets of primitives at a different frame of the plurality of frames; and
presenting a particular frame from the plurality of frames by combining a visualization generated from rendering the single set of primitives at the particular frame with a visualization generated from rendering the first set of primitives and a visualization generated from rendering a subset of the second set of primitives in which the second duration spans the particular frame.
4 . The method of claim 1 , wherein the temporal value of a specific primitive from the second set of primitives specifies a time or a frame from the plurality of frames at which the specific primitive is presented with the visual characteristic attributes unmodified and a width or a duration over which to blend in and blend out the visual characteristic attributes.
5 . The method of claim 1 ,
wherein rendering each primitive of the first set of primitives for the fixed first duration comprises presenting a visualization for each primitive of the first set of primitives for a single frame of the plurality of frames, and
wherein rendering each particular primitive of the second set of primitives for the second duration comprises presenting a visualization for each particular primitive of the second set of primitives over a set of the plurality of frames determined from the temporal value of the particular primitive.
6 . The method of claim 1 ,
wherein rendering each primitive of the first set of primitives for the fixed first duration comprises presenting a visualization for each primitive of the first set of primitives across all frames of the plurality of frames, and
wherein rendering each particular primitive of the second set of primitives for the second duration comprises presenting a visualization for each particular primitive of the second set of primitives across N number of the plurality of frames, wherein the N number of the plurality of frames is greater than 1, is less than all frames of the plurality of frames, and is determined from the temporal value of the particular primitive.
7 . The method of claim 1 , wherein rendering each particular primitive of the second set of primitives for the second duration comprises:
presenting a visualization for each particular primitive of the second set of primitives with visual characteristic attributes of the particular primitive unchanged at a particular frame of the plurality of frames based on a first value of the temporal value of the particular primitive and blending the visual characteristic attributes in and out for one or more frames before and after the particular frame based on a second value of the temporal value.
8 . The method of claim 1 , wherein the temporal value corresponds to a function that specifies a particular time or a particular frame of the plurality of frames at which to present visual characteristic attributes of a primitive as defined for the primitive and that specifies a duration over which to blend in and out the visual characteristic attributes of the primitive before and after the particular time or the particular frame.
9 . The method of claim 1 ,
wherein receiving the first encoding comprises receiving data for the first set of primitives at a first frame rate that is equal to a frame rate of the 3D content, and
wherein receiving the second encoding comprises receiving data for the second set of primitives at a second frame rate that is less than the frame rate of the 3D content.
10 . The method of claim 1 ,
wherein receiving the first encoding comprises receiving data for the first set of primitives at a first frame of the plurality of frames, and
wherein receiving the second encoding comprises receiving data for the second set of primitives at a frame rate that is less than a frame rate of the 3D content.
11 . The method of claim 1 further comprising:
decoding the first encoding with a first decompression scheme; and
decoding the second encoding with a different second decompression scheme.
12 . The method of claim 11 ,
wherein decoding the first encoding comprises performing a two-dimensional (2D) image decompression of the first encoding that is compressed with a 2D image compression technique, and
wherein decoding the second encoding comprises performing a 2D video decompression of the second encoding that is compressed with a video codec.
13 . The method of claim 1 further comprising:
decoding the first encoding using a 2D image decoding technique; and
decoding the second encoding using a 2D video codec at a frame rate that is less than a frame rate of the 3D content.
14 . The method of claim 1 ,
wherein the first encoding encodes the first set of primitives with a different set of one or more grids for each frame of the plurality of frames and with the different set of one or more grids for a particular frame of the plurality of frames encoding data of one or more primitives from the first set of primitives that are presented at the particular frame without change, and
wherein the second encoding encodes the second set of primitives as different sets of one or more grids across a subset of the plurality of frames.
15 . A device comprising:
one or more hardware processors configured to:
request three-dimensional (3D) content that changes across a plurality of frames;
receive a first encoding of the 3D content comprising a first set of primitives with each primitive of the first set of primitives being defined with a position in a 3D space and visual characteristic attributes;
receive a second encoding of the 3D content comprising a second set of primitives with each primitive of the second set of primitives being defined with a position in the 3D space, visual characteristic attributes, and a temporal value; and
present the 3D content by rendering each primitive of the first set of primitives for a fixed first duration and by rendering each particular primitive of the second set of primitives for a second duration that is defined by the temporal value of the particular primitive.
16 . The device of claim 15 , wherein the fixed first duration corresponds to a single frame or all frames of the plurality of frames, and wherein the second duration is longer than a single frame and less than all frames of the plurality of frames.
17 . The device of claim 15 , wherein the one or more hardware processors are further configured to:
receive a third encoding of the 3D content comprising different sets of primitives at a frame rate of the 3D content; and
wherein presenting the 3D content comprises:
rendering a single set of primitives from the different sets of primitives at a different frame of the plurality of frames; and
presenting a particular frame from the plurality of frames by combining a visualization generated from rendering the single set of primitives at the particular frame with a visualization generated from rendering the first set of primitives and a visualization generated from rendering a subset of the second set of primitives in which the second duration spans the particular frame.
18 . The device of claim 15 , wherein the temporal value of a specific primitive from the second set of primitives specifies a time or a frame from the plurality of frames at which the specific primitive is presented with the visual characteristic attributes unmodified and a width or a duration over which to blend in and blend out the visual characteristic attributes.
19 . The device of claim 15 ,
wherein rendering each primitive of the first set of primitives for the fixed first duration comprises presenting a visualization for each primitive of the first set of primitives for a single frame of the plurality of frames, and
wherein rendering each particular primitive of the second set of primitives for the second duration comprises presenting a visualization for each particular primitive of the second set of primitives over a set of the plurality of frames determined from the temporal value of the particular primitive.
20 . A non-transitory computer-readable medium storing instructions that, when executed by one or more processors of a device, cause the device to perform operations comprising:
requesting three-dimensional (3D) content that changes across a plurality of frames;
receiving a first encoding of the 3D content comprising a first set of primitives with each primitive of the first set of primitives being defined with a position in a 3D space and visual characteristic attributes;
receiving a second encoding of the 3D content comprising a second set of primitives with each primitive of the second set of primitives being defined with a position in the 3D space, visual characteristic attributes, and a temporal value; and
presenting the 3D content by rendering each primitive of the first set of primitives for a fixed first duration and by rendering each particular primitive of the second set of primitives for a second duration that is defined by the temporal value of the particular primitive.