Method and apparatus of point-cloud streaming
Systems and methods for identifying three dimensional (3D) content to stream. A method includes receiving information indicating a view field of a client device and identifying an amount of bandwidth available to stream the 3D content to the client device. The method includes assigning, for one or more of the objects, the bandwidth by, for each of the objects: determining whether the each object is within the view field of the client device based on a position of the each object; assigning a greater amount of the bandwidth to objects at least partially within the view field. The method includes streaming the 3D content to the client device according to the assigned bandwidth.
1. A server, the server comprising:
a memory configured to store three-dimensional (3D) point cloud data for one or more objects within 3D content;
a communication interface configured to receive, from a client device, information indicating a view field of the client device in the 3D content; and
a processor configured to:
identify an amount of bandwidth available to stream the 3D content to the client device, and
identify a first object and a second object of the one or more objects,
when at least one vertex of the first object is within a range of the view field, determine that the first object is within the view field of the client device, as indicated by the information by comparing a position of the one or more objects within the 3D content to any portion of the one or more objects that are within the view field of the client device,
when no vertices of the second object are within the range of the view field, determine that the second object is out of the view field, and
assign a first amount of the bandwidth that is available to the first object and a second amount of the bandwidth that is available to the second object,
wherein the first amount of the bandwidth is greater than the second amount of the bandwidth, and
wherein the communication interface is further configured to stream the 3D content including the first object and the second object to the client device according to the assigned bandwidth.
2. The server of claim 1 , wherein the information indicating the view field is at least one of a projection matrix or a model view matrix.
3. The server of claim 1 , wherein:
the 3D point cloud data comprise the one or more objects, each of the one or more objects contain one or more hierarchical levels of nodes from a root node level to a maximum level, and
to create the one or more hierarchical levels of nodes, for the first object, of the one or more objects, the processor is configured to:
set the first object as the root node level, and
incrementally divide each level node into multiple sub-level nodes from the root node level.
4. The server of claim 3 , wherein the processor is further configured to:
assign, for the one or more hierarchical levels of nodes, the bandwidth to stream the 3D point cloud data, wherein to assign the bandwidth for each of a given level of nodes, the given level of nodes starting from the root node level, the processor is configured to:
determine whether any nodes, within the given level, is within the view field of the client device based on a position of each node of the given level within the 3D content;
assign a greater amount of the bandwidth to a first subset of nodes of the given level that are within the view field of the client device than a second subset of nodes of the given level that are not within the view field of the client device; and
for the second subset of nodes that is out of the view field of the client device, the processor is configured to:
determine whether the one or more nodes, of the second subset of nodes, are within a threshold distance from the view field of the client device,
assign a lesser amount of the bandwidth to the one or more nodes that are within the threshold distance from the view field of the client device than the first subset of nodes that are within the view field of the client device, and
increase the given level by one level until the maximum level when an unassigned bandwidth exists.
5. The server of claim 4 , wherein:
when a vertex of a first node is within the range of the view field, the processor is configured to determine that the first node is within the view field; and
when no vertex of the first node is within the range of the view field, the processor is configured to determine that the first node is out of the view field.
6. The server of claim 1 , wherein:
the second object includes multiple objects; and
the processor is further configured to:
determine whether at least one object of the second object, which are not within the view field of the client device, is within a threshold distance from the view field of the client device;
assign a third amount of the bandwidth to each of the at least one object that is within the threshold distance from the view field of the client device; and
assign the second amount of the bandwidth to any of the at least one object that is not within the threshold distance,
wherein the third amount of the bandwidth is greater than the second amount of the bandwidth, and
wherein the first amount of the bandwidth is greater than the third amount of the bandwidth.
7. A method, the method comprising:
storing three-dimensional (3D) point cloud data for one or more objects within 3D content;
receiving, from a client device, information indicating a view field of the client device in the 3D content;
identifying an amount of bandwidth available to stream the 3D content to the client device;
identifying a first object and a second object of the one or more objects;
when at least one vertex of the first object is within a range of the view field, determining that the first object is within the view field of the client device, as indicated by the information by comparing a position of the one or more objects within the 3D content to any portion of the one or more objects that are within the view field of the client device;
when no vertices of the second object are within the range of the view field, determining that the second object is out of the view field; and
assigning a first amount of the bandwidth that is available to the first object and a second amount of the bandwidth that is available to the second object, wherein the first amount of the bandwidth is greater than the second amount of the bandwidth; and
streaming the 3D content including the first object and the second object to the client device according to the assigned bandwidth.
8. The method of claim 7 , wherein the information indicating the view field is at least one of a projection matrix or a model view matrix.
9. The method of claim 7 , wherein:
the 3D point cloud data comprise the one or more objects, each of the one or more objects contain one or more hierarchical levels of nodes from a root node level to a maximum level, and
creating the hierarchical levels of nodes, for the first object of the one or more objects, the method comprises:
setting the first object as the root node level, and
incrementally dividing each level node into multiple sub-level nodes from the root node level.
10. The method of claim 9 , the method further comprising:
assigning, for the one or more hierarchical levels of nodes, the bandwidth to stream the 3D point cloud data, wherein assigning the bandwidth for each of a given level of nodes, the given level of nodes starting from the root node level, comprises:
determining whether any nodes, within the given level, is within the view field of the client device based on a position of each node of the given level within the 3D content;
assigning a greater amount of the bandwidth to a first subset of nodes of the given level that are within the view field of the client device than a second subset of nodes of the given level that are not within the view field of the client device; and
for the second subset of nodes that is out of the view field of the client device, the method comprises:
determining whether the one or more nodes, of the second subset of nodes, are within a threshold distance from the view field of the client device,
assigning a lesser amount of the bandwidth to the one or more nodes that are within the threshold distance from the view field of the client device than the first subset of nodes that are within the view field of the client device, and
increasing the given level by one level until the maximum level when an unassigned bandwidth exists.
11. The method of claim 10 , the method further comprising:
when a vertex of a first node is within the range of the view field, determining that the first node is within the view field; and
when no vertex of the first node is within the range of the view field, determining that the first node is out of the view field.
12. The method of claim 7 , wherein the second object includes multiple objects; and the method further comprises:
determining whether at least one object of the second object, which are not within the view field of the client device, is within a threshold distance from the view field of the client device;
assigning a third amount of the bandwidth to each of the at least one object that is within the threshold distance from the view field of the client device; and
assign the second amount of the bandwidth to any of the at least one object that is not within the threshold distance,
wherein the third amount of the bandwidth is greater than the second amount of the bandwidth, and
wherein the first amount of the bandwidth is greater than the third amount of the bandwidth.
13. A non-transitory computer-readable medium comprising program code, wherein the program code that, when executed by a processor, causes the processor to:
store three-dimensional (3D) point cloud data for one or more objects within 3D content;
receive, from a client device, information indicating a view field of the client device in the 3D content via a communication interface;
identify an amount of bandwidth available to stream the 3D content to the client device;
identify a first object and a second object of the one or more objects;
when at least one vertex of the first object is within a range of the view field, determine that the first object is within the view field of the client device, as indicated by the information by comparing a position of the one or more objects within the 3D content to any portion of the one or more objects that are within the view field of the client device;
when no vertices of the second object are within the range of the view field, determine that the second object is out of the view field, and
assign a first amount of the bandwidth that is available to the first object and a second amount of the bandwidth that is available to the second object, wherein the first amount of the bandwidth is greater than the second amount of the bandwidth; and
stream the 3D content including the first object and the second object to the client device according to the assigned bandwidth.
14. The non-transitory computer-readable medium of claim 13 , wherein the information indicating the view field is at least one of a projection matrix or a model view matrix.
15. The non-transitory computer-readable medium of claim 13 , wherein the 3D point cloud data comprise the one or more objects, each of the one or more objects contain one or more hierarchical levels of nodes from a root node level to a maximum level.
16. The non-transitory computer-readable medium of claim 15 , wherein the program code that, when executed by the processor, further causes the processor to:
assign, for the one or more hierarchical levels of nodes, the bandwidth to stream the 3D point cloud data, wherein the program code for assigning the bandwidth for each of a given level of nodes, the given level of nodes starting from the root node level, comprises program code that, when executed by the processor, causes the processor to:
determine whether any nodes, within the given level, is within the view field of the client device based on a position of each node of the given level within the 3D content;
assign a greater amount of the bandwidth to a first subset of nodes of the given level that are within the view field of the client device than a second subset of nodes of the given level that are not within the view field of the client device; and
wherein to assign the bandwidth for the second subset of nodes that is out of the view field of the client device comprises program code that, when executed by the processor, causes the processor to:
determine whether the one or more nodes, of the second subset of nodes, are is within a threshold distance from the view field of the client device,
assign a lesser amount of the bandwidth to the one or more nodes that are within the threshold distance from the view field of the client device than the first subset of nodes that are within the view field of the client device, and
increase the given level by one level until the maximum level when an unassigned bandwidth exists.
17. The non-transitory computer-readable medium of claim 13 , wherein:
the second object includes multiple objects; and
the program code that, when executed by the processor, further causes the processor to:
determine whether at least one object of the second object, which are not within the view field of the client device, is within a threshold distance from the view field of the client device;
assign a third amount of the bandwidth to each of the at least one object that is within the threshold distance from the view field of the client device; and
assign the second amount of the bandwidth to any of the at least one object that is not within the threshold distance,
wherein the third amount of the bandwidth is greater than the second amount of the bandwidth, and
wherein the first amount of the bandwidth is greater than the third amount of the bandwidth.