Systems and methods to simulate physical objects occluding virtual objects in an interactive space
A system and method to simulate physical objects occluding virtual objects within an interactive space may obtain output signals conveying three-dimensional positions of surfaces of a user object. The three-dimensional positions may be conveyed in a point cloud. Point cloud images may be obtained from the output signals. An individual point cloud image may depict a two-dimensional representation of the point cloud. A current point cloud image may be aggregated with one or more previous aggregate point cloud images to generate a current aggregate point cloud image. A mesh may be generated based on the current aggregate point cloud image. The mesh may include a set of vertices. The mesh may be generated by assigning individual vertices to individual points in the current aggregate point cloud image.
1. A system configured to simulate physical objects occluding virtual objects within an interactive space, the system comprising:
a distancing device configured to generate output signals conveying three-dimensional positions of surfaces of real-world objects, the real-world objects including a user object, wherein the three-dimensional positions are conveyed in a point cloud, the point cloud including a set of points that lie on the surfaces of the user object; and
one or more physical processors configured by machine-readable instructions to:
obtain, from the output signals, point cloud images, an individual point cloud image depicting a two-dimensional representation of the point cloud;
generate aggregate point cloud images over time, such that a current point cloud image is aggregated with one or more previous aggregate point cloud images to generate a current aggregate point cloud image; and
generate a mesh based on the current aggregate point cloud image, the mesh including a set of vertices, wherein generating the mesh comprises assigning individual vertices of the mesh to individual points represented in the current aggregate point cloud image,
wherein generating the current aggregate point cloud image comprises assigning weights to the current point cloud image and the one or more previous aggregate point cloud images.
2. The system of claim 1 , further comprising:
a client device configured to be worn on a head of a user;
one or more image-forming components incorporated into the client device, the one or more image-forming components being configured to generate light rays to form images of virtual content and provide the images to one or both eyes of the user; and
wherein the one or more physical processors are further configured by the machine-readable instructions to:
control the one or more image-forming components to generate an image forming a virtual object, the virtual object being perceived at a first position in the real world;
determine, from the output signals, a second position of the user object, the second position of the user object corresponding to a position of the mesh such that the mesh is also at the second position;
determine, based on the first position and the second position, a portion of overlap between the mesh and the virtual object; and
control the one or more image-forming components to modify the image of the virtual object based on the portion of overlap, the modification including removing one or more parts of the virtual object that correspond to the portion of overlap so as to cause the virtual object to be perceived as being occluded by the user object.
3. The system of claim 2 , wherein the one or more image-forming components are configured such that when the client device is installed on the head of the user, the images are presented to the user and superimposed over views of the real world to create an augmented reality environment.
4. The system of claim 3 , wherein the distancing device is incorporated into the client device.
5. The system of claim 1 , wherein the one or more physical processors are further configured by the machine-readable instructions to:
determine, based on the output signals, an estimated velocity of the user object; and
adjust the weights based on the estimated velocity.
6. The system of claim 5 , wherein determining the weights based on the estimated velocity comprises:
increasing the weight for the current point cloud image and decreasing the weight for the one or more previous aggregate point cloud images as the estimated velocity increases; and
decreasing the weight for the current point cloud image and increasing the weight for the one or more previous aggregate point cloud images as the estimated velocity decreases.
7. The system of claim 1 , wherein generating the mesh further comprises blurring edges of the mesh.
8. The system of claim 1 , wherein the point cloud images are generated by projecting the point cloud on a concave spherical surface.
9. The system of claim 1 , wherein the distancing device includes one or more of a camera, a three-dimensional scanner, a depth sensor, a rangefinder, a laser gauge, a time-of-flight sensor, a volumetric imaging sensor, an IR camera/emitter pair, a passive structured light stereo pair, or a passive unstructured light stereo pair.
10. A method to simulate physical objects occluding virtual objects within an interactive space, the method comprising:
obtaining output signals conveying three-dimensional positions of surfaces of real-world objects, the real-world objects including a user object, wherein the three-dimensional positions are conveyed in a point cloud, the point cloud including a set of points that lie on the surfaces of the user object; and
obtaining, from the output signals, point cloud images, an individual point cloud image depicting a two-dimensional representation of the point cloud;
generating aggregate point cloud images over time, such that a current point cloud image is aggregated with one or more previous aggregate point cloud images to generate a current aggregate point cloud image; and
generating a mesh based on the current aggregate point cloud image, the mesh including a set of vertices, wherein generating the mesh comprises assigning individual vertices of the mesh to individual points represented in the current aggregate point cloud image,
wherein generating the current aggregate point cloud image comprises assigning weights to the current point cloud image and the one or more previous aggregate point cloud images.
11. The method of claim 10 , further comprising:
generating light rays to form images of virtual content and providing the images to one or both eyes of the user, the images including an image forming a virtual object, the virtual object being perceived at a first position in the real world;
determining, from the output signals, a second position of the user object, the second position of the user object corresponding to a position of the mesh such that the mesh is also at the second position;
determining, based on the first position and the second position, a portion of overlap between the mesh and the virtual object; and
modifying the image of the virtual object based on the portion of overlap, the modification including removing one or more parts of the virtual object that correspond to the portion of overlap so as to cause the virtual object to be perceived as being occluded by the user object.
12. The method of claim 11 , wherein the images are presented to the user and superimposed over views of the real world to create an augmented reality environment.
13. The method of claim 10 , wherein the output signals are obtained from a distancing device.
14. The method of claim 10 , further comprising:
determining, based on the output signals, an estimated velocity of the user object; and
adjusting the weights based on the estimated velocity.
15. The method of claim 14 , wherein determining the weights based on the estimated velocity comprises:
increasing the weight for the current point cloud image and decreasing the weight for the one or more previous aggregate point cloud images as the estimated velocity increases; and
decreasing the weight for the current point Cloud image and increasing the weight for the one or more previous aggregate point cloud images as the estimated velocity decreases.
16. The method of claim 10 , wherein generating the mesh further comprises blurring edges of the mesh.
17. The method of claim 10 , wherein the point cloud images are generated by projecting the point cloud on a concave spherical surface.
18. The method of claim 13 , wherein the distancing device includes one or more of a camera, a three-dimensional scanner, a depth sensor, a rangefinder, a laser gauge, a time-of-flight sensor, a volumetric imaging sensor, an IR camera/emitter pair, a passive structured light stereo pair, or a passive unstructured light stereo pair.