Generating and modifying an artificial reality environment using occlusion surfaces at predetermined distances

1. A method comprising, by a computing system:

generating a depth map of a real environment as seen from a viewpoint of a user, the depth map comprising pixels having corresponding depth values of one or more physical objects in the real environment;

generating, based on the depth map, a two-dimensional occlusion surface representing at least a visible portion of the one or more physical objects that are located within a predetermined depth range defined relative to the viewpoint;

posing the two-dimensional occlusion surface in a three-dimensional coordinate system such that the two-dimensional occlusion surface is located at a predetermined distance from the viewpoint;

determining a visibility of a first virtual object relative to the one or more physical objects by comparing a model of the virtual object with the two-dimensional occlusion surface;

identifying, based on the depth map, a first physical object of the one or more physical objects that is located at a depth that is less than the depth range defined relative to the viewpoint;

generating a pre-occluded two-dimensional surface representing a second virtual object occluded by the first physical object, wherein the pre-occluded two-dimensional surface is based on a visibility of the first physical object from the viewpoint of a user;

posing the pre-occluded two-dimensional surface in the three-dimensional coordinate system; and

generating an output image based on the determined visibility of the first virtual object and the posed pre-occluded two-dimensional surface representing the second virtual object occluded by the first physical object.

2. The method of claim 1 , further comprising:

generating, based on the based on the depth map, a second two-dimensional occlusion surface representing at least a visible portion of the one or more physical objects that are located within a second predetermined depth range relative to the viewpoint that is greater than the first depth range; and

posing the second two-dimensional occlusion surface in the three-dimensional coordinate system such that the second two-dimensional occlusion surface is located at a second predetermined distance from the viewpoint.

3. The method of claim 1 , wherein posing the two-dimensional occlusion surface comprises adjusting a position or orientation of the two-dimensional occlusion surface to account for a change in perspective of the plurality of physical objects, the change in perspective resulting from one or more movements of the camera worn by the user subsequent to a time the two-dimensional occlusion surface is generated.

4. The method of claim 3 , wherein adjusting the position or orientation of the two-dimensional occlusion surface is performed by one or more components of the computing system that are physically connected to a headset worn by the user.

5. The method of claim 1 , wherein the depth map is calculated based on one or more images received from a camera worn by the user that is connected to a head-mounted display.

6. The method of claim 1 , wherein the two-dimensional occlusion surface comprises a plurality of texels, wherein each texel is assigned a value that specifies a level of transparency that is to be accorded to the texel.

7. The method of claim 1 , wherein the predetermined depth range defined relative to the viewpoint is between 1 meter and 2 meters from the viewpoint.

8. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

generate a depth map of a real environment as seen from a viewpoint of a user, the depth map comprising pixels having corresponding depth values of one or more physical objects in the real environment;

generate, based on the depth map, a two-dimensional occlusion surface representing at least a visible portion of the one or more physical objects that are located within a predetermined depth range defined relative to the viewpoint;

pose the two-dimensional occlusion surface in a three-dimensional coordinate system such that the two-dimensional occlusion surface is located at a predetermined distance from the viewpoint;

determine a visibility of a first virtual object relative to the one or more physical objects by comparing a model of the virtual object with the two-dimensional occlusion surface;

identify, based on the depth map, a first physical object of the one or more physical objects that is located at a depth that is less than the depth range defined relative to the viewpoint;

generate a pre-occluded two-dimensional surface representing a second virtual object occluded by the first physical object, wherein the pre-occluded two-dimensional surface is based on a visibility of the first physical object from the viewpoint of a user;

pose the pre-occluded two-dimensional surface in the three-dimensional coordinate system; and

generate an output image based on the determined visibility of the first virtual object and the posed pre-occluded two-dimensional surface representing the second virtual object occluded by the first physical object.

9. The media of claim 8 , wherein the software is further operable, when executed, to:

generate, based on the depth map, a second two-dimensional occlusion surface representing at least a visible portion of the one or more physical objects that are located within a second predetermined depth range relative to the viewpoint that is greater than the first depth range; and

pose the second two-dimensional occlusion surface in the three-dimensional coordinate system such that the second two-dimensional occlusion surface is located at a second predetermined distance from the viewpoint.

10. The media of claim 9 , wherein the two-dimensional occlusion surface comprises a plurality of texels, wherein each texel is assigned a value that specifies a level of transparency that is to be accorded to the texel.

11. The media of claim 8 , wherein posing the two-dimensional occlusion surface comprises adjusting a position or orientation of the two-dimensional occlusion surface to account for a change in perspective of the plurality of physical objects, the change in perspective resulting from one or more movements of the camera worn by the user subsequent to a time the two-dimensional occlusion surface is generated.

12. The media of claim 11 , wherein adjusting the position or orientation of the two-dimensional occlusion surface is performed by one or more components of the computing system that are physically connected to a headset worn by the user.

13. The media of claim 8 , wherein the depth map is calculated based on one or more images received from a camera worn by the user that is connected to a head-mounted display.

14. A system comprising:

one or more processors; and

one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to:

generate a depth map of a real environment as seen from a viewpoint of a user, the depth map comprising pixels having corresponding depth values of one or more physical objects in the real environment;

generate, based on the depth map, a two-dimensional occlusion surface representing at least a visible portion of the one or more physical objects that are located within a predetermined depth range defined relative to the viewpoint;

pose the two-dimensional occlusion surface in a three-dimensional coordinate system such that the two-dimensional occlusion surface is located at a predetermined distance from the viewpoint;

determine a visibility of a first virtual object relative to the one or more physical objects by comparing a model of the virtual object with the two-dimensional occlusion surface;

identify, based on the depth map, a first physical object of the one or more physical objects that is located at a depth that is less than the depth range defined relative to the viewpoint;

generate a pre-occluded two-dimensional surface representing a second virtual object occluded by the first physical object, wherein the pre-occluded two-dimensional surface is based on a visibility of the first physical object from the viewpoint of a user;

pose the pre-occluded two-dimensional surface in the three-dimensional coordinate system; and

generate an output image based on the determined visibility of the first virtual object and the posed pre-occluded two-dimensional surface representing the second virtual object occluded by the first physical object.

15. The system of claim 14 , wherein the processors are further operable when executing the instructions to:

generate, based on the based on the depth map, a second two-dimensional occlusion surface representing at least a visible portion of the one or more physical objects that are located within a second predetermined depth range relative to the viewpoint that is greater than the first depth range; and

pose the second two-dimensional occlusion surface in the three-dimensional coordinate system such that the second two-dimensional occlusion surface is located at a second predetermined distance from the viewpoint.

16. The system of claim 14 , wherein posing the two-dimensional occlusion surface comprises adjusting a position or orientation of the two-dimensional occlusion surface to account for a change in perspective of the plurality of physical objects, the change in perspective resulting from one or more movements of the camera worn by the user subsequent to a time the two-dimensional occlusion surface is generated.

17. The system of claim 16 , wherein adjusting the position or orientation of the two-dimensional occlusion surface is performed by one or more components of the computing system that are physically connected to a headset worn by the user.

18. The system of claim 14 , wherein the depth map is calculated based on one or more images received from a camera worn by the user that is connected to a head-mounted display.

19. The system of claim 14 , wherein the two-dimensional occlusion surface comprises a plurality of texels, wherein each texel is assigned a value that specifies a level of transparency that is to be accorded to the texel.