Feed-forward collision avoidance for artificial reality environments

1. An artificial reality system comprising:

one or more image capture devices configured to capture image data representative of a three-dimensional (3D) physical environment having one or more physical objects;

a head mounted display (HMD) configured to output artificial reality content associated with a 3D virtual environment;

a physical overlay engine; and

a rendering engine configured to render, based on a pose of the HMD, output image data comprising the artificial reality content without including image data representative of the physical objects in the 3D physical environment;

wherein in response to a user-initiated activation condition associated with a hand-held device, the physical overlay engine determines, from the image data, data representing the physical objects, and generates, from the data representing the physical objects, overlay image data representing the physical objects, and

wherein the rendering engine is configured to render the overlay image data representing the physical objects with the output image data comprising the artificial reality content in response to the activation condition, and to continue to render the overlay image data while the activation condition is present or for a predetermined or configurable time after the activation condition is initiated.

2. The artificial reality system of claim 1 , wherein the physical overlay engine is configured to generate the overlay image data to include contour data for the physical objects.

3. The artificial reality system of claim 1 , wherein the artificial reality content comprises fully immersive artificial reality content.

4. The artificial reality system of claim 1 , further comprising:

a hand-held device tracker configured to determine a current orientation of the hand-held device, wherein the activation condition comprises one of:

a detection of a button press of at least one button of the hand-held device, or

a determination that the current orientation of the hand-held device comprises an overlay activation orientation.

5. The artificial reality system of claim 1 , further comprising a pose tracker, wherein the pose tracker is configured to detect that the HMD has crossed a virtual boundary, and wherein the activation condition comprises detection of the HMD crossing the virtual boundary.

6. The artificial reality system of claim 1 , further comprising a pose tracker, wherein the pose tracker is configured to:

detect a distance of the HMD from one of the physical objects in the 3D physical environment; and

wherein the activation condition comprises a determination that the distance is less than a configurable or predetermined threshold.

7. The artificial reality system of claim 1 , further comprising:

a hand-held device tracker configured to determine a distance of the hand-held device from one of the physical objects in the 3D physical environment;

wherein the activation condition comprises a determination that the distance is less than a configurable or predetermined threshold.

8. The artificial reality system of claim 1 , wherein the physical overlay engine includes an edge detector configured to determine one or more edges of the one or more physical objects in the 3D physical environment, and wherein the data representing the physical objects is determined according to the one or more edges of the one or more physical objects.

9. A method comprising:

obtaining image data representative of a three-dimensional (3D) physical environment having one or more physical objects;

rendering, by a rendering engine of an artificial reality system and based on a pose of a head mounted display (HMD) representing an estimated position and orientation of the HMD within the 3D physical environment, output image data comprising artificial reality content without including image data representative of the one or more physical objects; and

in response to a user-initiated activation condition associated with a hand-held device,

determining, from the image data, data representing the one or more physical objects,

generating, from the data representing the physical objects, overlay image data representing the physical objects, and

rendering, by the rendering engine while the activation condition is present or for a predetermined or configurable time after the activation condition is initiated, the overlay image data representing the physical objects with the output image data comprising the artificial reality content.

10. The method of claim 9 , wherein generating the overlay image data comprises generating contour data for the one or more physical objects.

11. The method of claim 9 , wherein generating the overlay image data comprises generating texture data for the one or more physical objects.

12. The method of claim 9 , wherein the activation condition comprises one of:

determining that a current orientation of the hand-held device of the artificial reality system comprises an overlay activation orientation; or

detecting a button press on the hand-held device of the artificial reality system.

13. The method of claim 9 , wherein the activation condition comprises detection of the HMD crossing a virtual boundary.

14. The method of claim 9 , wherein the activation condition comprises detecting that the HMD is within a configurable or predetermined distance from one of the physical objects in the 3D physical environment.

15. The method of claim 9 , wherein the activation condition comprises detecting that the hand-held device is within a configurable or predetermined distance from one of the physical objects in the 3D physical environment.

16. The method of claim 9 , further comprising:

determining one or more edges of the one or more physical objects in the 3D physical environment, wherein the data representing the one or more physical objects is determined according to the one or more edges of the one or more physical objects.

17. A non-transitory, computer-readable medium comprising instructions that, when executed, cause one or more processors of an artificial reality system including a head mounted display (HMD) to:

obtain image data representative of a three-dimensional (3D) physical environment having one or more physical objects;

determine an HMD pose representing an estimated position and orientation of the HMD within the 3D physical environment;

render, based on the HMD pose, output image data comprising artificial reality content without including image data representative of the one or more physical objects; and

in response to a user-initiated activation condition associated with a hand-held device,

determine, from the image data, data representing the one or more physical objects, and

generate, from the data representing the one or more physical objects, overlay image data representing the physical objects, and

render the overlay image data representing the physical objects with the output image data comprising the artificial reality content while the activation condition is present or for a predetermined or configurable time after the activation condition is initiated.