Six degree of freedom tracking with scale recovery and obstacle avoidance

1. A method comprising:

presenting, on a display of a virtual reality or mixed reality system, a virtual scene to a user;

detecting an edge of a physical object within a physical environment from one or more images captured by the virtual reality or mixed reality system, the physical object not displayed as part of the virtual scene;

determining a six degree of freedom pose associated with the user;

determining the user is within a threshold distance from the physical object; and

displaying, in response to the user being within the threshold distance from the physical object, the edge of the physical object within the virtual scene being presented to the user.

2. The method as recited in claim 1 , wherein detecting the edge of the physical object includes:

detecting a first line segment based on a first color gradient within the one or more images;

detecting a second line segment based on a second color gradient within the one or more images;

merging the first line segment to the second line segment into the edge based at least in part on a similarly of the first color gradient to the second color gradient; and

locating the edge within a three-dimensional model of the physical environment.

3. The method as recited in claim 2 , wherein detecting the edge of the physical object further comprises adjusting location of the edge within the three-dimensional model.

4. The method as recited in claim 1 , wherein detecting the edge of the physical object includes:

detecting a first edgelet from the one or more images;

detecting a second edgelet from the one or more images;

detecting a continuous gradient between the first edgelet and the second edgelet;

joining the first edgelet with the second edgelet into a joined edgelet based at least in part on the continues gradient;

estimating a contour based at least in part on the joined edgelet; and

determining a surface based at least in part on the contour; and

wherein the edge is an edge of the surface.

5. The method as recited in claim 1 , wherein the one or more images are captured by a monocular camera.

6. The method as recited in claim 1 , further comprising:

determining a direction of movement associated with the user based at least in part on inertial measurement unit data collected by an inertial measurement unit of the virtual reality or mixed reality system;

determining a collision risk between the user and the physical object based at least in part on the direction of movement of the user, the six degree of freedom pose of the user, and a location of the edge; and

wherein displaying the edge of the physical object within the virtual scene is based at least in part on the collision risk being greater than a threshold.

7. The method as recited in claim 1 , wherein displaying the edge of the physical object within a virtual scene includes:

determining a field of view of the user based at least in part on the six degree of freedom pose of the user;

determining the edge is not within the field of view; and

displaying an indicator of the edge within a field of view of the user.

8. The method as recited in claim 7 , wherein the indicator is at least one of a ray projecting from the edge or the edge into a field of view of the user.

9. A system comprising:

an image capture device, for capturing image data associated with a physical environment surrounding the system;

at least one display device for presenting a virtual scene to a user of the system;

one or more processors; and

one or more non-transitory computer-readable media storing computer-executable instructions, which when executed by the one or more processors cause the one or more processors to perform operations comprising:

detecting, based at least in part on the image data, an edge of a physical object within the physical environment, the physical object excluded from the virtual scene;

determining a six degree of freedom pose associated with the user of the system;

determining the user is within a threshold distance from the physical object; and

displaying, in response to the user is within the threshold distance from the physical object, the edge of the physical object within the virtual scene being presented to the user.

10. The system as recited in claim 9 , wherein detecting the edge of the physical object includes:

detecting a first line segment based on a first color gradient within the image data;

detecting a second line segment based on a second color gradient within the image data;

merging the first line segment to the second line segment into the edge based at least in part on a similarly of the first color gradient to the second color gradient; and

locating the edge within a three-dimensional model of the physical environment.

11. The system as recited in claim 10 , wherein detecting the edge of the physical object further comprises adjusting location of the edge within the three-dimensional model.

12. The system as recited in claim 9 , wherein detecting the edge of the physical object includes:

detecting a first edgelet based at least in part on the image data;

detecting a second edgelet based at least in part on the image data;

detecting a continuous gradient between the first edgelet and the second edgelet;

joining the first edgelet with the second edgelet into a joined edgelet based at least in part on the continues gradient;

estimating a contour based at least in part on the joined edgelet; and

determining a surface based at least in part on the contour; and

wherein the edge is an edge of the surface.

13. The system as recited in claim 9 , further comprising:

an inertial measurement unit to capture orientation data associated with the system; and

wherein the operation further comprise:

determining a direction of movement associated with the user based at least in part on orientation data collected;

determining a collision risk between the user and the physical object based at least in part on the direction of movement of the user, the six degree of freedom pose of the user, and a location of the edge; and

wherein displaying the edge of the physical object within the virtual scene is based at least in part on the collision risk being greater than a threshold.

14. The method as recited in claim 9 , wherein displaying the edge of the physical object within the virtual scene includes:

determining a field of view of the user based at least in part on the six degree of freedom pose of the user;

determining the edge is not within the field of view; and

displaying an indicator of the edge within a field of view of the user.

15. The method as recited in claim 14 , wherein the indicator is at least one of a ray projecting from the edge or the edge into a field of view of the user.

16. One or more non-transitory computer-readable media storing computer-executable instructions, which when executed by one or more processors cause the one or more processors to perform operations comprising:

presenting, via a display device, a virtual scene;

detecting, based at least in part on image data received from an image device, an edge of a physical object within the physical environment, the physical object not presented as part of the virtual scene;

determining a six degree of freedom pose associated with the user;

determining the user is within a threshold distance from the physical object; and

causing, in response to the user is within the threshold distance from the physical object, the display device to present the edge of the physical object within the virtual scene.

17. The non-transitory computer-readable media as recited in claim 16 , wherein detecting the edge of the physical object includes:

detecting a first line segment based on a first color gradient within the image data;

detecting a second line segment based on a second color gradient within the image data;

merging the first line segment to the second line segment into the edge based at least in part on a similarly of the first color gradient to the second color gradient; and

locating the edge within a three-dimensional model of the physical environment.

18. The non-transitory computer-readable media as recited in claim 16 , wherein detecting the edge of the physical object includes:

detecting a first edgelet based at least in part on the image data;

detecting a second edgelet based at least in part on the image data;

detecting a continuous gradient between the first edgelet and the second edgelet;

joining the first edgelet with the second edgelet into a joined edgelet based at least in part on the continues gradient;

estimating a contour based at least in part on the joined edgelet; and

determining a surface based at least in part on the contour; and

wherein the edge is an edge of the surface.

19. The non-transitory computer-readable media as recited in claim 16 , wherein the operation further comprise:

an inertial measurement unit data associated with the user;

determining a direction of movement associated with the user based at least in part on orientation data collected;

determining a collision risk between the user and the physical object based at least in part on the direction of movement of the user, the six degree of freedom pose of the user, and a location of the edge; and

wherein displaying the edge of the physical object within the virtual scene is based at least in part on the collision risk being greater than a threshold.

20. The non-transitory computer-readable media as recited in claim 16 , wherein causing the display device to present the edge of the physical object within the virtual scene includes:

determining a field of view of the user based at least in part on the six degree of freedom pose of the user;

determining the edge is not within the field of view; and

displaying an indicator of the edge within a field of view of the user.