Occlusion of virtual objects in augmented reality by physical objects
Systems and methods for occluding virtual objects in an augmented reality environment are described. A method includes generating a distance map associated with a physical object, determining a location of a virtual object in an artificial-reality environment, determining, based at least in part upon the distance map and the location of the virtual object, a first portion of the physical object that is closer than a first portion of the virtual object from the viewpoint of the user, determining a second portion of the virtual object that is not obscured by the physical object from the viewpoint of the user, and rendering a display image. The display image comprises a first set of pixels corresponding to the first portion of the physical object in a first state, and the display image comprises a second set of pixels corresponding to the second portion of the virtual object in a second state.
1 . A method comprising, by one or more computing devices:
generating a distance map associated with a physical object, wherein the physical object is present in an environment of a user of a head-mounted display, and the distance map relates a distance from the physical object to a viewpoint of the user;
determining a location of a virtual object, where the virtual object is to be presented to the user using the head-mounted display, in an artificial-reality environment;
determining, based at least in part upon the distance map and the location of the virtual object, a first portion of the physical object that is closer than a first portion of the virtual object from the viewpoint of the user;
determining, based at least in part upon the distance map and the location of the virtual object, a second portion of the virtual object that is not obscured by the physical object from the viewpoint of the user; and
rendering, from the viewpoint of the user, a display image, wherein the display image comprises a first set of pixels corresponding to the first portion of the physical object in a first state, and the display image comprises a second set of pixels corresponding to the second portion of the virtual object in a second state that is distinct from the first state.
2 . The method of claim 1 , wherein the first portion of the physical object is determined by tracing a ray from the viewpoint of the user to a point on the physical object, and identifying a surface that the ray intersects first.
3 . The method of claim 1 , wherein the first portion of the physical object is determined by representing the physical object as a planar object, and identifying a portion of the planar object as being in front of a surface representing the virtual object.
4 . The method of claim 1 , wherein the second portion of the virtual object is determined based on determining, based on the distance map and the location of the virtual object, that a second portion of the physical object is farther than the second portion of the virtual object.
5 . The method of claim 1 , wherein the first state is an unilluminated state.
6 . The method of claim 1 , wherein the first state is associated with a first opacity.
7 . The method of claim 6 , wherein the second state is associated with a second opacity that is distinct from the first opacity.
8 . The method of claim 1 , further comprising:
accessing an image comprising at least a portion of the physical object, wherein the first state comprises color values, associated with the physical object, based on the accessed image.
9 . The method of claim 1 , wherein the second state comprises color values based on the virtual object.
10 . The method of claim 1 , wherein the viewpoint of the user relates to a viewpoint of an eye of the user.
11 . The method of claim 1 , wherein the distance map is based on data captured using a depth sensor.
12 . A non-transitory, computer-readable storage medium including instructions that, when executed by one or more computing devices, cause the one or more computing devices to:
generate a distance map associated with a physical object, wherein the physical object is present in an environment of a user of a head-mounted display, and the distance map relates a distance from the physical object to a viewpoint of the user;
determine a location of a virtual object, where the virtual object is to be presented to the user using the head-mounted display, in an artificial-reality environment;
determine, based at least in part upon the distance map and the location of the virtual object, a first portion of the physical object that is closer than a first portion of the virtual object from the viewpoint of the user;
determine, based at least in part upon the distance map and the location of the virtual object, a second portion of the virtual object that is not obscured by the physical object from the viewpoint of the user; and
render, from the viewpoint of the user, a display image, wherein the display image comprises a first set of pixels corresponding to the first portion of the physical object in a first state, and the display image comprises a second set of pixels corresponding to the second portion of the virtual object in a second state that is distinct from the first state.
13 . The non-transitory, computer-readable storage medium of claim 12 , wherein the first state is an unilluminated state.
14 . The non-transitory, computer-readable storage medium of claim 12 , further including instructions that, when executed, cause the one or more computing devices to:
access an image comprising at least a portion of the physical object, wherein the first state comprises color values, associated with the physical object, based on the accessed image.
15 . The non-transitory, computer-readable storage medium of claim 12 , wherein the first state is associated with a first opacity and the second state is associated with a second opacity that is distinct from the first opacity.
16 . The non-transitory, computer-readable storage medium of claim 12 , wherein the viewpoint of the user relates to a left eye or a right eye of the user.
17 . A system including:
one or more processors; and
one or more computer-readable non-transitory storage media, operably coupled to at least one of the one or more processors, including instructions that, when executed by the at least one of the one or more processors, cause the system to:
generate a distance map associated with a physical object, wherein the physical object is present in an environment of a user of a head-mounted display, and the distance map relates a distance from the physical object to a viewpoint of the user;
determine a location of a virtual object, where the virtual object is to be presented to the user using the head-mounted display, in an artificial-reality environment;
determine, based at least in part upon the distance map and the location of the virtual object, a first portion of the physical object that is closer than a first portion of the virtual object from the viewpoint of the user;
determine, based at least in part upon the distance map and the location of the virtual object, a second portion of the virtual object that is not obscured by the physical object from the viewpoint of the user; and
render, from the viewpoint of the user, a display image, wherein the display image comprises a first set of pixels corresponding to the first portion of the physical object in a first state, and the display image comprises a second set of pixels corresponding to the second portion of the virtual object in a second state that is distinct from the first state.
18 . The system of claim 17 , wherein the first state is an unilluminated state.
19 . The system of claim 17 , further configured to:
access an image comprising at least a portion of the physical object, wherein the first state comprises color values, associated with the physical object, based on the accessed image.
20 . The system of claim 17 , wherein the viewpoint of the user relates to a left eye or a right eye of the user.