Performing operations using a mirror in an artificial reality environment
This disclosure describes an artificial reality system that presents artificial reality content in the context of a physical environment that includes a mirror or other reflective surface. In one example, this disclosure describes a method that includes capturing capture data representative of a physical environment, wherein the physical environment includes a reflective surface and a plurality of objects, determining a pose of the HMD, determining a map of the physical environment, wherein the map includes position information about the reflective surface and position information about each of the plurality of physical objects in the physical environment, identifying a visible object from among the plurality of physical objects, and generating artificial reality content associated with the visible object.
1. An artificial reality system comprising:
an image capture system configured to capture image data representative of a first physical environment that includes a mirror and a plurality of objects, wherein the image capture system includes a head-mounted display (HMD) worn by a first user;
a mapping engine configured to determine, based on the image data, a map of the physical environment including position information about the HMD, the mirror, and each of the plurality of physical objects; and
an application engine configured to:
determine, based on the position of the HMD and the mirror, that a reflected image of the first user is visible,
identity, from the image data, an input gesture performed by the first user,
generate, responsive to the input gesture, artificial reality content that includes an image representing a second user,
enable a communication session between the first user and the second user,
identify, based on the image data, one or more file transfer gestures performed by the first user, wherein the one or more file transfer gestures include the first user touching the mirror at a location corresponding to a location that the second user touches a second mirror, and
responsive to identifying the one or more file transfer gestures, enable a transfer of a file.
2. The artificial reality system of claim 1 , further comprising:
a rendering engine configured to render, for display at the HMD, artificial reality content overlaid on an image of the mirror.
3. The artificial reality system of claim 1 , wherein the physical environment is a first physical environment, wherein the HMD worn by the first user is a first HMD, and wherein the application engine is further configured to:
receive image data captured by a second HMD worn by the second user in a second physical environment, wherein the image data captured by the second HMD includes images reflected by the second mirror in the second physical environment, and wherein the second physical environment is located a remote distance from the first physical environment.
4. The artificial reality system of claim 3 , wherein to generate artificial reality content that includes an image representing the second user, the application engine is further configured to:
generate artificial reality content using the image data captured by the second HMD.
5. The artificial reality system of claim 4 , wherein to generate artificial reality content using the image data captured by the second HMD, the application engine is further configured to:
present an image representing the second user when the second user is positioned so that a reflected image of the second user in the second mirror is visible to the second HMD; and
present an image excluding the second user when the second user is not positioned so that a reflected image of the second user in the second mirror is visible to the second HMD.
6. The artificial reality system of claim 4 , wherein the application engine is further configured to:
output at least some of the image data representative of the first physical environment; and
enable the second HMD worn by the second user in the second physical environment to present, to the second user, image data representative of the first physical environment.
7. The artificial reality system of claim 1 , wherein to enable communication between the first user and the second user, the application engine is further configured to:
enable two-way audio and video communications between the first user and the second user.
8. A method comprising:
capturing image data representative of a physical environment that includes a mirror, a plurality of objects, and a head-mounted display (HMD) worn by a first user;
determining, based on the image data, a map of the physical environment including position information about the HMD, the mirror, and each of the plurality of physical objects;
determining, based on the position of the HMD and the mirror, that a reflected image of the first user is visible;
identifying, from the image data, an input gesture performed by the first user;
generating, responsive to the input gesture, artificial reality content that includes an image representing a second user;
enabling communication between the first user and the second user;
identifying, based on the image data, one or more file transfer gestures performed by the first user, wherein the one or more file transfer gestures include the first user touching the mirror at a location corresponding to a location that the second user touches a second mirror; and
responsive to identifying the one or more file transfer gestures, enabling transfer of a file.
9. The method of claim 8 , further comprising:
rendering, for display at the HMD, artificial reality content overlaid on an image of the mirror.
10. The method of claim 8 , wherein the physical environment is a first physical environment, wherein the HMD worn by the first user is a first HMD, and wherein the method further comprises:
receiving image data captured by a second HMD worn by the second user in a second physical environment, wherein the image data captured by the second HMD includes images reflected by the second mirror in the second physical environment, and wherein the second physical environment is located a remote distance from the first physical environment.
11. The method of claim 10 , wherein generating artificial reality content that includes an image representing the second user includes:
generating artificial reality content using the image data captured by the second HMD.
12. The method of claim 11 , wherein generating artificial reality content using the image data captured by the second HMD includes:
presenting an image representing the second user when the second user is positioned so that a reflected image of the second user in the second mirror is visible to the second HMD; and
presenting an image excluding the second user when the second user is not positioned so that a reflected image of the second user in the second mirror is visible to the second HMD.
13. The method of claim 11 , further comprising:
outputting at least some of the image data representative of the first physical environment; and
enabling the second HMD worn by the second user in the second physical environment to present, to the second user, image data representative of the first physical environment.
14. The method of claim 8 , wherein enabling communication between the first user and the second user includes:
enabling two-way audio and video communications between the first user and the second user.
15. A non-transitory computer-readable medium comprising instructions for causing processing circuitry of an artificial reality system to perform operations comprising:
capturing image data representative of a physical environment that includes a mirror, a plurality of objects, and a head-mounted display (HMD) worn by a first user;
determining, based on the image data, a map of the physical environment including position information about the HMD, the mirror, and each of the plurality of physical objects;
determining, based on the position of the HMD and the mirror, that a reflected image of the first user is visible;
identifying, from the image data, an input gesture performed by the first user;
generating, responsive to the input gesture, artificial reality content that includes an image representing a second user;
enabling communication between the first user and the second user;
identifying, based on the image data, one or more file transfer gestures performed by the first user, wherein the one or more file transfer gestures include the first user touching the mirror at a location corresponding to a location that the second user touches a second mirror; and
responsive to identifying the one or more file transfer gestures, enabling transfer of a file.
16. The non-transitory computer-readable medium of claim 15 , further comprising instructions for causing the processing circuitry to perform operations comprising:
rendering, for display at the HMD, artificial reality content overlaid on an image of the mirror.