Granted Patent B2

US 10,564,731 · App. 15/803,517 · Granted Feb 18, 2020

Inventors: Matthew Bell (Palo Alto, CA); Tipatat Chennavasin (Menlo Park, CA); Charles H. Clanton (Burlingame, CA); Michael Hulme (Morgan Hill, CA); Eyal Ophir (Stanford, CA); Matthew Vieta (Mountain View, CA)

Assignee: Facebook, Inc.

G06F3/017G01B11/14G06F3/011G06F3/0304G06F3/04812G06F3/04883G06K9/00335A63F2300/1087

Abstract

Claims (83)

1. A method comprising:

receiving information corresponding to a physical space from one or more cameras;

mapping a position of a user in the physical space to a position in a virtual space based at least in part on the received information;

presenting, via an output device, a representation of the user in the position within the virtual space;

presenting, via the output device, a virtual object within the virtual space in a first state;

detecting, via the one or more cameras, an interaction with the virtual object based on a movement of a first part of the user from a first position in the physical space to a second position in the physical space;

determining a second state for presenting the virtual object based at least in part on the interaction and the first state;

presenting, via the output device, the representation of the user in the second position;

presenting, via the output device, the virtual object within the virtual space in the second state;

identifying a first zone within the physical space;

mapping the first zone to a portion of the virtual space; and

in response to detection of at least a portion of the user approaching an inter-zone boundary between the first zone and a second zone from a first direction, adjusting a display characteristic of the virtual object to indicate that a change to portions of the user that are capable of interacting with the virtual object will occur if the user continues to move in the first direction.

2. The method of claim 1 , wherein the information corresponding to the physical space includes a display distance indicating a distance of the user to the output device, the method comprising determining the second state for presenting the virtual object based at least in part on the interaction, the first state, and the display distance.

3. The method of claim 1 , wherein the information corresponding to the physical space includes a display distance indicating a distance of the user to the output device, the method comprising:

determining that the display distance corresponds to a threshold distance; and

detecting the interaction in response to said determining.

4. The method of claim 1 , further comprising:

identifying a first zone within the physical space;

mapping the first zone to a portion of the virtual space; and

in response to detection of at least a portion of the user approaching an inter-zone boundary between the first zone and a second zone from a first direction, adjusting a display characteristic of the virtual object to indicate that a change to portions of the user that are capable of interacting with the virtual object will occur if the user continues to move in the first direction.

5. The method of claim 4 , wherein the representation of the user comprises a representation of a hand of the user.

6. The method of claim 1 , wherein the information corresponding to the physical space includes a display distance indicating a distance of the user to the output device, the method comprising generating the representation of the user at least in part by: (a) projecting the second position along two dimensions of the virtual space and (b) projecting the distance of the user along a third dimension of the virtual space.

7. The method of claim 6 further comprising:

determining that the distance along the third dimension is less than a distance of the virtual object along the third dimension; and

overlaying a portion of the virtual object with a portion of the representation of the user.

8. The method of claim 6 further comprising:

determining that the distance along the third dimension is greater than a distance of the virtual object along the third dimension; and

overlaying a portion of the representation of the user with a portion of the virtual object.

9. A non-transitory computer readable storage having executable instructions stored thereon, the executable instructions configured to cause a computing system having one or more hardware processors to perform operations comprising:

receiving information corresponding to a physical space from one or more cameras;

mapping a position of a user in the physical space to a position in a virtual space based at least in part on the received information;

presenting, via an output device, a representation of the user in the position within the virtual space;

presenting, via the output device, a virtual object within the virtual space in a first state;

detecting, via the one or more cameras, an interaction with the virtual object based on a movement of a first part of the user from a first position in the physical space to a second position in the physical space;

determining a second state for presenting the virtual object based at least in part on the interaction and the first state;

presenting, via the output device, the representation of the user in the second position; and

presenting, via the output device, the virtual object within the virtual space in the second state;

identifying a first zone within the physical space;

mapping the first zone to a portion of the virtual space; and

in response to detection of at least a portion of the user approaching an inter-zone boundary between the first zone and a second zone from a first direction, adjusting a display characteristic of the virtual object to indicate that a change to portions of the user that are capable of interacting with the virtual object will occur if the user continues to move in the first direction.

10. The non-transitory computer readable storage of claim 9 ,

wherein the information corresponding to the physical space includes a display distance indicating a distance of the user to the output device, and

wherein the computer system further performs operations comprising determining the second state for presenting the virtual object based at least in part on the interaction, the first state, and the display distance.

11. The non-transitory computer readable storage of claim 9 ,

wherein the information corresponding to the physical space includes a display distance indicating a distance of the user to the output device, and

wherein the computer system further performs operations comprising: determining that the display distance corresponds to a threshold distance; and

further comprising detecting the interaction in response to said determining.

12. The non-transitory computer readable storage of claim 9 , wherein the representation of the user comprises a representation of a hand of the user.

13. The non-transitory computer readable storage of claim 9 ,

wherein the information corresponding to the physical space includes a display distance indicating a distance of the user to the output device, and

wherein the computer system further performs operations comprising generating the representation of the user at least in part by: (a) projecting the second position along two dimensions of the virtual space and (b) projecting the distance of the user along a third dimension of the virtual space.

14. The non-transitory computer readable storage of claim 13 , wherein the computer system further performs operations comprising:

determining that the distance along the third dimension is greater than a distance of the virtual object along the third dimension; and

overlaying a portion of the representation of the user with a portion of the virtual object.

15. A computing system comprising:

a camera; and

one or more hardware processors configured to execute instructions to cause the computing system to:

receive information corresponding to a physical space from the camera;

map a position of a user in the physical space to a position in a virtual space based at least in part on the received information;

present, via an output device, a representation of the user in the position within the virtual space;

present, via the output device, a virtual object within the virtual space in a first state;

detect, via the camera, an interaction with the virtual object based on a movement of a first part of the user from a first position in the physical space to a second position in the physical space;

determine a second state for presenting the virtual object based at least in part on the interaction and the first state;

present, via the output device, the representation of the user in the second position; and

present, via the output device, the virtual object within the virtual space in the second state;

identify a first zone within the physical space;

map the first zone to a portion of the virtual space; and

in response to detection of the at least a first portion of the user approaching an inter-zone boundary between the first zone and a second zone from a first direction, adjust a display characteristic of the virtual object to indicate that a change to portions of the user that are capable of interacting with the virtual object will occur if the user continues to move in the first direction.

16. The computing system of claim 15 ,

wherein the information corresponding to the physical space includes a display distance indicating a distance of the user to the output device, and

wherein the instructions further cause the computer system to determine the second state for presenting the virtual object based at least in part on the interaction, the first state, and the display distance.

17. The computing system of claim 15 ,

wherein the information corresponding to the physical space includes a display distance indicating a distance of the user to the output device, and

wherein the instructions further cause the computer system to:

determine that the display distance corresponds to a threshold distance; and

in response to determining that the display distance corresponds to the threshold distance, detect the interaction.

18. The computing system of claim 15 , wherein the representation of the user comprises a representation of a hand of the user.

19. The computing system of claim 15 ,

wherein the information corresponding to the physical space includes a display distance indicating a distance of the user to the output device, and

wherein the instructions further cause the computer system to generate the representation of the user at least in part by: (a) projecting the second position along two dimensions of the virtual space and (b) projecting the distance of the user along a third dimension of the virtual space.

20. The computing system of claim 19 , wherein the instructions further cause the computer system to:

determine that the distance along the third dimension is less than a distance of the virtual object along the third dimension; and

overlay a portion of the virtual object with a portion of the representation of the user.

Assignments (8)

Continuity (5)