Methods and apparatuses for applying free space inputs for surface constrained controls
A method, system, apparatus, and/or device for sensing an input in an augmented reality construct. The method, system, apparatus, and/or device may include a mixed-reality device, comprises memory and one or more processors communicatively coupled to a sensor, a touch device, and a display device that is at least partially transparent and configured to display a virtual object. The store instructions executable by the one or more processors to generate virtual object data for displaying a virtual object by the display device; output the virtual object data to the display device; receive, from the sensor, interaction data corresponding to a free-space interaction by a hand of a user with the virtual object; generate touch input data based on the interaction data; and output the touch input data to the touch device.
1. A method, comprising:
receiving, by a first processing device of a head-worn device, a first set of data indicative of a free-space input by a hand of a user wearing the head-worn device, wherein the head-worn device is an augmented-reality device or a virtual-reality device; and
in response to the free-space input satisfying a free-space input standard, wherein the free-space input standard comprises a first standard for a motion of the hand in three-dimensional space:
generating a second set of data indicative of a virtual surface-constrained input, wherein:
the virtual surface-constrained input conforms with the virtual surface-constrained input standard; and
the virtual surface-constrained input standard comprises a second standard for a motion of the hand with a physical surface; and
sending, to a second processing device, the virtual surface-constrained input, wherein the second processing device is configured to execute software configured to receive a virtual surface-constrained input in conformance with the virtual surface-constrained input standard.
2. The method of claim 1 , wherein the free-space input is data indicative of a gesture of the hand or a posture of the hand.
3. The method of claim 1 , wherein the free-space input is data indicative of the hand interacting with a virtual object.
4. The method of claim 1 , further comprising determining, by the first processing device, whether the interaction by the hand with the virtual object corresponds to a touch input recognizable by a touch device.
5. The method of claim 1 , further comprising storing, by a memory device, one or more instructions executable by the first processing device.
6. The method of claim 1 , wherein the virtual surface-constrained input comprises a touch surface input from a touch device.
7. The method of claim 1 , wherein the virtual surface-constrained input comprises a touch down motion of an end-effector of the hand on a screen of a touch device.
8. The method of claim 1 , wherein the virtual surface-constrained input comprises a swipe motion on a screen of a touch device.
9. The method of claim 1 , wherein the virtual surface-constrained input comprises a peripheral device input, wherein the peripheral device input comprises:
a keystroke on a keyboard coupled to a touch device; or
a mouse click on a mouse coupled to the touch device.
10. The method of claim 1 , further comprising determining a virtual boundary for a virtual object, wherein an interaction by the hand with the virtual object comprises a gesture or a posture of the hand at or within the virtual boundary.
11. A device comprising:
an augmented-reality or virtual-reality display configured to be worn on a head of a user;
a sensor configured to detect a free-space input by a hand of the user; and
a first processing device coupled to the sensor and the augmented-reality or virtual-reality display, wherein the first processing device is configured to:
receive a first set of data indicative of a free-space input by a hand of a user wearing the augmented-reality or virtual-reality display, wherein the augmented-reality or virtual-reality display; and
in response to the free-space input satisfying a free-space input standard, wherein the free-space input standard comprises a first standard for a motion of the hand in three-dimensional space, in response to the free-space input satisfying the free-space input standard:
generate a second set of data indicative of a virtual surface-constrained input, wherein the virtual surface-constrained input conforms with the virtual surface-constrained input standard; and
send, to a second processing device, the virtual surface-constrained input, wherein the second processing device is configured to execute software configured to receive a virtual surface-constrained input in conformance with the virtual surface-constrained input standard.
12. The device of claim 11 , wherein the free-space input is data indicative of a gesture of the hand or a posture of the hand.
13. The device of claim 11 , wherein the free-space input is data indicative of the hand interacting with a virtual object.
14. The device of claim 11 , wherein the first processing device is configured to determine whether an interaction by the hand with a virtual object corresponds to a touch input recognizable by a touch device.
15. The device of claim 11 , wherein the virtual surface-constrained input comprises data is indicative of:
a first touch device instruction associated with a first bounce acceleration of a first interaction by a finger of the hand; or
a second touch device instruction associated with a second bounce acceleration of a second interaction by the finger of the hand, wherein:
the second bounce acceleration is different from the first bounce acceleration;
the second touch device instruction is different from the first touch device instruction; and
the first bounce acceleration or the second bounce acceleration is based on a rapid reversal of a motion of the finger.
16. A system comprising:
a head-worn device comprising:
an augmented-reality or virtual-reality display configured to be worn on a head of a user;
a sensor configured to detect a free-space input by a hand of the user;
a first processing device of the head-worn device coupled to the sensor, wherein the first processing device is configured to:
define a free-space input standard;
define a virtual surface-constrained input standard;
receive a first set of data indicative of the free-space input; and
in response to the free-space input satisfying the free-space input standard: and
generate a second set of data indicative of a virtual surface-constrained input, wherein the virtual surface-constrained input conforms with the virtual surface-constrained input standard; and
a second processing device configured to receive the virtual surface-constrained input, wherein the second processing device is configured to execute software configured to receive a virtual surface-constrained input in conformance with the virtual surface-constrained input standard.
17. The system of claim 14 , wherein the first processing device is further configured to generate virtual object data for displaying a virtual object by a mixed-reality device.
18. The system of claim 14 , wherein the first processing device is further configured to generate a free-space input instruction associated with a touch input of a touch device.
19. The system of claim 14 , wherein the first processing device is configured to determine whether an interaction by the hand with a virtual object corresponds to a touch input recognizable by a touch device.
20. The system of claim 17 , wherein the interaction with the virtual object comprises:
a first motion of the hand along a plane formed by a rectangular and planar boundary based on virtual object data; or
a second motion of the hand along a surface of a three-dimensional volume.