Extramissive spatial imaging digital eye glass apparatuses, methods and systems for virtual or augmediated vision, manipulation, creation, or interaction with objects, materials, or other entities
A sensing and display apparatus, comprising: a first phenomenon interface configured to operatively interface with a first augmediated-reality space, and a second phenomenon interface configured to operatively interface with a second augmediated-reality space, is implemented as an extramissive spatial imaging digital eye glass.
1 . An apparatus comprising:
an inward facing display;
an external facing display;
at least one memory; and
at least one processor coupled to the at least one memory and configured to:
output, for display on the inward facing display, a virtual icon in an augmented reality space;
obtain eye tracking information associated with a user of the apparatus;
output, for display on the external facing display, an image of a portion of the user based on the eye tracking information;
visually modify, based on the eye tracking information, the virtual icon;
detect, from image data after visually modifying the virtual icon, a hand gesture including at least two fingers of a hand of the user; and
generate, based on the detected hand gesture, an output associated with the virtual icon in the augmented reality space.
2 . The apparatus of claim 1 , wherein the hand gesture includes physical contact between an index finger and a thumb of the hand.
3 . The apparatus of claim 1 , wherein, to visually modify the virtual icon, the at least one processor is configured to focus the virtual icon.
4 . The apparatus of claim 1 , wherein, to visually modify the virtual icon, the at least one processor is configured to highlight the virtual icon.
5 . The apparatus of claim 1 , wherein, to visually modify the virtual icon, the at least one processor is configured to move a location of the virtual icon.
6 . The apparatus of claim 1 , wherein, to generate the output, the at least one processor is configured to move a position of the virtual icon.
7 . The apparatus of claim 1 , wherein the virtual icon represents a software application.
8 . The apparatus of claim 7 , wherein, to generate the output, the at least one processor is configured to open the software application.
9 . The apparatus of claim 7 , wherein the at least one processor is configured to:
detect an additional gesture from the user; and
cause a virtual menu of the software application to be scrolled based on the detected additional gesture.
10 . The apparatus of claim 1 , wherein the virtual icon includes a virtual menu, and wherein to generate the output, the at least one processor is configured to cause the virtual menu to be scrolled based on the detected hand gesture.
11 . The apparatus of claim 1 , wherein, to generate the output, the at least one processor is configured to output a virtual line for display between at least one finger of the hand and the virtual icon.
12 . The apparatus of claim 1 , wherein, to generate the output, the at least one processor is configured to:
determine a first position of the hand based on the image data;
determine a second position of the hand based on the image data, wherein the second position is different than the first position; and
update a position of the virtual icon based on the second position of the hand.
13 . The apparatus of claim 12 , wherein the at least one processor is configured to:
determine a third position of the hand, wherein the third position is different than the second position; and
visually deemphasize the virtual icon based on the third position of the hand.
14 . The apparatus of claim 1 , wherein the at least one processor is configured to:
render virtual content associated with the augmented reality space; and
composite image data from one or more image sensors with the virtual content.
15 . The apparatus of claim 14 , wherein the virtual content associated with the augmented reality space includes a plurality of virtual icons representing a plurality of software applications.
16 . The apparatus of claim 1 , wherein the portion of the user includes eyes of the user.
17 . The apparatus of claim 1 , wherein the augmented reality space includes a virtual reality space.
18 . The apparatus of claim 1 , wherein the apparatus is a head-mounted device.
19 . The apparatus of claim 1 , wherein the apparatus is part of a head-mounted device.
20 . The apparatus of claim 1 , further comprising at least one camera configured to capture the image data.
21 . The apparatus of claim 1 , further comprising at least one display configured to display virtual content.
22 . The apparatus of claim 1 , wherein the hand gesture is a pinch gesture.
23 . A method comprising:
displaying, on an inward facing display of a device, a virtual icon in an augmented reality space;
obtaining eye tracking information associated with a user of the device;
displaying, on an external facing display of the device, an image of a portion of the user based on the eye tracking information;
visually modifying, based on the eye tracking information, the virtual icon;
detecting, from image data after visually modifying the virtual icon, a hand gesture including at least two fingers of a hand of the user; and
generating, based on the detected hand gesture, an output associated with the virtual icon in the augmented reality space.
24 . The method of claim 23 , wherein the hand gesture includes physical contact between an index finger and a thumb of the hand.
25 . The method of claim 23 , wherein visually modifying the virtual icon comprises focusing the virtual icon.
26 . The method of claim 23 , wherein visually modifying the virtual icon comprises highlighting the virtual icon.
27 . The method of claim 23 , wherein visually modifying the virtual icon comprises moving a location of the virtual icon.
28 . The method of claim 23 , wherein generating the output comprises moving a position of the virtual icon.
29 . The method of claim 23 , wherein the virtual icon represents a software application.
30 . The method of claim 29 , wherein generating the output comprises opening the software application.
31 . The method of claim 29 , further comprising:
detecting an additional gesture from the user; and
causing a virtual menu of the software application to be scrolled based on the detected additional gesture.
32 . The method of claim 23 , wherein the virtual icon includes a virtual menu, and wherein generating the output comprises causing the virtual menu to be scrolled based on the detected hand gesture.
33 . The method of claim 23 , wherein generating the output comprises displaying a virtual line between at least one finger of the hand and the virtual icon.
34 . The method of claim 23 , wherein generating the output comprises:
determining a first position of the hand based on the image data;
determining a second position of the hand based on the image data, wherein the second position is different than the first position; and
updating a position of the virtual icon based on the second position of the hand.
35 . The method of claim 34 , further comprising:
determining a third position of the hand, wherein the third position is different than the second position; and
visually deemphasizing the virtual icon based on the third position of the hand.
36 . The method of claim 23 , further comprising:
rendering virtual content associated with the augmented reality space; and
compositing image data from one or more image sensors with the virtual content.
37 . The method of claim 36 , wherein the virtual content associated with the augmented reality space includes a plurality of virtual icons representing a plurality of software applications.
38 . The method of claim 23 , wherein the portion of the user includes eyes of the user.
39 . The method of claim 23 , wherein the augmented reality space includes a virtual reality space.
40 . The method of claim 23 , wherein the device is a head-mounted device.
41 . The method of claim 23 , wherein the device is part of a head-mounted device.
42 . The method of claim 23 , wherein the hand gesture is a pinch gesture.