Artificial reality systems with hand gesture-contained content window
An artificial reality system is described that renders, presents, and controls user interface elements within an artificial reality environment, and performs actions in response to one or more detected gestures of the user. The artificial reality system includes an image capture device configured to capture image data representative of a physical environment, a head-mounted display (HMD) configured to output artificial reality content, a gesture detector configured to identify, from the image data, a gesture comprising a configuration of a hand that is substantially stationary for at least a threshold period of time and positioned such that a thumb of the hand and at least one other finger form approximately a circle or approximately a circular segment, a user interface (UI) engine to generate a UI element in response to the identified gesture, and a rendering engine to render the UI element as an overlay to the artificial reality content.
1. An artificial reality system comprising:
an image capture device configured to capture image data representative of a physical environment;
a head-mounted display (HMD) configured to output artificial reality content;
a gesture detector configured to identify, from the image data, a gesture comprising a configuration of a hand that is substantially stationary for at least a threshold period of time and positioned such that a thumb of the hand and at least one other finger of the hand form an approximate circle or an approximate circular segment;
a user interface (UI) engine configured to generate a reproduction of a portion of the image data in response to the identified gesture, the portion of the image data being representative of a portion of the physical environment captured by the image capture device; and
a rendering engine configured to render the reproduction of the portion of the image data representative of the portion of the physical environment captured by the HMD within a boundary defined by an inner ring formed by the approximate circle or approximate circular segment while maintaining display of a remainder of the artificial reality content.
2. The artificial reality system of claim 1 , wherein to identify the gesture, the gesture detector is configured to determine that the hand is positioned such that a normal from an area within the approximate circle or approximate circular segment is facing the HMD.
3. The artificial reality system of claim 1 , wherein the artificial reality content comprises a virtual reality environment and wherein the rendering engine renders the reproduction of the portion of the image data representative of the portion of the physical environment captured by the HMD as an overlay to the virtual reality environment.
4. The artificial reality system of claim 1 , wherein the reproduction of the portion of the image data corresponds to a portion of the physical environment that lies along the normal from the area within the circle or the circular segment facing the HMD.
5. The artificial reality system of claim 1 , wherein the UI engine is further configured to generate a UI element comprising video data, and wherein the rendering engine is further configured to display the UI element within the boundary defined by the inner ring formed by the approximate circle or approximate circular segment and as an overlay to the artificial reality content.
6. The artificial reality system of claim 5 , wherein the video data comprises a plurality of moving pictures.
7. The artificial reality system of claim 1 , wherein to detect the gesture, the gesture detector is further configured to identify the gesture as corresponding to an entry in a gesture library.
8. The artificial reality system of claim 7 , wherein the entry is included in one or more entries of the gesture library, each of the one or more entries corresponding to a predetermined gesture.
9. The artificial reality system of claim 1 , wherein the image capture device is integrated within the HMD.
10. A non-transitory computer-readable storage medium encoded with instructions that, when executed, cause processing circuitry of an artificial reality system to:
receive, from an image capture device, image data representative of a physical environment;
output, via a head-mounted display (HMD), artificial reality content;
identify, from the image data, a gesture comprising a configuration of a hand that is substantially stationary for at least a threshold period of time and positioned such that a thumb of the hand and at least one other finger of the hand form an approximate circle or an approximate circular segment;
generate a reproduction of a portion of the image data in response to the identified gesture, the portion of the image data being representative of a portion of the physical environment captured by the image capture device; and
render the portion of the image data representative of the portion of the physical environment captured by the HMD within a boundary defined by an inner ring formed by the approximate circle or approximate circular segment while maintaining display of a remainder of the artificial reality content.
11. The artificial reality system of claim 1 , wherein the portion of the image data corresponding to the portion of the physical environment that lies along the normal represents a passthrough reproduction of the portion of the physical environment that lies along the normal with respect to the artificial reality content output by the HMD.
12. The artificial reality system of claim 1 , wherein the rendering engine is further configured to render a visual representation of the hand positioned such that the thumb and the at least one other finger form the approximate circle or approximate circular segment.
13. A method comprising:
capturing, by an image capture device, image data representative of a physical environment;
outputting, by a head-mounted display (HMD), artificial reality content;
identifying, by a gesture detector, from the image data, a gesture comprising a configuration of a hand that is substantially stationary for at least a threshold period of time and positioned such that a thumb of the hand and at least one other finger of the hand form an approximate circle or an approximate circular segment;
generating, by a user interface (UI) engine, in response to identifying of the gesture, a reproduction of a portion of the image data, the portion of the image data representative of a portion of the physical environment captured by the HMD; and
rendering, by a rendering engine, the reproduction of the portion of the image data representative of the portion of the physical environment captured by the HMD within a boundary formed by an inner ring defined by the approximate circle or approximate circular segment while maintaining display of a remainder of the artificial reality content.
14. The method of claim 13 , wherein identifying the gesture comprises determining, by the gesture detector, that the hand is positioned such that a normal from an area within the approximate circle or the approximate circular segment is facing the HMD.
15. The method of claim 13 , wherein the artificial reality content comprises a virtual reality environment and wherein rendering the reproduction of a portion of the image data representative of the portion of the physical environment comprises rendering the reproduction of the portion of the image data representative of the physical environment as an overlay to the virtual reality environment.
16. The method of claim 15 , wherein the portion of the image data corresponds to a portion of the physical environment that lies along the normal from the area within the circle or the circular segment facing the HMD.
17. The method of claim 13 , wherein the method further comprises rendering video data to be displayed within the boundary defined by the inner ring formed by the approximate circle or approximate circular segment and as an overlay to the artificial reality content.
18. The method of claim 13 , wherein identifying the gesture comprises identifying, by the gesture detector, the gesture as corresponding to an entry in a gesture library.
19. The method of claim 18 , wherein the entry is included in one or more entries of the gesture library, each of the one or more entries corresponding to a predetermined gesture.