Two-handed gesture sequences in virtual, augmented, and mixed reality (xR) applications
View Patent ↗Systems and methods for enabling two-handed gesture sequences in virtual, augmented, and mixed reality (xR) applications are described. In an illustrative, non-limiting embodiment, an Information Handling System (IHS) may include a processor and a memory coupled to the processor, the memory having program instructions stored thereon that, upon execution, cause the IHS to: receive a gesture sequence from a user wearing an HMD coupled to the IHS, where the HMD is configured to display an xR application; identify the gesture sequence as a two-handed sequence; and execute a command in response to the two-handed sequence.
1. An Information Handling System (IHS), comprising:
a processor; and
a memory coupled to the processor, the memory having program instructions stored thereon that, upon execution, cause the IHS to:
receive a gesture sequence in a plurality of video frames from a user wearing a Head-Mounted Device (HMD) coupled to the IHS, wherein the HMD is configured to display a virtual, augmented, or mixed reality (xR) application;
perform a gesture recognition process upon: a first set of frames in the plurality of video frames corresponding to a start phase, a second set of frames in the plurality of video frames corresponding to a motion phase, and a third set of frames in the plurality of video frames corresponding to an end phase;
identify the gesture sequence as a two-handed sequence; and
execute a command in response to the two-handed sequence, wherein the wherein the two-handed sequence is selected from a set comprising at least one of:
(a) an active user handoff sequence comprising: (i) a first cusped hand with palm facing up, followed by (ii) a second cusped hand with palm facing up and joining the first cusped hand at an obtuse angle;
(b) a workspace lock sequence comprising: (i) a first hand with crossed middle and index fingers hovering over a first portion of an object, followed by (ii) a second hand with crossed middle and index fingers hovering over a second portion of the object; and wherein executing the command further comprises hiding the object from another user in a shared xR workspace; or
(c) a workspace unlock sequence comprising: (i) the first hand with uncrossed middle and index fingers hovering over the first portion of an object, followed by (ii) the second hand with uncrossed middle and index fingers hovering over the second portion of the object; and wherein executing the command further comprises showing the object to another user in a shared xR workspace.
2. The IHS of claim 1 , wherein the plurality of video frames is captured using a camera mounted on the HMD.
3. The IHS of claim 1 , wherein the set further comprises a display-on sequence comprising: (i) two hands with palms in covering the user's face, followed by (ii) two hands with palms out uncovering the user's face; and wherein the command further comprises turning on a display of the HMD.
4. The IHS of claim 1 , wherein the set further comprises a display-off sequence comprising: (i) two hands with palms out uncovering the user's face, followed by (ii) two hands with palms in covering the user's face; and wherein the command further comprises turning off a display of the HMD.
5. The IHS of claim 1 , wherein the set further comprises a video display overlay sequence comprising: (i) two hands making a frame shape until a display prompt appears, followed by (ii) moving the frame shape to a selected location on a display; and wherein the command further comprises locking the frame shape at the selected location.
6. The IHS of claim 1 , wherein the set further comprises a minimize all workspaces sequence comprising: (i) two hands horizontally leveled with palms facing down at a top or bottom portion of a display, followed by (ii) two hands moving horizontally leveled with palm facing down to an opposite portion of the display across two or more workspaces; and wherein executing the command further comprises minimizing the workspaces and displaying placeholders.
7. The IHS of claim 1 , wherein the set further comprises an open gesture sequence comprising: (i) two hands next to each other with palms facing out, followed by (ii) two hands flipping over with palms facing in; and wherein executing the command further comprises opening a file, application, or workspace.
8. The IHS of claim 1 , wherein the set further comprises a close gesture sequence comprising: (i) two hands next to each other with palms facing in, followed by (ii) two hands flipping over next to each other with palms facing out; and wherein executing the command further comprises closing a file, application, or workspace.
9. The IHS of claim 1 , wherein executing the command further comprises displaying an indication of the active user handoff in a shared xR workspace.
10. The IHS of claim 9 , wherein the program instructions, upon execution, further cause the IHS to display the active user indication over a head of an active user in the shared xR workspace.
11. The IHS of claim 9 , wherein the program instructions, upon execution, further cause the IHS to allow another user to perform a grabbing gesture sequence to retrieve the active user handoff indicator from the shared xR workspace.
12. The IHS of claim 9 , wherein the program instructions, upon execution, further cause the IHS to detect a collaborator list gesture sequence comprising a hand with palm out moving the tips of four extended fingers, wherein executing the command further comprises displaying a list of collaborators in the xR application, and wherein the program instructions, upon execution, further cause the IHS to select a second user from the list of collaborators.
13. The IHS of claim 1 , wherein to receive the gesture sequence, the program instructions, upon execution, further cause the IHS to receive an audio signal captured using a microphone mounted on the HMD.
14. The IHS of claim 13 , wherein the two-handed sequence is an audio mute or unmute gesture sequence comprising each hand covering or uncovering each respective ear, and wherein executing the command further comprises muting or unmuting two audio channels.
15. A method, comprising:
receiving a gesture sequence in a plurality of video frames from a user wearing a Head-Mounted Device (HMD) coupled to the IHS, wherein the HMD is configured to display a virtual, augmented, or mixed reality (xR) application;
performing a gesture recognition process upon: a first set of frames in the plurality of video frames corresponding to a start phase, a second set of frames in the plurality of video frames corresponding to a motion phase, and a third set of frames in the plurality of video frames corresponding to an end phase;
identifying the gesture sequence as a two-handed sequence; and
executing a command in response to the two-handed sequence, wherein the wherein the two-handed sequence is at least one of:
(a) an active user handoff sequence comprising: (i) a first cusped hand with palm facing up, followed by (ii) a second cusped hand with palm facing up and joining the first cusped hand at an obtuse angle;
(b) a workspace lock sequence comprising: (i) a first hand with crossed middle and index fingers hovering over a first portion of an object, followed by (ii) a second hand with crossed middle and index fingers hovering over a second portion of the object; and wherein executing the command further comprises hiding the object from another user in a shared xR workspace; or
(c) a workspace unlock sequence comprising: (i) the first hand with uncrossed middle and index fingers hovering over the first portion of an object, followed by (ii) the second hand with uncrossed middle and index fingers hovering over the second portion of the object and wherein executing the command further comprises showing the object to another user in a shared xR workspace.
16. A hardware memory device of an Information Handling System (IHS) having program instructions stored thereon that, upon execution by a hardware processor, cause the IHS to:
receive a gesture sequence in a plurality of video frames from a user wearing a Head-Mounted Device (HMD) coupled to the IHS, wherein the HMD is configured to display a virtual, augmented, or mixed reality (xR) application;
perform a gesture recognition process upon: a first set of frames in the plurality of video frames corresponding to a start phase, a second set of frames in the plurality of video frames corresponding to a motion phase, and a third set of frames in the plurality of video frames corresponding to an end phase;
identify the gesture sequence as a two-handed sequence; and
execute a command in response to the two-handed sequence, wherein the wherein the two-handed sequence is at least one of:
(a) an active user handoff sequence comprising: (i) a first cusped hand with palm facing up, followed by (ii) a second cusped hand with palm facing up and joining the first cusped hand at an obtuse angle;
(b) a workspace lock sequence comprising: (i) a first hand with crossed middle and index fingers hovering over a first portion of an object, followed by (ii) a second hand with crossed middle and index fingers hovering over a second portion of the object; and wherein executing the command further comprises hiding the object from another user in a shared xR workspace; or
(c) a workspace unlock sequence comprising: (i) the first hand with uncrossed middle and index fingers hovering over the first portion of an object, followed by (ii) the second hand with uncrossed middle and index fingers hovering over the second portion of the object and wherein executing the command further comprises showing the object to another user in a shared xR workspace.