GESTURES FOR NAVIGATING AND SELECTING ITEMS USING A LASER PROJECTED EPHEMERAL USER INTERFACE
Embodiments are disclosed for gestures for navigating and selecting items using a laser projected ephemeral user interface (UI). Also disclosed is a multi-layer menu system that allows a user to select different menu layers by moving their hand towards or away from a depth sensor. Also disclosed is a depth picker control that display different elements of a 3D stack of elements in response to the user moving their palm towards or away from the depth sensor.
1 . A method comprising:
detecting, using a first machine learning model, a three-dimensional (3D) image of a hand based on depth images of the hand;
predicting, using a second machine learning model, keypoints on the hand;
estimating, based on the keypoints and depth images, a normal to a palm of the hand;
computing a difference between the estimated normal and a resting normal;
projecting the difference into a two-dimensional (2D) plane of an ephemeral user interface projected on the user's hand palm;
focusing an underlying cursor on an affordance in the ephemeral user interface based on a location of the projected difference in the 2D plane;
detecting a gesture made by the user's hand; and
responsive to detecting the gesture, selecting the affordance.
2 . A method comprising:
projecting an ephemeral user interface onto a hand palm, the ephemeral user interface including a first set of affordances;
detecting a change in depth of the hand palm relative to a depth sensor; and
responsive to the change in depth, projecting a second set of affordances in the ephemeral user interface.
3 . A method comprising:
projecting a virtual stack of elements on a hand palm with a first element in the stack displaying a first value;
detecting a change in depth of the hand palm relative to a depth sensor; and
responsive to the change in depth, replacing the first element with a second element of the stack displaying a second value.