In-air ultrasound pen gestures
Implementations of the technology described herein provide a method for detecting gesture commands using an ultrasonic pen system. The system has a pen and a user device. Detection of gesture commands is based on two-dimensional gestures relative to the screen of a user device, three-dimensional gestures relative to the screen of the user device, roll/rotation around a longitudinal axis of the pen body, and micro-twisting around the longitudinal axis of the pen body. The user device receives the gestures and translates them into commands such as UNDO and BACK.
1. A method of detecting gesture commands using an ultrasonic pen and a user device, comprising:
acquiring a plurality of positions of the ultrasonic pen along one or more of an x-axis, a y-axis, and a z-axis of the user device;
recognizing a micro-twisting gesture of an entirety of the ultrasonic pen about a longitudinal axis of the ultrasonic pen, the micro-twisting gesture including a sequence of movements, the recognizing of the sequence of movements including:
recognizing a first movement around the longitudinal axis; and
recognizing a second movement, subsequent to the first movement, around the longitudinal axis in a direction opposite the first movement; and
performing a command based on the recognized sequence of movements included in the micro-twisting gesture.
2. The method of claim 1 , wherein acquiring the positions includes:
receiving an ultrasonic signal;
calculating a distance traveled by the ultrasonic signal; and
identifying the plurality of acquired positions based on the calculated distance.
3. The method of claim 1 , wherein the plurality of acquired positions are acquired from a top side or a bottom side of the user device.
4. The method of claim 1 , wherein the plurality of acquired positions are acquired from a left side or a right side of the user device.
5. An apparatus for detecting gesture commands using an ultrasonic pen and a user device, comprising a processor and a memory, the processor and memory being configured to:
acquire a plurality of positions of the ultrasonic pen along an x-axis, a y-axis, and a z-axis of the user device;
recognize a micro-twisting gesture of an entirety of the ultrasonic pen about a longitudinal axis of the ultrasonic pen, the micro-twisting gesture including a sequence of movements, wherein to recognize the sequence of movements, the processor and memory are further configured to:
recognize a first movement around the longitudinal axis; and
recognize a second movement, subsequent to the first movement, around the longitudinal axis in a direction opposite the first movement; and
perform a command based on the recognized sequence of movements included in the micro-twisting gesture.
6. The apparatus of claim 5 , further comprising a microphone array configured to receive an ultrasonic signal, wherein the processor and memory are further configured to:
calculate a distance traveled by the ultrasonic signal; and
identify the plurality of acquired positions based on the calculated distance.
7. The apparatus of claim 5 , wherein the first movement is in a first direction along one or more of the x-axis, the y-axis, and the z-axis, and the second movement is in a second direction along one or more of the x-axis, the y-axis, and the z-axis, the second direction being different from the first direction.
8. The apparatus of claim 5 , wherein the plurality of acquired positions are acquired from a top side or a bottom side of the user device.
9. A non-transitory computer-readable storage medium including information that, when accessed by a machine, cause the machine to perform operations for detecting gesture commands using an ultrasonic pen and a user device, the operations comprising:
acquiring a plurality of positions of the ultrasonic pen along one or more of an x-axis, a y-axis, and a z-axis of the user device;
recognizing a micro-twisting gesture of an entirety of the ultrasonic pen about a longitudinal axis of the ultrasonic pen, the micro-twisting gesture including a sequence of movements, the recognizing of the sequence of movements including:
recognizing a first movement around the longitudinal axis; and
recognizing a second movement, subsequent to the first movement, around the longitudinal axis in a direction opposite the first movement; and
performing a command based on the recognized sequence of movements included in the micro-twisting gesture.
10. The non-transitory computer-readable storage medium of claim 9 , wherein acquiring the positions includes:
receiving an ultrasonic signal;
calculating a distance traveled by the ultrasonic signal; and
identifying the plurality of acquired positions based on the calculated distance.
11. The non-transitory computer-readable storage medium of claim 9 , wherein the plurality of acquired positions are acquired from a left side or a right side of the user device.
12. The non-transitory computer-readable storage medium of claim 9 , wherein the plurality of acquired positions are acquired from a top side or a bottom side of the user device.