Determining user handedness and orientation using a touchscreen device
View Patent ↗The present disclosure provides techniques for determining the position and/or orientation of a pointing device relative to the screen on a touchscreen device. A method may include receiving first orientation data from a first device that may include a capacitive touch surface. A touch point may be received indicating a location of a touch by a user on the capacitive touch surface. Second orientation data may be received from a second device. The first and second orientation data may be correlated to determine a relative orientation of the first device to the second device. A position of a pointing device may be determined based on the touch point and the relative orientation of the first and second devices. Additionally, multiple distances relative to a capacitive touch surface may be received, and based on the multiple distances, a position of a user's finger, hand, and/or arm may be determined.
1. A computer-implemented method comprising:
obtaining, by a mobile computing device, wearable device orientation data indicating a trajectory associated with a wearable device;
obtaining data corresponding to a trajectory associated with an input that was received through a proximity-sensitive display of the mobile computing device;
obtaining data indicating whether the wearable device is worn on a left hand or a right hand;
generating correlation data that reflects an extent to which the trajectory associated with the wearable device correlates with the trajectory associated with the input; and
classifying the input as a left-handed input or a right-handed input based at least on (i) the correlation data, and (ii) the data indicating whether the wearable device is worn on a left hand or a right hand.
2. The computer-implemented method of claim 1 , wherein:
the wearable device includes a wrist-wearable device;
the proximity-sensitive display includes a capacitive touch surface; and
obtaining data corresponding to the trajectory associated with the input that was received through the proximity-sensitive display of the mobile computing device comprises obtaining data identifying one or more of (i) a coordinate of the capacitive touch surface at which the input was received, (ii) a segment at which the input was received among multiple segments of the capacitive touch surface, and (iii) a quadrant of the capacitive touch surface where the input was received.
3. The computer-implemented method of claim 1 , wherein:
the data corresponding to the trajectory associated with the input that was received through the proximity-sensitive display of the mobile computing device includes one or more of (i) an angle associated with a yaw of the mobile computing device, (ii) an angle associated with a pitch of the mobile computing device, (iii) an angle associated with a roll of the mobile computing device, and (iv) an acceleration of the mobile computing device; and
the wearable device orientation data includes one or more of (I) an angle associated with a yaw of the wearable device, (II) an angle associated with a pitch of the wearable device, (III) an angle associated with a roll of the wearable device, and (IV) an acceleration of the wearable device.
4. The computer-implemented method of claim 1 , further comprising:
determining a relative orientation of the mobile computing device to the wearable device based on the generated correlation data; and
determining a position of a hand associated with the input based on (i) the determined relative orientation and (ii) the obtained data corresponding to the trajectory associated with the input that was received through the proximity-sensitive display of the mobile computing device.
5. The computer-implemented method of claim 4 , further comprising:
arranging an interface on the proximity-sensitive display based on the determined position of the hand, wherein:
the interface is arranged by one or more of moving, resizing, and repositioning the interface; and
the interface is arranged at a location on the proximity-sensitive display where the interface is likely to be visible to a user associated with the input.
6. The computer-implemented method of claim 1 , further comprising:
performing a first operation in the mobile computing device in response to classifying the input as a left-handed input, the left-handed input including a left-handed gesture; and
performing a second operation in the mobile computing device in response to classifying the input as a right-handed gesture, the second operation being different than the first operation and the right-handed input including a right-handed gesture.
7. The computer-implemented method of claim 1 , wherein generating correlation data that reflects an extent to which the trajectory associated with the wearable device correlates with the trajectory associated with the input comprises determining that the trajectory associated with the input matches the trajectory of the wearable device based on (i) the obtained data corresponding to the trajectory associated with the input and (ii) the trajectory associated with the wearable device orientation data; and
wherein classifying the input as a left-handed input or a right-handed input comprises determining that the wearable device is worn on a hand associated with the classified input.
8. A non-transitory computer-readable storage medium encoded with a computer program, the computer program comprising instructions that, upon execution by a computer, cause the computer to perform operations comprising:
obtaining wearable device orientation data indicating a trajectory associated with a wearable device;
obtaining data corresponding to a trajectory associated with an input that was received through a proximity-sensitive display of the mobile computing device;
obtaining data indicating whether the wearable device is worn on a left hand or a right hand;
generating correlation data that reflects an extent to which the trajectory associated with the wearable device correlates with the trajectory associated with the input; and
classifying the input as a left-handed input or a right-handed input based at least on (i) the correlation data, and (ii) the data indicating whether the wearable device is worn on a left hand or a right hand.
9. The non-transitory computer-readable storage medium of claim 8 , wherein:
the wearable device includes a wrist-wearable device;
the proximity-sensitive display includes a capacitive touch surface; and
obtaining data corresponding to the trajectory associated with the input that was received through the proximity-sensitive display of the mobile computing device comprises obtaining data identifying one or more of (i) a coordinate of the capacitive touch surface at which the input was received, (ii) a segment at which the input was received among multiple segments of the capacitive touch surface, and (iii) a quadrant of the capacitive touch surface where the input was received.
10. The non-transitory computer-readable storage medium of claim 8 , wherein:
the data corresponding to the trajectory associated with the input that was received through the proximity-sensitive display of the mobile computing device includes one or more of (i) an angle associated with a yaw of the mobile computing device, (ii) an angle associated with a pitch of the mobile computing device, (iii) an angle associated with a roll of the mobile computing device, and (iv) an acceleration of the mobile computing device; and
the wearable device orientation data includes one or more of (I) an angle associated with a yaw of the wearable device, (II) an angle associated with a pitch of the wearable device, (III) an angle associated with a roll of the wearable device, and (IV) an acceleration of the wearable device.
11. The non-transitory computer-readable storage medium of claim 8 , further comprising:
determining a relative orientation of the mobile computing device to the wearable device based on the generated correlation data; and
determining a position of a hand associated with the input based on (i) the determined relative orientation and (ii) the obtained data corresponding to the trajectory associated with the input that was received through the proximity-sensitive display of the mobile computing device.
12. The non-transitory computer-readable storage medium of claim 11 , further comprising:
arranging an interface on the proximity-sensitive display based on the determined position of the hand, wherein:
the interface is arranged by one or more of moving, resizing, and repositioning the interface; and
the interface is arranged at a location on the proximity-sensitive display where the interface is likely to be visible to a user associated with the hand associated with the input.
13. The non-transitory computer-readable storage medium of claim 8 , further comprising:
performing a first operation in the mobile computing device in response to classifying the input as a left-handed input, the left-handed input including a left-handed gesture; and
performing a second operation in the mobile computing device in response to classifying the input as a right-handed gesture, the second operation being different than the first operation and the right-handed input including a right-handed gesture.
14. The non-transitory computer-readable storage medium of claim 8 , wherein generating correlation data that reflects an extent to which the trajectory associated with the wearable device correlates with the trajectory associated with the input comprises determining that the trajectory associated with the input matches the trajectory of the wearable device based on (i) the obtained data corresponding to the trajectory associated with the input and (ii) the trajectory associated with the wearable device orientation data; and
wherein classifying the input as a left-handed input or a right-handed input comprises determining that the wearable device is worn on a hand associated with the classified input.
15. A system comprising:
one or more processors and one or more computer storage media storing instructions that are operable and when executed by the one or more processors, cause the one or more processors to perform operations comprising:
obtaining wearable device orientation data indicating of a trajectory associated with a wearable device;
obtaining data corresponding to a trajectory associated with an input that was received through a proximity-sensitive display of the mobile computing device;
obtaining data indicating whether the wearable device is worn on a left hand or a right hand;
generating correlation data that reflects an extent to which the trajectory associated with the wearable device correlates with the trajectory associated with the input; and
classifying the input as a left-handed input or a right-handed input based at least on (i) the correlation data, and (ii) the data indicating whether the wearable device is worn on a left hand or a right hand.
16. The system of claim 15 , wherein:
the wearable device includes a wrist-wearable device;
the proximity-sensitive display includes a capacitive touch surface;
the data corresponding to the trajectory associated with the input that was received through the proximity-sensitive display of the mobile computing device includes one or more of (i) an angle associated with a yaw of the mobile computing device, (ii) an angle associated with a pitch of the mobile computing device, (iii) an angle associated with a roll of the mobile computing device, and (iv) an acceleration of the mobile computing device;
the wearable device orientation data includes one or more of (v) an angle associated with a yaw of the wearable device, (vi) an angle associated with a pitch of the wearable device, (vii) an angle associated with a roll of the wearable device, and (viii) an acceleration of the wearable device; and
obtaining data corresponding to the trajectory associated with the input that was received through the proximity-sensitive display of the mobile computing device comprises obtaining data identifying one or more of (ix) a coordinate of the capacitive touch surface at which the input was received, (x) a segment at which the input was received among multiple segments of the capacitive touch surface, and (xi) a quadrant of the capacitive touch surface where the input was received.
17. The system of claim 15 , further comprising:
determining a relative orientation of the mobile computing device to the wearable device based on the generated correlation data; and
determining a position of a hand associated with the input based on (i) the determined relative orientation and (ii) the obtained data corresponding to the trajectory associated with the input that was received through the proximity-sensitive display of the mobile computing device.
18. The system of claim 17 , further comprising:
arranging an interface on the proximity-sensitive display based on the determined position of the hand, wherein:
the interface is arranged by one or more of moving, resizing, and repositioning the interface; and
the interface is arranged at a location on the proximity-sensitive display where the interface is likely to be visible to a user associated with the hand associated with the input.
19. The system of claim 15 , further comprising:
performing a first operation in the mobile computing device in response to classifying the input as a left-handed input, the left-handed input including a left-handed gesture; and
performing a second operation in the mobile computing device in response to classifying the input as a right-handed gesture, the second operation being different than the first operation and the right-handed input including a right-handed gesture.
20. The system of claim 15 , wherein generating correlation data that reflects an extent to which the trajectory associated with the wearable device correlates with the trajectory associated with the input comprises determining that the trajectory associated with the input matches the trajectory of the wearable device based on (i) the obtained data corresponding to the trajectory associated with the input and (ii) the trajectory associated with the wearable device orientation data; and
wherein classifying the input as a left-handed input or a right-handed input comprises determining that the wearable device is worn on a hand associated with the classified input.