IP Library Granted Patent US 9,727,161
Granted Patent B2
US 9,727,161 · App. 14/303,203 · Granted Aug 8, 2017

Sensor correlation for pen and touch-sensitive computing device interaction

Inventors: Ken Hinckley (Redmond, WA); Hrvoje Benko (Seattle, WA); Michel Pahud (Kirkland, WA); Andrew D. Wilson (Seattle, WA); Pourang Polad Irani (Winnipeg, CA); Francois Guimbretiere (Ithaca, NY)
Assignee: Microsoft Technology Licensing, LLC
G06F3/0416G06F3/0383G06F3/03545G06F3/0482G06F3/0487G06F3/04883G06F2203/04104G06F2203/04808
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,727,161
App. No.
14/303,203
Granted
Aug 8, 2017
Kind
B2
Abstract

Pen and computing device sensor correlation technique embodiments correlate sensor signals received from various grips on a touch-sensitive pen and touches to a touch-sensitive computing device in order to determine the context of such grips and touches and to issue context-appropriate commands to the touch-sensitive pen or the touch-sensitive computing device. A combination of concurrent sensor inputs received from both a touch-sensitive pen and a touch-sensitive computing device are correlated. How the touch-sensitive pen and the touch-sensitive computing device are touched or gripped are used to determine the context of their use and the user's intent. A context-appropriate user interface action based can then be initiated. Also the context can be used to label metadata.

Claims (56)

1. A computer-implemented process for initiating user interface actions, comprising using a computer for:

receiving a combination of concurrent sensor inputs received from a touch-sensitive pen and a touch-sensitive computing device, comprising:

one or more sensor inputs based on grip patterns on the touch-sensitive pen,

one or more sensor inputs based touch patterns on the touch-sensitive computing device;

determining the user's preferred hand and non-preferred hand from the sensor inputs, wherein the user's preferred hand is determined by:

sensing a bump in a motion of the touch-sensitive pen when a contact is made with the touch-sensitive computing device; and

correlating the bump in a touch-sensitive pen's motion with a motion of the touch-sensitive computing device to determine that the hand holding the touch-sensitive pen is the preferred hand;

using the sensor inputs to determine how the touch-sensitive pen is being gripped concurrent with how the touch-sensitive computing device is being touched; and

initiating a user interface action based on a combination of the grip pattern on the touch-sensitive pen device and the touch pattern on the touch-sensitive computing device.

2. The computer-implemented process of claim 1 , further comprising using the determination of the user's preferred hand as metadata.

3. The computer-implemented process of claim 1 further comprising using the signals representing the orientation of the touch-sensitive pen in determining the grip patterns.

4. The computer-implemented process of claim 1 further comprising:

sensing motion of the touch-sensitive computing device;

correlating the sensed touch-sensitive computing device motion with the touch patterns on the touch-sensitive computing device; and

using the correlated motion and touch patterns on the touch-sensitive computing device to suppress accidental screen content rotation on a display of the touch-sensitive computing device.

5. The computer-implemented process of claim 1 , further comprising:

using the grip patterns, determining whether the user is making a gesture with a bare hand or whether the user is making a gesture with a hand holding the touch-sensitive pen.

6. The computer-implemented process of claim 1 , further comprising:

determining that the touch-sensitive pen is held in a stowed position in the user's preferred hand;

detecting a contact from the user's preferred hand on a display of the touch-sensitive computing device; and

displaying context-specific tools in response to the contact.

7. The computer-implemented process of claim 1 , further comprising:

determining that the touch-sensitive pen is held in the user's preferred hand in a grip indicating that the touch-sensitive pen is not in a position to write;

detecting a contact from the user's non-preferred hand on a display of the touch-sensitive computing device; and

displaying context-specific tools in response to the contact.

8. The computer-implemented process of claim 1 , further comprising:

determining that the touch-sensitive pen is held in a grip indicating that the touch-sensitive pen is in a position to write in the user's preferred hand;

detecting a contact from the user's non-preferred hand on a display of the touch-sensitive computing device; and

displaying context-specific tools in response to the contact.

9. The computer-implemented process of claim 1 , further comprising:

determining that the touch-sensitive pen is held in a grip indicating that the touch-sensitive pen is in a position to write in the user's preferred hand;

detecting a multi-finger touch gesture from the user's non-preferred hand on the display of the touch-sensitive computing device; and

displaying context-specific tools in response to the multi-finger touch gesture.

10. The computer-implemented process of claim 1 , further comprising:

determining that the touch-sensitive pen is held in a grip indicating that the touch-sensitive pen is not in a position to write in the user's preferred hand;

detecting a two-finger touch gesture from the user's preferred hand on the display of the touch-sensitive computing device; and

displaying a magnifier tool that magnifies content on a display of the touch-sensitive computing device in response to the two-finger touch gesture.

11. The computer-implemented process of claim 1 , further comprising:

determining a touch on a touch-sensitive screen of the touch-sensitive computing device is made by a user's thumb at the same time a new hand grip pattern on a corresponding back portion of the touch-sensitive computing device's case is detected; and

designating that the touch made by the thumb on the touch-sensitive screen is unintentional.

12. The computer-implemented process of claim 1 , further comprising:

determining which hand a user is holding the touch-sensitive computing device with; and

displaying a thumb menu in a location on the screen of the touch-sensitive computing device that can be activated by the user with the thumb of the hand holding the touch-sensitive computing device.

13. The computer-implemented process of claim 12 wherein the thumb menu fades away when the user no longer touches the touch-sensitive computing device.

14. A computer-implemented process for finding meta information, comprising:

using a computing device for:

correlating signals of simultaneous contacts on two or more touch-sensitive devices by a single user;

determining the user's preferred hand from the correlated signals of the simultaneous sensor inputs, wherein the user's preferred hand is determined by:

sensing a bump in a motion of the a first touch-sensitive device when a contact is made with the first touch-sensitive computing device; and

correlating the bump in the first touch-sensitive device's motion with a motion of the second touch-sensitive device to determine that the hand holding the first touch-sensitive device is the preferred hand;

determining a context of the contacts on the two or more touch-sensitive devices based on the correlation of the signals;

labeling the determined context of the contact as metadata for use in an application.

15. The computer-implemented process of claim 14 wherein one touch-sensitive device is a touch-sensitive pen and the other touch-sensitive device is a touch-sensitive computing device, and wherein the context of the contacts is determined by how the touch-sensitive pen is being gripped and how the touch-sensitive computing device is being touched with a user's bare hand.

16. The computer-implemented process of claim 14 , further comprising using the meta data to initiate a command in an application.

17. The computer-implemented process of claim 14 , wherein one or more of the signals represent a finger print of a user which is used to identify the user.

18. The computer-implemented process of claim 14 , further comprising using historical signal data to improve the correlation of the signals.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 9, 2015
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 039025/0454 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 7, 2014
From: HINCKLEY, KEN; BENKO, HRVOJE; PAHUD, MICHEL; WILSON, ANDREW D.; GUIMBRETIERE, FRANCOIS; IRANI, POURANG POLAD
To: MICROSOFT CORPORATION
Reel/Frame 034130/0505 →
Continuity (1)
Related Publication 20150363035A1 · Dec 17, 2015