IP Library Granted Patent US 9,262,016
Granted Patent B2
US 9,262,016 · App. 13/143,266 · Granted Feb 16, 2016

Gesture recognition method and interactive input system employing same

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,262,016
App. No.
13/143,266
Granted
Feb 16, 2016
Kind
B2
Abstract

A gesture recognition method comprises capturing images, processing the images to identify at least two clusters of touch points associated with at least two pointers, recognizing a gesture based on motion of the clusters, and updating a display in accordance with the recognized gesture.

Claims (29)

1. A gesture recognition method comprising:

capturing images using imaging sensors having fields of view aimed generally across or at an input surface from different vantages;

processing the captured images to detect a pair of hands brought into contact with said input surface and for each detected hand calculating a bounding box, the calculated bounding box surrounding either a cluster of proximate touch points resulting from multiple fingers of the hand being in contact with said input surface or a single large touch region exceeding a threshold size resulting from a palm region of the hand being in contact with said input surface;

creating an observation for each bounding box in each captured image, each observation in each captured image defined by the area formed between two straight lines, one line of which extends from the focal point of the imaging sensor that captured the image and crosses the right edge of the bounding box and the other line of which extends from the focal point of the imaging sensor that captured the image and crosses the left edge of the bounding box;

in response to relative movement of the hands over the input surface, recognizing a gesture based on corresponding relative movement of the created observations;

executing a command associated with the recognized gesture; and

updating an image displayed on said input surface in accordance with the executed command.

2. An interactive input system comprising:

an input surface;

at least two imaging sensors having fields of view aimed generally across or at said input surface from different vantages; and

processing structure communicating with said at least one imaging sensor, said processing structure being configured to:

analyze images captured by said at least one imaging sensor to detect multiple hands brought into contact with said input surface,

for each detected hand, calculate a bounding box, the bounding box surrounding either a cluster of proximate touch points resulting from multiple fingers of the hand being in contact with said input surface or a single large touch region exceeding a threshold size resulting from a palm region of the hand being in contact with said input surface,

create an observation for each bounding box in each captured image, wherein each observation in each captured image defined by the area formed between two straight lines, one line of which extends from the focal point of the imaging sensor that captured the image and crosses the right edge of the bounding box and the other line of which extends from the focal point of the imaging sensor that captured the image and crosses the left edge of the bounding box;

in response to relative movement of the hands over the input surface, recognize a gesture based on corresponding relative movement of the created bounding boxes,

execute a command associated with said recognized gesture, and

update an image displayed on said input surface in accordance with the executed command.

3. The method of claim 1 , wherein said gesture is one of a zoom gesture and a pan gesture.

4. The method of claim 3 , wherein said zoom gesture is recognized by detecting an increase or a decrease in the distance between extreme edges of the created observations.

5. The method of claim 4 , wherein said command is an enlarge command if the distance increases and a shrink command if the distance decreases.

6. The method of claim 3 , wherein said pan gesture is recognized by detecting movement of said created observations in the same direction with the spacing between said created observations being maintained.

7. A non-transitory computer readable medium having computer program code stored thereon, the computer program code, when executed by a processor, causing an apparatus at least to perform the method of claim 1 .

8. The interactive input system of claim 2 , wherein said processing structure is configured to recognize one of a zoom gesture and a pan gesture.

9. The interactive input system of claim 8 , wherein said processing structure is configured to recognize said zoom gesture by detecting an increase or a decrease in the distance between extreme edges of the created observations.

10. The interactive input system of claim 9 , wherein said command is an enlarge command if the distance increases and a shrink command if the distance decreases.

11. The interactive input system of claim 8 , wherein said processing structure is configured to recognize said pan gesture by detecting movement of said created observations in the same direction with the spacing between said created observations being maintained.

12. The interactive input system of claim 2 , wherein said input surface is a display surface of a display unit.

13. The interactive input system of claim 2 , wherein said input surface is a generally horizontal table surface.

14. The interactive input system of claim 13 further comprising a projector for projecting said image on said input surface.

Assignments (7)
RELEASE OF SECURITY INTEREST Recorded Dec 2, 2016
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: SMART TECHNOLOGIES ULC; SMART TECHNOLOGIES INC.
Reel/Frame 040819/0306 →
RELEASE OF SECURITY INTEREST Recorded Dec 2, 2016
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: SMART TECHNOLOGIES ULC; SMART TECHNOLOGIES INC.
Reel/Frame 040798/0077 →
RELEASE OF ABL SECURITY INTEREST Recorded Nov 29, 2016
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: SMART TECHNOLOGIES ULC; SMART TECHNOLOGIES INC.
Reel/Frame 040711/0956 →
RELEASE OF TERM LOAN SECURITY INTEREST Recorded Nov 29, 2016
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: SMART TECHNOLOGIES ULC; SMART TECHNOLOGIES INC.
Reel/Frame 040713/0123 →
SECURITY AGREEMENT Recorded Aug 1, 2013
From: SMART TECHNOLOGIES ULC; SMART TECHNOLOGIES INC.
To: MORGAN STANLEY SENIOR FUNDING INC.
Reel/Frame 030935/0848 →
SECURITY AGREEMENT Recorded Aug 1, 2013
From: SMART TECHNOLOGIES ULC; SMART TECHNOLOGIES INC.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 030935/0879 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 5, 2011
From: MCGIBNEY, GRANT; XU, QIZHI JOANNA; BENNER, ERIK; TSE, EDWARD
To: SMART TECHNOLOGIES ULC
Reel/Frame 027017/0299 →