IP Library Granted Patent US 9,996,160
Granted Patent B2
US 9,996,160 · App. 14/183,171 · Granted Jun 12, 2018

Method and apparatus for gesture detection and display control

Inventor: Makoto Tachibana (Tokyo, JP)
Assignees: Sony Corporation; Sony Mobile Communications Inc.
G06F3/017G06F3/0485G06F3/0486
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,996,160
App. No.
14/183,171
Granted
Jun 12, 2018
Kind
B2
Abstract

A device includes a display and one or more motion sensors configured to track positions of two or more feature points of an object when the object is used to perform a gesture at a position remote from the display, and generate, based on the tracked positions of the feature points, sequential position data that indicates a relative position of the feature points with respect to each other and to the display. The device includes circuitry configured to determine, based on the sequential position data, whether the gesture corresponds to one of a plurality of predetermined input operations, and execute processing related to the predetermined input operation when a correspondence is determined. The circuitry is configured to determine whether the gesture corresponds to the predetermined input operation based on variations in the relative positions of the feature points with respect to each other and to the display.

Claims (46)

1. A device comprising:

one or more motion sensors configured to

track positions of two or more fingers of a user's hand when the user's hand is used to perform a gesture at a position remote from the one or more motion sensors; and

generate sequential position data based on the tracked positions of the fingers, wherein the sequential position data includes coordinates of the fingers and a distance between the fingers during a time period corresponding to the gesture;

a memory that stores the sequential position data received from the one or more motion sensors; and

circuitry configured to

determine, based on the sequential position data, whether the gesture corresponds to one of a plurality of predetermined input operations;

execute processing related to the predetermined input operation when a correspondence between the gesture and the predetermined input operation is determined;

execute processing related to a scroll operation, of the plurality of predetermined input operations, when it is determined that the distance between the fingers remains substantially the same while at least one of the coordinates is varied; and

temporarily suspend the processing related to the scroll operation when the distance between the fingers is decreased below a predetermined threshold distance while at least one of the coordinates is varied during the time period corresponding to the gesture.

2. The device according to claim 1 , wherein:

the circuitry is configured to determine whether the gesture corresponds to one of the predetermined input operations based on variations in the distance and at least one of the coordinates during the time period corresponding to the gesture.

3. The device according to claim 1 , wherein

the circuitry is configured to resume the scroll operation when the one or more motion sensors subsequently detect that the distance between the fingers exceed the predetermined threshold while at least one of the coordinates is varied during the time period corresponding to the gesture.

4. The device according to claim 2 , wherein

when the circuitry determines that the distance between the fingers decreases and then subsequently increases while the coordinates of one of the fingers remain substantially the same, the circuitry executes processing related to a second input operation, of the plurality of predetermined input operations.

5. The device according to claim 4 , wherein

the second input operation is a click operation.

6. The device according to claim 2 , wherein

when the circuitry determines that a decrease in the distance between the fingers immediately precedes a change in the coordinates of both the fingers while the decreased distance is maintained substantially the same, the circuitry executes processing related to a third input operation, of the plurality of predetermined input operations.

7. The device according to claim 6 , wherein

the third input operation is a drag operation.

8. The device according to claim 1 , wherein

the determination as to whether the gesture corresponds to one of the predetermined input operations is further based on a current operating state of the device.

9. The device according to claim 8 , wherein

when two of the plurality of predetermined input operations correspond to the same gesture, the circuitry is configured to determine which of the plurality of predetermined input operations to execute based on the current operating state of the device.

10. The device according to claim 1 , wherein

when two of the plurality of predetermined input operations correspond to the same gesture, the circuitry is configured to determine which of the plurality of predetermined input operations to execute based on an initial coordinate of the fingers at a start of the time period corresponding to the gesture.

11. The device according to claim 1 , wherein

when two of the plurality of predetermined input operations correspond to the same gesture and both of the predetermined input operations are available for execution in a currently active application, the circuitry is configured to assign a new gesture correspondence to one of the two predetermined input operations.

12. A method of determining input operations for execution on an electronic device, the method comprising:

tracking, by one or more motion sensors, positions of two or more fingers of a user's hand when the user's hand is used to perform a gesture at a position remote from the one or more motion sensors;

generating, by the one or more motion sensors, sequential position data based on the tracked positions of the fingers, wherein the sequential position data includes coordinates of the fingers and a distance between the fingers during a time period corresponding to the gesture;

storing, in a memory, the sequential position data received from the one or more motion sensors;

determining, by circuitry, based on the sequential position data, whether the gesture corresponds to one of a plurality of predetermined input operations;

executing, by the circuitry, processing related to the predetermined input operation when a correspondence between the gesture and the predetermined input operation is determined;

executing, by the circuitry, processing related to a scroll operation, of the plurality of predetermined input operations, when it is determined that the distance between the fingers remains substantially the same while at least one of the coordinates is varied; and

temporarily suspending, by the circuitry, the processing related to the scroll operation when the distance between the fingers is decreased below a predetermined threshold distance while at least one of the coordinates is varied during the time period corresponding to the gesture.

13. A non-transitory computer readable medium having instructions stored therein that when executed by one or more processors cause an electronic device including one or more motion sensors to perform a method of determining input operations for execution on the electronic device, the method comprising:

tracking, by the one or more motion sensors, positions of two or more fingers of a user's hand when the user's hand is used to perform a gesture at a position remote from the one or more motion sensors;

generating sequential position data based on the tracked positions of the fingers, wherein the sequential position data includes coordinates of the fingers and a distance between the fingers during a time period corresponding to the gesture;

storing, in a memory, the sequential position data received from the one or more motion sensors;

determining, based on the sequential position data, whether the gesture corresponds to one of a plurality of predetermined input operations;

executing processing related to the predetermined input operation when a correspondence between the gesture and the predetermined input operation is determined;

executing processing related to a scroll operation, of the plurality of predetermined input operations, when it is determined that the distance between the fingers remains substantially the same while at least one of the coordinates is varied; and

temporarily suspending the processing related to the scroll operation when the distance between the fingers is decreased below a predetermined threshold distance while at least one of the coordinates is varied during the time period corresponding to the gesture.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 26, 2019
From: SONY MOBILE COMMUNICATIONS, INC.
To: SONY CORPORATION
Reel/Frame 049404/0887 →
ASSIGNMENT OF PARTIAL RIGHTS Recorded Apr 20, 2018
From: SONY CORPORATION
To: SONY MOBILE COMMUNICATIONS INC.
Reel/Frame 045987/0643 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 14, 2014
From: TACHIBANA, MAKOTO
To: SONY CORPORATION
Reel/Frame 032441/0705 →
Continuity (1)
Related Publication 20150234467A1 · Aug 20, 2015