IP Library Granted Patent US 10,503,373
Granted Patent B2
US 10,503,373 · App. 13/420,275 · Granted Dec 10, 2019

Visual feedback for highlight-driven gesture user interfaces

Inventors: Carsten Schwesig (San Francisco, CA); Marvin Demerchant (San Diego, CA); Steven Friedlander (San Diego, CA); Brian Johnson (San Francisco, CA); Sebrina Yeh (Laguna Beach, CA); Hyehoon Yi (Escondido, CA); David Young (San Diego, CA)
Assignee: SONY INTERACTIVE ENTERTAINMENT LLC
G06F3/04842G06F3/017G06F3/0304G06F3/0482
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,503,373
App. No.
13/420,275
Granted
Dec 10, 2019
Kind
B2
Abstract

Disclosed are approaches for user interface control. In one embodiment, a method can include: indicating a highlight of a first object of a plurality of objects displayed on a display screen; accepting a signal deriving from a first gesture input of a user indicating a neutral position relative to the highlighted first object; accepting a signal deriving from a second gesture input of the user indicating a directional position at least a predetermined distance from the neutral position; and indicating, in response to the second gesture input, a movement of the highlight from the first object to a second object adjacent to the first object, and where the directional position of the second gesture input correlates to a position on the display screen of the second object relative to the first object.

Claims (51)

1. A method of controlling a user interface, the method comprising:

highlighting a first object on a display screen, wherein the first object is one of a plurality of objects displayed on the display screen;

accepting, by a computing device, a signal deriving from a first gesture input of a user, wherein the first gesture input, provided by a body part of the user without making any direct contact with the display screen, indicates a neutral position of the body part relative to the highlighted first object on the display screen;

providing on the display screen, in response to the first gesture input of the user, a graphical representation at a first position, the first position having a predetermined relationship with the position of the highlighted first object;

accepting, by the computing device, a signal deriving from a second gesture input of the user, wherein the second gesture input, caused by movement of the body part through a gesture distance from the neutral position along a gesture direction from the neutral position, indicates a directional position;

moving the graphical representation, in response to the second gesture input, to a second position on the display screen indicative of the directional position;

detecting whether the movement of the graphical representation to the second position crosses a circular ring threshold, centered at the neutral position corresponding to the screen position of the highlighted first object and characterized by a predetermined radius, the circular ring threshold being independent of positioning of any of the plurality of objects other than the highlighted first object; and

in response to detecting that the movement of the graphical representation to the second position does cross the circular ring threshold at a point on the display screen between the first object and a second object adjacent to the first object in the plurality of objects, highlighting the second object, wherein the directional position of the second gesture input correlates to a position on the display screen of the second object relative to the first object.

2. The method of claim 1 , wherein the highlight comprises a dimension, the method further comprising:

determining the movement of the highlight by using at least the directional position, the neutral position, and the dimension of the highlight.

3. The method of claim 1 , further comprising:

accepting, by the computing device a signal deriving from a third gesture input of the user, wherein the third gesture input indicates selection of the highlighted second object.

4. The method of claim 3 , wherein the third gesture input comprises a hand of the user in a grasp state.

5. The method of claim 1 , wherein the first gesture input comprises a hand of the user in a flat upright state.

6. The method of claim 5 , wherein the second gesture input comprises the flat upright state hand of the user being moved according to the directional position.

7. The method of claim 1 , further comprising:

performing an initialization procedure prior to accepting the signal deriving from the first gesture input of the user.

8. The method of claim 1 , wherein each of the plurality of objects represents media content that is playable on the display screen.

9. The method of claim 1 , wherein:

the movement of the highlight is performed at a first predetermined speed if the directional position lies between inner and outer concentric rings envisaged around the neutral position; and

the movement of the highlight is performed at a second predetermined speed if the directional position exceeds the outer concentric ring, wherein the second predetermined speed is greater than the first predetermined speed.

10. An apparatus configured to control a user interface, the apparatus comprising:

one or more processors; and

a computer-readable non-transitory storage medium having one or more instructions stored thereon for execution by the one or more processors, and when executed being operable to perform:

highlighting a first object on a display screen, wherein the first object is one of a plurality of objects displayed on the display screen;

accepting, by a computing device, a signal deriving from a first gesture input of a user, wherein the first gesture input, provided by a body part of the user without making any direct contact with the display screen, indicates a neutral position of the body part relative to the highlighted first object on the display screen;

providing on the display screen, in response to the first gesture input of the user, a graphical representation at a first position coupled to the highlighted first object;

accepting, by the computing device, a signal deriving from a second gesture input of the user, wherein the second gesture input, caused by movement of the body part through a gesture distance from the neutral position along a gesture direction from the neutral position, indicates a directional position;

moving the graphical representation, in response to the second gesture input, to a second position on the display screen indicative of the directional position;

detecting whether the movement of the graphical representation to the second position crosses a circular ring threshold, centered at the neutral position corresponding to the screen position of the highlighted first object, and characterized by a predetermined radius, the circular ring threshold being independent of positioning of any of the plurality of objects other than the highlighted first object; and

in response to detecting that the movement of the graphical representation to the second position does cross the circular ring threshold at a point on the display screen between the first object and a second object adjacent to the first object in the plurality of objects, highlighting the second object, wherein the directional position of the second gesture input correlates to a position on the display screen of the second object relative to the first object.

11. The apparatus of claim 10 , wherein the highlight comprises a dimension, the one or more instructions when executed being further operable to perform:

determining the movement of the highlight by using at least the directional position, the neutral position, and the dimension of the highlight.

12. The apparatus of claim 10 , the one or more instructions when executed being further operable to perform:

accepting, by the computing device, a signal deriving from a third gesture input of the user, wherein the third gesture input indicates selection of the highlighted second object.

13. The apparatus of claim 12 , wherein the third gesture input comprises a hand of the user in a grasp state.

14. The apparatus of claim 10 , wherein the first gesture input comprises a hand of the user in a flat upright state, and wherein the second gesture input comprises the flat upright state hand of the user being moved according to the directional position.

15. The apparatus of claim 10 , the one or more instructions when executed being further operable to perform:

performing an initialization procedure prior to accepting the signal deriving from the first gesture input of the user.

16. The apparatus of claim 10 , wherein each of the plurality of objects represents media content that is playable on the display screen.

17. The apparatus of claim 1 , wherein:

the movement of the highlight is performed at a first predetermined speed if the directional position lies between inner and outer concentric rings envisaged around the neutral position; and

the movement of the highlight is performed at a second predetermined speed if the directional position exceeds the outer concentric ring, wherein the second predetermined speed is greater than the first predetermined speed.

18. A system for controlling a user interface, the system comprising:

means for indicating a highlight of a first object on a display screen, wherein the first object is one of a plurality of objects displayed on the display screen;

means for accepting, by a computing device, a signal deriving from a first gesture input of a user, wherein the first gesture input, provided by a body part of the user without making any direct contact with the display screen, indicates a neutral position of the body part relative to the highlighted first object on the display screen;

means for providing on the display screen, in response to the first gesture input of the user, a graphical representation at a first position coupled to the highlighted first object;

means for accepting, by the computing device, a signal deriving from a second gesture input of the user, wherein the second gesture input, caused by movement of the body part through a gesture distance from the neutral position along a gesture direction from the neutral position, indicates a directional position;

means for moving the graphical representation, in response to the second gesture input, to a second position on the display screen indicative of the directional;

means for detecting whether the movement of the graphical representation to the second position crosses a circular ring threshold, centered at the neutral position corresponding to the screen position of the highlighted first object, and characterized by a predetermined radius, the circular ring threshold being independent of positioning of any of the plurality of objects other than the highlighted first object; and

means for highlighting, in response to detecting that the movement of the graphical representation to the second position does cross the circular ring threshold at a point on the display screen between the first object and a second object adjacent to the first object in the plurality of objects, wherein the directional position of the second gesture input correlates to a position on the display screen of the second object relative to the first object.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 10, 2018
From: SONY CORPORATION; SONY NETWORK ENTERTAINMENT INTERNATIONAL LLC
To: SONY INTERACTIVE ENTERTAINMENT LLC
Reel/Frame 046725/0835 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 14, 2012
From: SCHWESIG, CARSTEN; DEMERCHANT, MARVIN; FRIEDLANDER, STEVEN; JOHNSON, BRIAN; YEH, SABRINA; YI, HYEHOON; YOUNG, DAVID
To: SONY CORPORATION; SONY NETWORK ENTERTAINMENT INTERNATIONAL LLC
Reel/Frame 027864/0101 →
Continuity (1)
Related Publication 20130246955A1 · Sep 19, 2013