IP Library Granted Patent US 12,373,037
Granted Patent B2
US 12,373,037 · App. 18/374,513 · Granted Jul 29, 2025

Capturing touchless inputs and controlling a user interface with the same

Inventor: Zouheir Taher Fadlallah (London, GB)
Assignee: Hussein Ghassan H. Tahir Fadlallah
G06F3/017G06F3/0483G06F3/0485H04M1/72445
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,373,037
App. No.
18/374,513
Granted
Jul 29, 2025
Kind
B2
Abstract

A method and apparatus for capturing touchless inputs and controlling an electronic device with the same are described. In one embodiment, the method for controlling a user interface of software on a portable electronic device, the method comprising: obtaining one or more touchless inputs by capturing movements of the portable electronic device resulting from one or more hand gestures made by a hand holding the portable electronic device; determining one or more operations to control the user interface being displayed on a display of the portable electronic device based on the one or more touchless inputs; and controlling the software using at least one of the one or more operations.

Claims (46)

1. A method for controlling a user interface of software on a portable electronic device, the method comprising:

monitoring movements of the portable electronic device by a user;

computing trigger values for the user based on the movements being monitored;

storing the trigger values in a movement profile of the user;

obtaining one or more touchless inputs by capturing, based on trigger values in a movement profile of the user, rotation of the portable electronic device in one of a plurality of directions followed by rotation in an opposite direction back toward the initial position corresponds to a touchless input, the plurality of directions including forward, backward, and right, left tilt directions, and clockwise and counterclockwise directions, wherein obtaining the one or more touchless inputs comprises capturing a first rotation in one of the clockwise and counterclockwise directions and determining whether the first rotation constitutes one touchless input of the one or more touchless inputs;

determining one or more operations based on the one or more touchless inputs, wherein operations associated with clockwise and counterclockwise directions of rotation correspond to performing a first operation and canceling the first operation, wherein determining the one or more operations based on the one or more touchless inputs comprises determining at least one operation of the one or more operations is for canceling the first operation upon determining that the first rotation constitutes the one touchless input; and

controlling the software on the portable electronic device using at least one of the one or more operations.

2. The method of claim 1 wherein cancelling the first operation includes returning the software to its state when the first operation was performed.

3. The method of claim 1 further comprising determining if one or more touchless inputs has been performed by the user by:

capturing motion data from an accelerometer, a gyroscope and a magnetometer of the portable electronic device; and

computing a rotation rate of the portable electronic device.

4. The method of claim 1 wherein the rotation of the portable electronic device occurs in response to one or more hand gestures made by a hand of a user holding the portable electronic device without touching a screen of the portable electronic device.

5. The method of claim 1 further comprising computing an amount of time for the user the user to rotate the portable electronic device and return it to its initial position.

6. The method of claim 1 wherein the movement profile includes a computed time to return to a neutral position as one of the trigger values used for determining that the rotation in the opposite direction is not part of another touchless input.

7. The method of claim 1 further comprising determining that the rotation in the opposite direction is not part of another touchless input based on one or more of the trigger values for the user in the movement profile.

8. The method of claim 1 wherein the movement profile includes a computed time to return to a neutral position as one of the trigger values used for determining that the rotation in the opposite direction is not part of another touchless input.

9. The method of claim 1 wherein the forward, backward, right and left tilt directions are associated with rotation of the portable electronic device along a positive pitch axis toward the user, along a negative pitch axis away from the user, along a positive roll axis toward the user's right side, and along a negative roll axis toward the user's left side, respectively.

10. The method of claim 1 wherein the software comprises a web browser and the one or more operations are browsing navigation operations performed by the web browser.

11. The method of claim 2 wherein the one or more operations comprises one or more of a group consisting of: scrolling a page being displayed on a display screen of the portable electronic device, scrolling to a top of the page being displayed on a display screen of the portable electronic device, scrolling to a bottom of the page being displayed on a display screen of the portable electronic device, proceeding forward to a next page, proceeding backwards to previous page, displaying a list of one or more bookmarks, bookmarking the page, importing one or more bookmarks, proceeding to the next bookmark in a list of bookmarks, proceeding to a previous bookmark in a list of bookmarks, cancelling a navigation operation, setting a homepage, performing a search operation, viewing a browsing history, opening a new tab, closing a new tab, highlighting a hyperlink on a page, selecting a hyperlink, entering data, entering a website, sharing information, and performing an editing function.

12. The method of claim 1 wherein one of the touchless inputs causes display of a menu with a plurality of menu items on the display, each menu item of the plurality of menu items corresponds to a user interface action selectable with another touchless input.

13. A portable electronic device comprising:

one or more sensors to capture movements of the portable electronic device and provide data regarding the movements;

a display;

a memory to store instructions and captured data related to the movements; and

one or more processors coupled to the memory and the display to execute the stored instructions to:

monitor movements of the portable electronic device by a user,

compute trigger values for the user based on the movements being monitored,

store the trigger values in a movement profile of the user,

obtain one or more touchless inputs by capturing, based on trigger values in a movement profile of the user, rotation of the portable electronic device in one of a plurality of directions followed by rotation in an opposite direction back toward the initial position corresponds to a touchless input, the plurality of directions including forward, backward, and right, left tilt directions, and clockwise and counterclockwise directions, wherein the one or more processors are operable to obtain the one or more touchless inputs by capturing a first rotation in one of the clockwise and counterclockwise directions and determining whether the first rotation constitutes one touchless input of the one or more touchless inputs,

determine one or more operations based on the one or more touchless inputs, wherein operations associated with clockwise and counterclockwise directions of rotation correspond to performing a first operation and cancel the first operation, wherein the one or more processors are operable to determine the one or more operations based on the one or more touchless inputs by determining at least one operation of the one or more operations is for canceling the first operation upon determining that the first rotation constitutes the one touchless input, and

control the software on the portable electronic device using at least one of the one or more operations.

14. The portable electronic device of claim 13 wherein the one or more one or more processors are operable to cancel the first operation by returning the software to its state when the first operation was performed.

15. The portable electronic device of claim 13 wherein the one or more one or more processors are operable to determine if one or more touchless inputs has been performed by the user by:

capturing motion data from an accelerometer, a gyroscope and a magnetometer of the portable electronic device; and

computing a rotation rate of the portable electronic device.

16. The portable electronic device of claim 13 wherein the rotation of the portable electronic device occurs in response to one or more hand gestures made by a hand of a user holding the portable electronic device without touching a screen of the portable electronic device.

17. The portable electronic device of claim 13 wherein the one or more one or more processors are operable to compute an amount of time for the user to rotate the portable electronic device and return it to its initial position.

18. The portable electronic device of claim 13 wherein the movement profile includes a computed time to return to a neutral position as one of the trigger values used for determining that the rotation in the opposite direction is not part of another touchless input.

19. One or more non-transitory computer readable storage media having instructions stored thereupon which, when executed by a system having at least a processor and a memory therein, cause the system to perform a method for controlling a user interface of software on a portable electronic device having operations comprising:

monitoring movements of the portable electronic device by a user;

computing trigger values for the user based on the movements being monitored;

storing the trigger values in a movement profile of the user;

obtaining one or more touchless inputs by capturing, based on trigger values in a movement profile of the user, rotation of the portable electronic device in one of a plurality of directions followed by rotation in an opposite direction back toward the initial position corresponds to a touchless input, the plurality of directions including forward, backward, and right, left tilt directions, and clockwise and counterclockwise directions, wherein obtaining the one or more touchless inputs comprises capturing a first rotation in one of the clockwise and counterclockwise directions and determining whether the first rotation constitutes one touchless input of the one or more touchless inputs;

determining one or more operations based on the one or more touchless inputs, wherein operations associated with clockwise and counterclockwise directions of rotation correspond to performing a first operation and canceling the first operation, wherein determining the one or more operations based on the one or more touchless inputs comprises determining at least one operation of the one or more operations is for canceling the first operation upon determining that the first rotation constitutes the one touchless input; and

controlling the software on the portable electronic device using at least one of the one or more operations.

20. The one or more non-transitory computer readable storage media of claim 19 wherein cancelling the first operation includes returning the software to its state when the first operation was performed.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 7, 2024
From: FADLALLAH, ZOUHEIR TAHER
To: TAHIR FADLALLAH, HUSSEIN GHASSAN H.
Reel/Frame 067659/0037 →
Continuity (2)
Continuation 17339804 · Jun 4, 2021
Related Publication 20240036653A1 · Feb 1, 2024
References Cited (24)
US 9354709B1 · Heller et al. · 2016 [cited by applicant]
US 9606647B1 · Spencer-Harper et al. · 2017 [cited by applicant]
US 11314353B1 · Files · 2022 [cited by examiner]
US 20080235621A1 · Boillot · 2008 [cited by applicant]
US 20090303204A1 · Nasiri et al. · 2009 [cited by applicant]
US 20100214243A1 · Birnbaum et al. · 2010 [cited by applicant]
US 20120032877A1 · Watkins et al. · 2012 [cited by applicant]
US 20120304082A1 · Patten et al. · 2012 [cited by applicant]
US 20130117651A1 · Waldman et al. · 2013 [cited by applicant]
US 20130321263A1 · Ho · 2013 [cited by applicant]
US 20140253438A1 · Hoffman et al. · 2014 [cited by applicant]
US 20140333530A1 · Agnetta et al. · 2014 [cited by applicant]
US 20140351268A1 · Weskamp · 2014 [cited by examiner]
US 20150277581A1 · Coombs · 2015 [cited by applicant]
US 20160034143A1 · Sharma · 2016 [cited by applicant]
US 20160349940A1 · Kadur et al. · 2016 [cited by applicant]
US 20170108936A1 · Feinstein · 2017 [cited by applicant]
US 20180348975A1 · Taniguchi · 2018 [cited by applicant]
US 20190026063A1 · Mabey et al. · 2019 [cited by applicant]
US 20190121956A1 · Turgeman · 2019 [cited by applicant]
US 20190163432A1 · Files et al. · 2019 [cited by applicant]
US 20190310753A1 · Gong et al. · 2019 [cited by applicant]
International Search Report and Written Opinion received for PCT Patent Application No. PCT/IB2022/055466, mailed on Sep. 21, 2022, 9 pages. [cited by applicant]
International Preliminary Report on Patentability received for PCT Patent Application No. PCT/IB2022/055466, mailed on Dec. 14, 2023, 8 pages. [cited by applicant]