IP Library Granted Patent US 10,620,910
Granted Patent B2
US 10,620,910 · App. 15/390,191 · Granted Apr 14, 2020

Hands-free navigation of touch-based operating systems

Inventor: Christopher Iain Parkinson (Richland, WA)
Assignee: REALWEAR, INC.
G06F3/167G06F3/0485G06F3/0488
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,620,910
App. No.
15/390,191
Granted
Apr 14, 2020
Kind
B2
Abstract

Aspects of the present invention allow hands-free navigation of touch-based operating systems. The system may automatically interface with a touch-based operating system and generate hands-free commands that are associated with touch-based commands. Embodiments may utilize motion and/or audio inputs to facilitate hands-free interaction with the touch-based operating system.

Claims (33)

1. A computer-implemented method for hands-free navigation of a head-mounted computing device, the method comprising:

initializing, by the head-mounted computing device (HMD), a hands-free interaction mode, wherein the hands-free interaction mode enables hands-free interaction with a user interface of a touch-based operating system of the HMD;

based on the hands-free interaction mode being initialized, detecting, by the HMD, an initial position of the head-mounted computing device based on sensor data received from a set of sensors coupled to the HMD;

based on the initial position being detected, displaying, by the HMD, a first instance of a user interface of the touch-based operating system on a display of the head-mounted computing device, the first instance of the user interface being displayed while the HMD is in the detected initial position;

determining, by the HMD, that a first displacement of the HMD is greater than a defined first threshold displacement relative to the detected initial position, wherein the first displacement is determined based on additional sensor data received from the set of sensors;

converting, by the HMD, the first displacement to a corresponding set of touch-based inputs for communication to the touch-based operating system based on the determination that the first displacement is greater than the defined first threshold displacement.

2. The computer-implemented method of claim 1 , further comprising receiving a first audible input associated with the user interface of the touch-based operating system and converting the first audible input to another corresponding set of touch-based inputs.

3. The computer-implemented method of claim 1 , wherein the user interface is associated with an electronic document in the touch-based operating system.

4. The computer-implemented method of claim 2 , wherein the set of touch-based inputs comprises a scrolling instruction having a fixed number of scroll units.

5. The computer-implemented method of claim 1 , wherein the defined first threshold displacement corresponds to a minimum angular displacement of the head-mounted computing device relative to the initial position.

6. The computer-implemented method of claim 1 , wherein the set of sensors comprise at least one of a camera, accelerometer, gyroscope, magnetometer first displacement is detected by one or more sensors of the head-mounted computing device.

7. The computer-implemented method of claim 1 , wherein detecting the first displacement comprises determining a duration of the first displacement.

8. The computer-implemented method of claim 1 , further comprising displaying, by the HMD, a second instance of the user interface of the touch-based operating system on the display of the head-mounted computing device in response to the converted first displacement.

9. The computer-implemented method of claim 8 , further comprising:

detecting, by the HMD, a second displacement of the head-mounted computing device, the second displacement being above a defined second threshold displacement;

converting, by the HMD, a second instruction based on the detected second displacement, wherein the second instruction corresponds to the one or more touch-based inputs; and

displaying, by the HMD, a third instance of the user interface of the touch-based operating system on the display of the head-mounted computing device.

10. A system for hands-free navigation of a head-mounted computing device (HMD), the system comprising:

one or more processors; and

one or more computer storage media storing computer-useable instructions that, when executed by the one or more processors, implement a method comprising:

initializing a hands-free interaction mode, wherein the hands-free interaction mode enables hand-free interaction with a user interface of a touch-based operating system of the HMD;

displaying a first instance of a user interface of the touch-based operating system on a display of the HMD;

detecting, while the first instance of the user interface is being displayed, a first hands-free input comprising one or more of:

a first displacement of the head-mounted computing device based on sensor data received from a set of sensors coupled to the HMD, the first displacement being above a defined threshold displacement, or

a first audible input corresponding to a touch-based control dialogue included in the displayed first instance of the user interface;

converting the detected first hands-free input to a corresponding set of touch-based inputs for communication to the touch-based operating system; and

displaying a second instance of the user interface of the touch-based operating system on the display of the HMD.

11. The system of claim 10 , wherein the first hands-free input comprises the first displacement and the first audible input.

12. The system of claim 10 , wherein the implemented method further comprises:

detecting a second hands-free input;

determining a second instruction based on the second hands-free input, wherein the second instruction corresponds to the one or more touch-based inputs; and

displaying a third instance of the user interface of the touch-based operating system on the display of the head-mounted computing device.

13. The system of claim 10 , wherein the implemented method further comprises detecting a hands-free initialization input and generating an instruction to initialize the hands-free interaction mode on the head-mounted computing device.

Assignments (5)
SECURITY INTEREST Recorded Jun 6, 2024
From: REALWEAR, INC.
To: WESTERN ALLIANCE BANK
Reel/Frame 067646/0492 →
RELEASE OF SECURITY INTEREST Recorded Aug 21, 2023
From: RUNWAY GROWTH CREDIT FUND INC.
To: REALWEAR INC.
Reel/Frame 064654/0807 →
SECURITY INTEREST Recorded Aug 1, 2019
From: REALWEAR, INC.
To: RUNWAY GROWTH CREDIT FUND INC.
Reel/Frame 049933/0662 →
SECURITY INTEREST Recorded Feb 23, 2019
From: REALWEAR, INC.
To: RUNWAY GROWTH CREDIT FUND INC.
Reel/Frame 048418/0485 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 17, 2017
From: PARKINSON, CHRISTOPHER IAIN
To: REALWEAR, INCORPORATED
Reel/Frame 042031/0798 →
Continuity (1)
Related Publication 20180181370A1 · Jun 28, 2018