IP Library Granted Patent US 10,824,235
Granted Patent B2
US 10,824,235 · App. 16/241,893 · Granted Nov 3, 2020

Methods, devices, and systems for displaying a user interface on a user and detecting touch gestures

Inventors: Sean Jason Keller (Bellevue, WA); Tristan Thomas Trutna (Seattle, WA); Hrvoje Benko (Seattle, WA)
Assignee: FACEBOOK TECHNOLOGIES, LLC
G06F3/016G06F1/163G06F3/011G06F3/014G06F3/015G06F3/017G06K19/07762G06T19/006
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,824,235
App. No.
16/241,893
Granted
Nov 3, 2020
Kind
B2
Abstract

An example method of identifying a touch gesture on a user is provided. The method includes receiving, by one or more transducers of a first wearable device attached to a first appendage of the user, a set of signals transmitted by a second wearable device attached to the user, wherein (i) receiving the set of signals creates a signal pathway between the first and second wearable devices, and (ii) signals in the set of signals propagate through at least the user's first appendage. The method also includes determining baseline characteristics for the signal pathway created between the first wearable device and the second wearable device, sensing a change in the baseline characteristics while receiving the set of signals, and in accordance with a determination that the sensed change in the baseline characteristics for the signal pathway satisfies a contact criterion, report a candidate touch event on the user's first appendage.

Claims (59)

1. A method, comprising:

at a first wearable device, attached to a first arm of a user, which includes one or more transducers:

receiving, by the one or more transducers, a set of signals transmitted by a second wearable device attached to a second arm of the user, wherein (i) receiving the set of signals creates a signal pathway between the first and second wearable devices, and (ii) signals in the set of signals propagate through at least the user's first arm;

determining baseline characteristics for the signal pathway created between the first wearable device and the second wearable device;

while receiving the set of signals sensing a change in the baseline characteristics caused by user input on a first affordance of a user interface projected or perceived on the user's arm; and

in accordance with a determination that the sensed change in the baseline characteristics for the signal pathway satisfies a contact criterion:

sending transducer data for a candidate touch event, including the sensed change in the baseline characteristics, to a computer system, wherein the transducer data sent to the computer system comprises information indicating an approximate location of the candidate touch event.

2. The method of claim 1 , wherein the computer system:

captures, via one or more cameras, the candidate touch event;

generates image data according to the capturing of the candidate touch event; and

executes a function associated with the first affordance of the user interface after processing the transducer data and the image data.

3. The method of claim 1 , wherein:

the baseline characteristics include a baseline phase value; and

sensing the change in the baseline characteristics for the signal pathway comprises detecting a phase value of the signal pathway that differs from the baseline phase value.

4. The method of claim 3 , wherein:

the contact criterion includes a phase difference threshold; and

reporting the candidate touch event is performed in accordance with a determination that a difference between the phase value and the baseline phase value satisfies the phase difference threshold.

5. The method of claim 1 , wherein:

the baseline characteristics include a baseline amplitude value; and

sensing the change in the baseline characteristics for the signal pathway comprises detecting an amplitude value of the signal pathway that differs from the baseline amplitude value.

6. The method of claim 5 , wherein:

the contact criterion includes an amplitude difference threshold; and

reporting the candidate touch event is performed in accordance with a determination that a difference between the amplitude value and the baseline amplitude value satisfies the amplitude difference threshold.

7. The method of claim 1 , wherein:

the baseline characteristics include a baseline amplitude value and a baseline phase value; and

sensing the change in the baseline characteristics for the signal pathway comprises detecting (i) an amplitude value of the signal pathway that differs from the baseline amplitude value, and (ii) a phase value of the signal pathway that differs from the baseline phase value.

8. The method of claim 7 , wherein:

the contact criterion includes an amplitude difference threshold and a phase difference threshold; and

reporting the candidate touch event is performed in accordance with a determination that: (i) a difference between the amplitude value and the baseline amplitude value satisfies the amplitude difference threshold, and (ii) a difference between the phase value and the baseline phase value satisfies the phase difference threshold.

9. The method of claim 1 , wherein:

the contact criterion includes a time threshold;

sensing the change in the baseline characteristics comprises sensing the change for a period of time; and

reporting the candidate touch event is performed in accordance with a determination that the period of time satisfies the time threshold.

10. The method of claim 1 , further comprising, at the first wearable device:

before receiving the set of signals:

receiving a plurality predetermined values for signals characteristics, wherein each of the predetermined values for the signals characteristics corresponds to a specific location of the first arm of the user.

11. The method of claim 1 , wherein the candidate touch event is selected from the group consisting of: a tap gesture, a press-and-hold gesture, a multi-tap gesture, a swipe gesture, a drag gesture, a pinch gesture, a pull gesture, a hover, and a twist gesture.

12. The method of claim 1 , wherein the computer system is an artificial-reality system selected from the group consisting of: an augmented-reality system, a virtual-reality system, and a mixed-reality system.

13. A non-transitory computer-readable storage medium storing one or more programs configured for execution by one or more processors of a first wearable device, attached to a first arm of a user, wherein the first wearable device has one or more transducers, and the one or more programs include instructions, which when executed by the one or more processors, cause the first wearable device to:

display a user interface on the first arm of the user, the user interface including one or more affordances;

receive, by the one or more transducers, a set of signals transmitted by a second wearable device attached to a second arm of the user, wherein (i) receiving the set of signals creates a signal pathway between the first and second wearable devices, and (ii) signals in the set of signals propagate through at least the user's first arm;

determine baseline characteristics for the signal pathway created between the first wearable device and the second wearable device;

sense a change in the baseline characteristics while receiving the set of signals;

in accordance with a determination that the sensed change in the baseline characteristics for the signal pathway satisfies a contact criterion:

determine an approximate location of a candidate touch event on the user's first arm; and

send transducer data corresponding to the candidate touch event, including the sensed change in the baseline characteristics, to a computer system, wherein the transducer data sent to the computer system comprises information indicating the approximate location of the candidate touch event, and the candidate touch event is associated with a first affordance of the one or more affordances included in the user interface.

14. A system comprising a first computer, a first wearable device, and a second wearable device, wherein:

the first wearable device is attached to a first arm of a user and comprises:

one or more transducers;

one or more processors; and

memory storing one or more programs, which when executed by the one or more processors cause the first wearable device to:

receive, by the one or more transducers, a set of signals transmitted by a second wearable device attached to the user, wherein (i) receiving the set of signals creates a signal pathway between the first and second wearable devices, and (ii) signals in the set of signals propagate through at least the user's first arm;

determine baseline characteristics for the signal pathway created between the first wearable device and the second wearable device;

while receiving the set of signals, sense a change in the baseline characteristics caused by user input on a first affordance of a user interface projected or perceived on the user's arm; and

in accordance with a determination that the sensed change in the baseline characteristics for the signal pathway satisfies a contact criterion:

send transducer data for a candidate touch event, including the sensed change in the baseline characteristics, to the first computer, wherein the transducer data sent to the first computer comprises information indicating an approximate location of the candidate touch event.

15. The system of claim 14 , wherein the computer system:

captures, via one or more cameras, the approximate location of the candidate touch event corresponding to a location of the affordance in the user interface displayed on the user's first arm; and

executes a function associated with the affordance in response to determining that the user intended to interact with the first affordance and in accordance with the approximate location of the candidate touch event.

Assignments (3)
CHANGE OF NAME Recorded Jun 18, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060390/0066 →
CHANGE OF NAME Recorded Jun 6, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 061033/0801 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 15, 2019
From: KELLER, SEAN JASON; TRUTNA, TRISTAN THOMAS; BENKO, HRVOJE
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 050067/0695 →
Continuity (3)
Provisional Application 62647559 · Mar 23, 2018
Provisional Application 62647560 · Mar 23, 2018
Related Publication 20190212823A1 · Jul 11, 2019
Cited By (1)
US 12,693,707