IP Library Granted Patent US 11,789,544
Granted Patent B2
US 11,789,544 · App. 17/575,676 · Granted Oct 17, 2023

Systems and methods for communicating recognition-model uncertainty to users

Inventors: Ting Zhang (Lake Jackson, TX); Aakar Gupta (Redmond, WA); Marcello Giordano (Toronto, CA); Tanya Renee Jonker (Seattle, WA); Hrvoje Benko (Seattle, WA)
Assignee: Meta Platforms Technologies, LLC
G06F3/017G06F3/016G06F18/2415
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,789,544
App. No.
17/575,676
Granted
Oct 17, 2023
Kind
B2
Abstract

The disclosed computer-implemented method may include (1) receiving information associated with a real-time output of a recognition model adapted to recognize at least one behavior of a user, (2) determining, based on the information, a level of uncertainty associated with the real-time output, (3) modulating at least one attribute of feedback based on the level of uncertainty, and (4) presenting the feedback to the user substantially contemporaneous with the real-time output of the recognition model. Various other methods, systems, and computer-readable media are also disclosed.

Claims (84)

1. A computer-implemented method comprising:

receiving information associated with a real-time output of a recognition model, the recognition model being adapted to recognize at least one behavior of a user;

determining, based on the information, a level of uncertainty associated with the real-time output;

modulating at least one attribute of feedback to convey the level of uncertainty; and

conveying the level of uncertainty to the user by presenting the feedback to the user substantially contemporaneous with the real-time output of the recognition model.

2. The computer-implemented method of claim 1 , wherein:

the information associated with the real-time output of the recognition model comprises a probability of the user performing the behavior;

the computer-implemented method further comprises at least one of:

performing a user-input operation when the probability of the user performing the behavior is above a predetermined threshold; or

refraining from performing the user-input operation when the probability of the user performing the behavior is below the predetermined threshold; and

the level of uncertainty associated with the real-time output is determined based on a distance between the probability and the predetermined threshold, the level of uncertainty being inversely proportional to the distance.

3. The computer-implemented method of claim 1 , wherein:

the information associated with the real-time output of the recognition model comprises a probability of the user performing the behavior; and

the attribute of the feedback is modulated to have a level of perceivability proportional to the probability of the user performing the behavior.

4. The computer-implemented method of claim 1 , wherein the attribute of the feedback is modulated to have a level of perceivability proportional to the level of uncertainty associated with the real-time output.

5. The computer-implemented method of claim 1 , wherein:

the recognition model comprises a pinch recognition model adapted to output probabilities of the user performing a pinch gesture;

the information comprises a probability of the user performing the pinch gesture;

the computer-implemented method further comprises performing a user-input operation when the probability of the user performing the pinch gesture is above a predetermined threshold; and

the feedback is presented to the user contemporaneous with performing the user-input operation when the probability of the user performing the pinch gesture is above the predetermined threshold.

6. The computer-implemented method of claim 1 , wherein:

the recognition model comprises a pinch recognition model adapted to output probabilities of the user performing a pinch gesture;

the information comprises a probability of the user performing the pinch gesture;

the computer-implemented method further comprises refraining from performing a user-input operation when the probability of the user performing the pinch gesture is below a predetermined threshold; and

the feedback is presented to the user contemporaneous with determining that the probability of the user performing the pinch gesture is below the predetermined threshold.

7. The computer-implemented method of claim 1 , wherein:

the recognition model comprises a hand-tracking model adapted to output:

position or orientation information for one or more portions of the user's hand; and

a confidence level of the position or orientation information; and

the level of uncertainty associated with the real-time output is determined based on the confidence level, the level of uncertainty being inversely proportional to the confidence level.

8. The computer-implemented method of claim 1 , wherein:

the feedback is haptic feedback; and

at least one attribute of the haptic feedback is based on the level of uncertainty associated with the real-time output.

9. The computer-implemented method of claim 1 , wherein:

the feedback is a vibration; and

at least one attribute of the vibration is based on the level of uncertainty associated with the real-time output.

10. The computer-implemented method of claim 1 , wherein modulating the attribute of the feedback to convey the level of uncertainty comprises modulating one or more of:

an amplitude of the feedback to convey the level of uncertainty to the user;

a frequency of the feedback to convey the level of uncertainty to the user;

a duration of the feedback to convey the level of uncertainty to the user;

a pattern of the feedback to convey the level of uncertainty to the user; and

a spatialization of the feedback to convey the level of uncertainty to the user.

11. The computer-implemented method of claim 1 , wherein the feedback indicates a method for reducing the level of uncertainty of the real-time output.

12. The computer-implemented method of claim 1 , further comprising:

receiving additional information associated with an additional real-time output of the recognition model;

determining, based on the additional information, an additional level of uncertainty associated with the additional real-time output; and

modulating the attribute of the feedback to convey the additional level of uncertainty.

13. A system comprising:

at least one physical processor; and

physical memory comprising computer-executable instructions that, when executed by the physical processor, cause the physical processor to:

receive information associated with a real-time output of a recognition model, the recognition model being adapted to recognize at least one behavior of a user;

determine, based on the information, a level of uncertainty associated with the real-time output;

modulate at least one attribute of feedback to convey the level of uncertainty; and

convey the level of uncertainty to the user by presenting the feedback to the user substantially contemporaneous with the real-time output of the recognition model.

14. The system of claim 13 , wherein:

the information associated with the real-time output of the recognition model comprises a probability of the user performing the behavior;

the computer-executable instructions, when executed by the physical processor, further cause the physical processor to:

perform a user-input operation when the probability of the user performing the behavior is above a predetermined threshold; or

refrain from performing the user-input operation when the probability of the user performing the behavior is below the predetermined threshold; and

the level of uncertainty associated with the real-time output is determined based on a distance between the probability and the predetermined threshold, the level of uncertainty being inversely proportional to the distance.

15. The system of claim 13 , wherein:

the information associated with the real-time output of the recognition model comprises a probability of the user performing the behavior; and

the attribute of the feedback is modulated to have a level of perceivability proportional to the probability of the user performing the behavior.

16. The system of claim 13 , wherein the attribute of the feedback is modulated to have a level of perceivability proportional to the level of uncertainty associated with the real-time output.

17. The system of claim 13 , wherein:

the recognition model comprises a pinch recognition model adapted to output probabilities of the user performing a pinch gesture;

the information comprises a probability of the user performing the pinch gesture;

the computer-executable instructions, when executed by the physical processor, further cause the physical processor to perform a user-input operation when the probability of the user performing the pinch gesture is above a predetermined threshold; and

the feedback is presented to the user contemporaneous with performing the user-input operation when the probability of the user performing the pinch gesture is above a predetermined threshold.

18. The system of claim 13 , wherein:

the recognition model comprises a pinch recognition model adapted to output probabilities of the user performing a pinch gesture;

the information comprises a probability of the user performing the pinch gesture;

the computer-executable instructions, when executed by the physical processor, further cause the physical processor to refrain from performing a user-input operation when the probability of the user performing the pinch gesture is below a predetermined threshold; and

the feedback is presented to the user contemporaneous with determining that the probability of the user performing the pinch gesture is below the predetermined threshold.

19. The system of claim 13 , wherein:

the recognition model comprises a hand-tracking model adapted to output:

position or orientation information for one or more portions of the user's hand; and

a confidence level of the position or orientation information; and

the level of uncertainty associated with the real-time output is determined based on the confidence level, the level of uncertainty being inversely proportional to the confidence level.

20. A non-transitory computer-readable medium comprising one or more computer-executable instructions that, when executed by at least one processor of a computing device, cause the computing device to:

receive information associated with a real-time output of a recognition model, the recognition model being adapted to recognize at least one behavior of a user;

determine, based on the information, a level of uncertainty associated with the real-time output;

modulate at least one attribute of feedback to convey the level of uncertainty; and

convey the level of uncertainty to the user by presenting the feedback to the user substantially contemporaneous with the real-time output of the recognition model.

Assignments (2)
CHANGE OF NAME Recorded May 27, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060203/0228 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 20, 2022
From: ZHANG, TING; GUPTA, AAKAR; GIORDANO, MARCELLO; JONKER, TANYA RENEE; BENKO, HRVOJE
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 059656/0417 →