IP Library › Granted Patent US 11,437,043
Granted Patent B1
US 11,437,043 · App. 16/712,613 · Granted Sep 6, 2022

Presence data determination and utilization

Inventors: Lizhen Peng (Seattle, WA); Alok Upadhyay (Seattle, WA); Jason Cline (Bellevue, WA); Ankur Gandhe (Bothell, WA)
Assignee: Amazon Technologies, Inc.
G10L17/12G06N20/00G10L17/02G10L17/18G10L17/24H04L67/24
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,437,043
App. No.
16/712,613
Granted
Sep 6, 2022
Kind
B1
Abstract

Systems and methods for presence ground truth approximation and utilization are disclosed. For example, a system detects the presence of a predefined subject, such as a person associated with a given user profile, and/or determines that authentication criteria for performing an action in association with the user profile has been satisfied. A period of time to associate data is determined, and data of one or more data types is labeled as being associated with the speaker identification event. That data may be formatted and input into one or more models to train those models to more accurately detect presence and/or determine whether authentication of a user profile should succeed.

Claims (108)

1. A system, comprising:

one or more processors; and

non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

receiving, from a first device and at a first time, wireless signal data sent by a second device and received by the first device, the first wireless signal data representing a unique identifier of the second device;

receiving, from the first device after the first time, audio data representing an utterance spoken by a user;

performing speaker identification processing on the audio data to determine a user account of the user that spoke to the first device;

determining a period of time to associate with presence of the user at the first device, the period of time including time prior to receiving the audio data;

determining the first time was in the period of time;

associating the unique identifier of the second device with the user account;

receiving, from a third device, second wireless signal data representing the unique identifier of the second device; and

determining, using the unique identifier, the user is near the third device.

2. The system of claim 1 , the operations further comprising:

determining, utilizing the audio data, speaker identification data associated with the audio data;

determining a confidence score associated with the speaker identification data;

determining that the confidence score satisfies a threshold confidence score; and

wherein performing the speaker identification processing on the audio data to determine the user account is in response to the confidence score satisfying the threshold confidence score.

3. The system of claim 1 , the operations further comprising:

receiving, from the first device, first data indicating that a wake word was detected by the first device;

determining, utilizing the user account, that a second device is associated with the account data;

determining that the wake word was undetected by the second device;

querying, utilizing the user account, a database for second data associated with the second device for the period of time; and

determining that instances of wireless signal data associated with the second device are unassociated with the user account.

4. The system of claim 1 , the operations further comprising:

receiving request data for utilizing an application in association with the first device, the application having a user-authentication restriction;

determining that an authentication criteria for utilizing the application has been satisfied; and

performing the speaker identification processing to determine the user account is in response to the authentication criteria being satisfied.

5. A method, comprising:

receiving first data of a first data type indicating presence of a user with respect to a first device;

determining account data associated with the first data;

determining a period of time to associate with the presence of the user at the first device, the period of time including time prior to receiving the first data;

receiving, during the period of time, second data of a second type different than the first type;

associating the second data with the account data;

receiving, from a second device, third data of the second data type; and

determining, from the third data, presence of a user with respect to the second device.

6. The method of claim 5 , wherein the first data type includes audio data, and the method further comprises:

determining, based at least in part on the first data, speaker identification data associated with the first data;

determining a confidence score associated with the speaker identification data;

determining that the confidence score satisfies a threshold confidence score; and

wherein the first data indicates presence of the user based at least in part on the confidence score satisfying the threshold confidence score.

7. The method of claim 5 , further comprising:

receiving, from the first device, fourth data indicating that a wake word was detected by the first device;

determining, based at least in part on the account data, that a third device is associated with the account data;

querying, in absence of the wake word being detected by the third device and based at least in part on the account data, a database for fifth data associated with the third device for the period of time; and

determining that data of the second data type associated with the third device are unassociated with the account data.

8. The method of claim 5 , further comprising:

receiving request data for utilizing an application in association with the first device;

determining that an authentication criteria for utilizing the application has been satisfied; and

wherein the first data indicates presence of the user based at least in part on the authentication criteria being satisfied.

9. The method of claim 5 , wherein the second data includes at least one of:

speaker identification data indicating speaker identification processing performed in association with audio data from the first device;

authentication data indicating authentication of the user in association with the first device;

intent data indicating an intent associated with a user utterance received at the first device; or

device data indicating an accessory device communicatively coupled to the first device.

10. The method of claim 5 , wherein the first data is received based at least in part on the first device detecting a wake word, and the method further comprises:

determining, based at least in part on the account data, that a device is associated with the account data;

determining that the third device detected the wake word at a time associated with detection of the wake word by the first device;

querying, based at least in part on the account data, a database for fourth data associated with the third device for the period of time; and

associating the fourth data with the account data.

11. The method of claim 5 , further comprising:

sending request data to the first device, the request data for feedback associated with the first data indicating presence of the user;

receiving response data confirming presence of the user; and

weighting the second data based at least in part on the response data confirming presence of the user.

12. The method of claim 5 , further comprising:

determining, utilizing a machine-learning model, a data type determined by the machine-learning model to have an impact on a confidence score associated with determining presence of the user;

querying a database for fourth data associated with the data type; and

associating the fourth data with the account data.

13. A system, comprising:

one or more processors; and

non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

receiving first data of a first data type indicating presence of a user with respect to a first device;

determining account data associated with the first data;

determining a period of time to associate with the presence of the user at the first device, the period of time including time prior to receiving the first data;

receiving, during the period of time, second data of a second type different than the first type;

associating the second data with the account data;

receiving, from a second device, third data of the second data type; and

determining, from the third data, presence of a user with respect to the second device.

14. The system of claim 13 , the operations further comprising:

determining, based at least in part on the first data, speaker identification data associated with the first data;

determining a confidence score associated with the speaker identification data;

determining that the confidence score satisfies a threshold confidence score; and

wherein the first data indicates presence of the user based at least in part on the confidence score satisfying the threshold confidence score.

15. The system of claim 13 , the operations further comprising:

receiving, from the first device, fourth data indicating that a wake word was detected by the first device;

determining, based at least in part on the account data, that a third device is associated with the account data;

querying, in absence of the wake word being detected by the third device and based at least in part on the account data, a database for fifth data associated with the second device for the period of time; and

determining that data of the second data type associated with the third device are unassociated with the account data.

16. The system of claim 13 , the operations further comprising:

receiving request data for utilizing an application in association with the first device;

determining that an authentication criteria for utilizing the application has been satisfied; and

wherein the first data indicates presence of the user based at least in part on the authentication criteria being satisfied.

17. The system of claim 13 , wherein the second data includes at least one of:

speaker identification data indicating speaker identification processing performed in association with audio data from the first device;

authentication data indicating authentication of the user in association with the first device;

intent data indicating an intent associated with a user utterance received at the first device; or

device data indicating an accessory device communicatively coupled to the first device.

18. The system of claim 13 , wherein the first data is received based at least in part on the first device detecting a wake word, and the operations further comprise:

determining, based at least in part on the account data, that a third device is associated with the account data;

determining that the third device detected the wake word at a time associated with detection of the wake word by the first device;

querying, based at least in part on the account data, a database for fourth data associated with the third device for the period of time; and

associating the fourth data with the account data.

19. The system of claim 13 , the operations further comprising:

sending request data to the first device, the request data for feedback associated with the first data indicating presence of the user;

receiving response data confirming presence of the user; and

weighting the second data based at least in part on the response data confirming presence of the user.

20. The system of claim 13 , the operations further comprising:

determining, utilizing a machine-learning model, a data type determined by the machine-learning model to have an impact on a confidence score associated with determining presence of the user;

querying a database for fourth data associated with the data type; and

associating the fourth data with the account data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 16, 2022
From: PENG, LIZHEN; UPADHYAY, ALOK; CLINE, JASON; GANDHE, ANKUR
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 059920/0232 →
Cited By (2)
US 12,236,957 US 12,664,305