IP Library Granted Patent US 11,810,554
Granted Patent B2
US 11,810,554 · App. 17/067,315 · Granted Nov 7, 2023

Audio message extraction

Inventors: Neil Christopher Fritz (Tempe, AZ); Lakshya Bhagat (Seattle, WA); Scott Southwood (Seattle, WA); Katelyn Doran (Seattle, WA); Brett Lounsbury (Phoenix, AZ); Christo Frank Devaraj (Seattle, WA)
Assignee: Amazon Technologies, Inc.
G10L15/1815G06F40/295G10L15/1822G10L15/22G10L15/30H04M7/0042H04W4/12G10L2015/088
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,810,554
App. No.
17/067,315
Granted
Nov 7, 2023
Kind
B2
Abstract

Audio data, corresponding to an utterance spoken by a person within a detection range of a voice communications device, can include an audio message portion. The audio data can be captured and analyzed to determine the intent to send a message. Based at least in part upon that intent, a remaining portion of the audio data can be analyzed to determine the intended message target or recipient, as well as the portion corresponding to the actual message payload. Once determined, the audio file can be trimmed to the message payload, and the message payload of the audio data can be delivered as an audio message to the target recipient.

Claims (70)

1. A computer-implemented method, comprising:

receiving audio input data corresponding to an utterance received by a voice communications device, the audio input data received in response to detection of a wakeword;

determining a messaging intent represented by the audio input data;

determining, based at least in part on the messaging intent, a recipient identifier, the recipient identifier determined with at least a minimum level of confidence;

determining a beginning of a message payload; and

generating audio message data, the audio message data accessible according to the recipient identifier, and the audio message data including the message payload starting from the beginning.

2. The computer-implemented method of claim 1 , further comprising:

determining a slot pattern corresponding to the messaging intent, the slot pattern including a target slot and a message payload slot;

determining data associated with the target slot;

determining data associated with the message payload slot; and

determining the recipient identifier and the message payload based upon the locations of the data associated with the target slot the data associated with the message payload slot, with respect to the audio input data.

3. The computer-implemented method of claim 1 , further comprising:

generating tokenized text data by performing automated speech recognition (ASR) on the audio input data.

4. The computer-implemented method of claim 3 , further comprising:

determining, by performing natural language processing (NLP) on the tokenized text data, at least the messaging intent, a first word corresponding to the target slot and the message payload slot, and a second word corresponding to the message payload slot.

5. The computer-implemented method of claim 1 , wherein the voice communications device is associated with an account.

6. The computer-implemented method of claim 1 , wherein the audio message data includes sender information based at least in part on the account.

7. The computer-implemented method of claim 1 , further comprising:

determining, for the audio input data, an identity of a user having spoken the utterance;

determining a contact list for the user based upon the identity; and

determining the recipient identifier based upon a lookup of a target from the target slot against the contact list for with the user.

8. The computer-implemented method of claim 1 , wherein the recipient identifier is a device identifier or an address.

9. The computer-implemented method of claim 1 , further comprising:

determining that the recipient identifier is unable to be determined with at least the minimum level of confidence based on the audio input data;

causing additional audio input data to be received, the additional audio input data including additional identifying information for a target of the messaging intent; and

determining the recipient identifier based upon the additional identifying information.

10. The computer-implemented method of claim 2 , further comprising:

determining message text data corresponding to the message payload slot; and

sending the message text data for access by a recipient device associated with the recipient identifier.

11. A system, comprising:

at least one device processor;

memory including instructions that, when executed by the at least one device processor, cause the system to:

receive audio input data corresponding to an utterance received by a voice communications device, the audio input data received in response to detection of a wakeword;

determine a messaging intent represented by the audio input data;

determine, based at least in part on the messaging intent, a recipient identifier, the recipient identifier determined with at least a minimum level of confidence;

determine a beginning of a message payload; and

generate audio message data, the audio message data accessible according to the recipient identifier, and the audio message data including the message payload starting from the beginning.

12. The system of claim 11 , wherein the instructions, when executed, further cause the system to:

determine a slot pattern corresponding to the messaging intent, the slot pattern including a target slot and a message payload slot;

determine data associated with the target slot;

determine data associated with the message payload slot; and

determine the recipient identifier and the message payload based upon the locations of the data associated with the target slot and data associated with the message payload, with respect to the audio input data.

13. The system of claim 11 , wherein the instructions, when executed, further cause the system to:

generate tokenized text data by performing automated speech recognition (ASR) on the audio input data.

14. The system of claim 13 , wherein the instructions, when executed, further cause the system to:

determine, by performing natural language processing (NLP) on the tokenized text data, at least the messaging intent, a first word corresponding to the target slot and the message payload slot, and a second word corresponding to the message payload slot.

15. The system of claim 11 , wherein the voice communications device is associated with an account, and wherein the audio message data includes sender information based at least in part on the account.

16. The system of claim 11 , wherein the instructions, when executed, further cause the system to:

determine, for the audio input data, an identity of a user having spoken the utterance; determine a contact list for the user based upon the identity; and

determine the recipient identifier based upon a lookup of a target from the target slot against the contact list for with the user.

17. The system of claim 11 , wherein the instructions, when executed, further cause the system to:

determine that the recipient identifier is unable to be determined with at least the minimum level of confidence based on the audio input data;

cause additional audio input data to be received, the additional audio input data including additional identifying information for a target of the messaging intent; and

determine the recipient identifier based upon the additional identifying information.

18. A communication management system, comprising:

at least one device processor;

a memory device including instructions that, when executed by the at least one device processor, enable the system to:

receive, by a microphone, audio input data;

generate text data by performing automated speech recognition (ASR) on the audio input data;

determine a messaging intent by performing natural language processing (NLP) on the text data, the NLP applying at least one contextual rule;

determine a slot pattern corresponding to the messaging intent, the slot pattern including a target slot and a message payload slot;

identify, based at least in part upon the target slot, a recipient identifier, having the recipient identifier identified with at least a minimum level of confidence;

determine data associated with the message payload slot;

generate, based upon the data associated with the message payload slot, audio message data; and

send the audio message data for playback on an audio playback device associated with the recipient identifier.

19. The communication management system of claim 18 , wherein the instructions, when executed, further cause the system to:

determine that the recipient identifier is unable to be determined with at least the minimum level of confidence based on the audio input data;

cause additional audio input data to be received, the additional audio input data including additional identifying information for a target of the messaging intent; and

determine the recipient identifier based upon the additional identifying information.

20. The communication management system of claim 18 , wherein the recipient identifier is a device identifier or an address.

Continuity (3)
Continuation 16431343 · Jun 4, 2019
Continuation 15392291 · Dec 28, 2016
Related Publication 20210074275A1 · Mar 11, 2021