IP Library Granted Patent US 12,475,883
Granted Patent B2
US 12,475,883 · App. 18/182,762 · Granted Nov 18, 2025

Multi-assistant natural language input processing

Inventors: Munir Mahmood (Bellevue, WA); Leopold Bushkin (Renton, WA); Alexander Thomas Loeb (Lynnwood, WA); Michael Schwartz (Seattle, WA); Mohammed Arif (Bellevue, WA); Rongzhou Shen (Bothell, WA); Vikram Kumar Gundeti (Bellevue, WA); Shemyla Anwar (Bothell, WA); Yaser Khan (Kirkland, WA); Edward Page Foyle (Seattle, WA); Bo Li (Kenmore, WA)
Assignee: Amazon Technologies, Inc.
G10L15/1815G10L13/00G10L15/22G10L15/30G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,475,883
App. No.
18/182,762
Granted
Nov 18, 2025
Kind
B2
Abstract

Techniques for a natural language processing (NLP) system to implement more than one assistant are described. The NLP system may receive a natural language input corresponding to more than one user command. The NLP system may respond to a first command, of the natural language input, using a TTS voice of a first NLP system assistant. The NLP system may respond to a second command, of the natural language input, using a TTS voice of a second NLP system assistant.

Claims (42)

1 . A computer-implemented method, comprising:

receiving, by a component corresponding to at least a first virtual assistant and a second virtual assistant, first input data representing a first natural language input provided to a first device;

receiving a device identifier corresponding to the first device;

determining, using the device identifier, that the first virtual assistant is, for the first device, preferred to the second virtual assistant;

based at least in part on the first virtual assistant being preferred, processing the first input data using first data corresponding to the first virtual assistant to determine response data responsive to the first natural language input;

generating output data by performing speech synthesis processing using the response data and second data representing a synthetic voice corresponding to the first virtual assistant, wherein the output data comprises output audio data representing synthetic speech in the synthetic voice; and

causing presentation of the output data.

2 . The computer-implemented method of claim 1 , wherein:

receiving the first input data comprises receiving first input audio data representing an utterance of the first natural language input, wherein the utterance was captured by at least one microphone of the first device; and

performing speech processing using the first input audio data.

3 . The computer-implemented method of claim 2 , wherein the utterance comprises a wakeword associated with the first virtual assistant.

4 . The computer-implemented method of claim 2 , wherein performing the speech processing comprises operating a speech processing component associated with the first virtual assistant.

5 . The computer-implemented method of claim 1 , further comprising:

including, in the output data, an identifier corresponding to the first virtual assistant.

6 . The computer-implemented method of claim 1 , further comprising:

processing the device identifier to determine a profile associated with the first device; and

determining the profile is associated with the first virtual assistant.

7 . The computer-implemented method of claim 1 , further comprising:

processing the device identifier to determine a device type of the first device; and

determining the device type is associated with the first virtual assistant.

8 . A system comprising:

at least one processor; and

at least one memory comprising instructions that, when executed by the at least one processor, cause the system to:

receive, by a component corresponding to at least a first virtual assistant and a second virtual assistant, first input data representing a first natural language input provided to a first device;

receive a device identifier corresponding to the first device;

determining, using the device identifier, that the first virtual assistant is, for the first device, preferred to the second virtual assistant;

based at least in part on the first virtual assistant being preferred, process the first input data using first data corresponding to the first virtual assistant to determine response data responsive to the first natural language input;

based at least in part on the first virtual assistant being preferred, generate output data by performing speech synthesis processing using the response data and second data representing a synthetic voice corresponding to the first virtual assistant, wherein the output data comprises output audio data representing synthetic speech in the synthetic voice; and

cause presentation of the output data.

9 . The system of claim 8 , wherein:

the instructions that cause the system to receive the first input data comprise instructions that, when executed by the at least one processor, cause the system to receive first input audio data representing an utterance of the first natural language input, wherein the utterance was captured by at least one microphone of the first device; and

the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to perform speech processing using the first input audio data.

10 . The system of claim 9 , wherein the utterance comprises a wakeword associated with the first virtual assistant.

11 . The system of claim 9 , wherein the instructions that cause the system to perform the speech processing comprise instructions that, when executed by the at least one processor, cause the system to operate a speech processing component associated with the first virtual assistant.

12 . The system of claim 8 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

include, in the output data, an identifier corresponding to the first virtual assistant.

13 . The system of claim 8 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

process the device identifier to determine a profile associated with the first device; and

determine the profile is associated with the first virtual assistant.

14 . The system of claim 8 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

process the device identifier to determine a device type of the first device; and

determine the device type is associated with the first virtual assistant.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 13, 2023
From: MAHMOOD, MUNIR; BUSHKIN, LEOPOLD; LOEB, ALEXANDER THOMAS; SCHWARTZ, MICHAEL; ARIF, MOHAMMED; SHEN, RONGZHOU; GUNDETI, VIKRAM KUMAR; ANWAR, SHEMYLA; KHAN, YASER; FOYLE, EDWARD PAGE; LI, BO
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 062963/0846 →
Continuity (3)
Continuation 17387157 · Jul 28, 2021
Continuation 16580643 · Sep 24, 2019
Related Publication 20230282206A1 · Sep 7, 2023
References Cited (40)
US 6292830B1 · Taylor · 2001 [cited by examiner]
US 9495129B2 · Fleizach · 2016 [cited by examiner]
US 9875235B1 · Das · 2018 [cited by examiner]
US 9966065B2 · Gruber · 2018 [cited by examiner]
US 10043516B2 · Saddler · 2018 [cited by examiner]
US 10089072B2 · Piersol · 2018 [cited by examiner]
US 10170123B2 · Orr · 2019 [cited by examiner]
US 10255265B2 · Das · 2019 [cited by examiner]
US 10332523B2 · Leong · 2019 [cited by examiner]
US 10374816B1 · Leblang · 2019 [cited by examiner]
US 10388272B1 · Thomson · 2019 [cited by examiner]
US 10402501B2 · Wang · 2019 [cited by examiner]
US 10497365B2 · Gruber · 2019 [cited by examiner]
US 10536286B1 · Leblang · 2020 [cited by examiner]
US 10536287B1 · Leblang · 2020 [cited by examiner]
US 10536288B1 · Leblang · 2020 [cited by examiner]
US 10573312B1 · Thomson · 2020 [cited by examiner]
US 10789041B2 · Kim · 2020 [cited by examiner]
US 10854188B2 · Nygaard · 2020 [cited by examiner]
US 10957329B1 · Liu · 2021 [cited by examiner]
US 10971153B2 · Thomson · 2021 [cited by examiner]
US 11120790B2 · Mahmood · 2021 [cited by examiner]
US 11188376B1 · Alexander · 2021 [cited by examiner]
US 11393477B2 · Mahmood · 2022 [cited by examiner]
US 11423451B1 · Chaudhari · 2022 [cited by examiner]
US 12094463B1 · Mars · 2024 [cited by examiner]
US 20170046124A1 · Nostrant · 2017 [cited by examiner]
US 20180090143A1 · Saddler · 2018 [cited by examiner]
US 20180144748A1 · Leong · 2018 [cited by examiner]
US 20180314689A1 · Wang · 2018 [cited by examiner]
US 20180341644A1 · Retkowski · 2018 [cited by examiner]
US 20190206411A1 · Li · 2019 [cited by examiner]
US 20200167630A1 · Cronin · 2020 [cited by examiner]
US 20200175961A1 · Thomson · 2020 [cited by examiner]
US 20200175987A1 · Thomson · 2020 [cited by examiner]
US 20210090555A1 · Mahmood · 2021 [cited by examiner]
US 20210090572A1 · Mahmood · 2021 [cited by examiner]
US 20210090575A1 · Mahmood · 2021 [cited by examiner]
US 20210398525A1 · Mahmood · 2021 [cited by examiner]
US 20230282206A1 · Mahmood · 2023 [cited by examiner]