IP Library › Granted Patent US 11,164,585
Granted Patent B2
US 11,164,585 · App. 16/435,096 · Granted Nov 2, 2021

Systems and methods for virtual assistant routing

Inventors: Jacek Spiewla (Northville, MI); Sorin M. Panainte (Northville, MI)
Assignee: Mitsubishi Electric Automotive America, Inc.
G10L15/32G10L15/26H04L67/32
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,164,585
App. No.
16/435,096
Granted
Nov 2, 2021
Kind
B2
Abstract

Systems, methods and software are disclosed for processing requests from users of an infotainment system. The method includes receiving a request from a user of the infotainment system. The method includes determining a domain for the received request based on information contained in the received request. The domain specifies one or more categories for the request. The method includes routing the received request to a virtual assistant assigned to handle requests for the determined domain. The virtual assistant is one of a plurality of virtual assistants respectively assigned to handle requests for a plurality of respectively assigned domains. The method includes transmitting a response to the request to the user.

Claims (78)

1. A method for processing requests from users of an infotainment system, comprising:

receiving a request from a user of the infotainment system;

determining a domain for the received request based on information contained in the received request, the domain specifying one or more categories for the request;

routing the received request to a virtual assistant assigned to handle requests for the determined domain, wherein the virtual assistant is one of a plurality of virtual assistants respectively assigned to handle requests for a plurality of respectively assigned domains, wherein routing the received request to the virtual assistant comprises routing the received request to one of the plurality of virtual assistants respectively assigned to handle requests for the determined domain;

receiving a designation of a default assistant that is different from the virtual assistants, and wherein routing the received request to the virtual assistant comprises:

determining an availability of the one of the plurality of virtual assistants respectively assigned to handle requests for the determined domain; and

if the one of the plurality of virtual assistants is available, then

transmitting the request to the one of the plurality of virtual assistants, else,

transmitting the request to the default virtual assistant; and

transmitting a response to the request to the user, wherein:

determining the availability of the one of the plurality of virtual assistants respectively assigned to handle requests for the determined domain comprises:

if, after a first request transmission attempt, the one of the plurality of virtual assistants assigned to handle requests for the determined domain is not available, then

for at least a second attempt, retransmitting the request to the one of the plurality of virtual assistants;

transmitting the request to the default virtual assistant comprises transmitting the request to the default virtual assistant when, after the at least a second attempt, the one of the plurality of virtual assistants is not available; and

the method further comprises:

determining an availability of the default virtual assistant, wherein determining the availability of the default assistant comprises:

if, after at least a first request transmission attempt, the default assistant is not available, then

for at least a second attempt, retransmitting the request to the default virtual assistant; and

when, after the at least a second attempt, the default virtual assistant is not available, then transmitting a message to the user indicating that no virtual assistants are available.

2. The method of claim 1 , wherein:

receiving the request comprises receiving the request as a compound request including two or more requests from the user;

determining the domain for the received request comprises iteratively determining a respective domain for each of the two or more requests; and

routing the received request to the virtual assistant comprises iteratively routing each of the two or more requests to the one of the plurality of virtual assistants assigned to handle requests for each respectively determined domain.

3. A method for processing requests from users of an infotainment system, comprising:

receiving a request from a user of the infotainment system, wherein receiving the request comprises receiving the request from the user as speech;

storing recorded audio of the received speech;

determining a domain for the received request based on information contained in the received request, the domain specifying one or more categories for the request, wherein determining the domain for the received request comprises determining the domain for the received request based on information contained in the speech;

routing the received request to a virtual assistant assigned to handle requests for the determined domain, wherein the virtual assistant is one of a plurality of virtual assistants respectively assigned to handle requests for a plurality of respectively assigned domains; and

receiving a user permission status from the user for whom the recorded audio of the request received as speech was stored to transmit the stored audio to a TTS engine for voice model generation; and

if an affirmative user permission status is received, then

transmitting the stored audio to the TTS engine;

receiving the generated voice model from the TTS engine; and

storing the received voice model according to a user identity of the user for whom the recorded audio of the request received as speech was stored,

else,

deleting the stored audio of the user for whom the recorded audio of the request received as speech was stored.

4. The method of claim 3 further comprising transmitting a message to the user requesting a user permission status to transmit the stored audio to a text-to-speech (TTS) engine.

5. The method of claim 3 further comprising identifying a spoken language based on the information contained in the received speech, wherein transmitting the response to the request to the user comprises transmitting the response in the identified spoken language.

6. The method of claim 3 further comprising:

determining a spoken language of the user based on the received speech of the user; and

selecting a language model according to the determined spoken language for at least one of: speech to text conversion, and text to speech conversion, in one or more of the receiving, determining, routing, and transmitting steps of the method,

wherein transmitting the response to the request to the user comprises transmitting the response in the determined spoken language.

7. A method for processing requests from users of an infotainment system, comprising:

receiving a request from a user of the infotainment system, wherein receiving the request comprises receiving the request from the user as speech and receiving the request further comprises receiving a plurality of requests from a plurality of users of the infotainment system as a multi-user speech signal;

separating the multi-user speech signal into a plurality of individual speech signals for each of the plurality of users;

determining a domain for the received request based on information contained in the received request, the domain specifying one or more categories for the request, wherein determining the domain for the received request comprises determining the domain for the received request based on information contained in the speech;

routing the received request to a virtual assistant assigned to handle requests for the determined domain, wherein the virtual assistant is one of a plurality of virtual assistants respectively assigned to handle requests for a plurality of respectively assigned domains; and

transmitting a response to the request to the user;

determining a time of arrival (TOA) for each of the plurality of individual speech signals, wherein:

transmitting the response to the request to the user comprises iteratively transmitting a plurality of responses to each of the plurality of requests; and

iteratively transmitting the plurality of responses comprises one of:

transmitting the plurality of responses according to an ascending rank order of determined TOA; and

transmitting the plurality of responses according to a descending rank order of determined TOA, wherein:

receiving the plurality of requests comprises receiving the plurality of requests as a multi-user speech signal and separating the multi-user speech signal into a plurality of individual speech signals for each of the plurality of users.

8. A method for processing requests from users of an infotainment system, comprising:

receiving a request from a user of the infotainment system, wherein receiving the request comprises receiving the request from the user as speech and receiving the request further comprises receiving a plurality of requests from a plurality of users of the infotainment system as a multi-user speech signal;

separating the multi-user speech signal into a plurality of individual speech signals for each of the plurality of users;

determining a domain for the received request based on information contained in the received request, the domain specifying one or more categories for the request, wherein determining the domain for the received request comprises determining the domain for the received request based on information contained in the speech;

routing the received request to a virtual assistant assigned to handle requests for the determined domain, wherein the virtual assistant is one of a plurality of virtual assistants respectively assigned to handle requests for a plurality of respectively assigned domains; and

transmitting a response to the request to the user;

for each of the plurality of individual speech signals, determining a processing time for determining the domain for the received request based on the information contained in the received request, wherein:

transmitting the response to the request to the user comprises iteratively transmitting a plurality of responses to each of the plurality of requests; and

iteratively transmitting the plurality of responses comprises transmitting plurality of responses according to one of: an ascending rank order, and a descending rank order, of determined processing time, wherein:

receiving the plurality of requests comprises receiving the plurality of requests as a multi-user speech signal and separating the multi-user speech signal into a plurality of individual speech signals for each of the plurality of users.

9. A method for processing requests from users of an infotainment system, comprising:

receiving a request from a user of the infotainment system, wherein receiving the request comprises receiving the request from the user as speech and receiving the request further comprises receiving a plurality of requests from a plurality of users of the infotainment system as a multi-user speech signal;

separating the multi-user speech signal into a plurality of individual speech signals for each of the plurality of users;

determining a domain for the received request based on information contained in the received request, the domain specifying one or more categories for the request, wherein determining the domain for the received request comprises determining the domain for the received request based on information contained in the speech;

routing the received request to a virtual assistant assigned to handle requests for the determined domain, wherein the virtual assistant is one of a plurality of virtual assistants respectively assigned to handle requests for a plurality of respectively assigned domains; and

transmitting a response to the request to the user, wherein:

each of the plurality of users are positioned in a respective one of a plurality of spatially-separated audio zones;

at least two of the plurality of spatially-isolated audio zones include a respective one or more audio loudspeakers; and

the method further comprises determining a respective one or more of the plurality of spatially-separated audio zones from where one or more of the plurality of requests originated, wherein:

receiving the plurality of requests comprises receiving the plurality of requests as a multi-user speech signal and separating the multi-user speech signal into a plurality of individual speech signals for each of the plurality of users.

10. The method of claim 9 , wherein transmitting the response to the request to the user comprises transmitting each of the plurality of responses to the respective one or more loudspeakers according to the determined respective one or more of the plurality of spatially-separated audio zones.

11. The method of claim 7 further comprising:

determining an identity of each of the plurality of users from whom one or more of the plurality of requests originated; and

prepending a name of a respectively determined user identity to each of a plurality of responses to each of the plurality of requests,

wherein transmitting the response to the request to the user comprises transmitting the name of the respectively determined user identity prior to transmitting the response.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 7, 2019
From: SPIEWLA, JACEK; PANAINTE, SORIN M.
To: MITSUBISHI ELECTRIC AUTOMOTIVE AMERICA, INC.
Reel/Frame 049408/0626 →
Continuity (1)
Related Publication 20200388285A1 · Dec 10, 2020