IP Library Granted Patent US 10,853,747
Granted Patent B2
US 10,853,747 · App. 15/815,353 · Granted Dec 1, 2020

Selection of computational agent for task performance

Inventors: Bo Wang (San Jose, CA); Lei Zhong (Mountain View, CA); Barnaby John James (Los Gatos, CA); Saisuresh Krishnakumaran (Mountain View, CA); Robert Stets (Mountain View, CA); Bogdan Caprita (Mountain View, CA); Valerie Nygaard (Mountain View, CA)
Assignee: GOOGLE LLC
G06Q10/063112G10L15/08G10L15/22G06F16/951G10L13/00G10L2015/088G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,853,747
App. No.
15/815,353
Granted
Dec 1, 2020
Kind
B2
Abstract

An example method includes receiving, by a computational assistant executing at one or more processors, a representation of an utterance spoken at a computing device; identifying, based on the utterance, a task to be performed; determining a capability level of a first party (1P) agent to perform the task; determining capability levels of respective third party (3P) agents of a plurality of 3P agents to perform the task; responsive to determining that the capability level of the 1P agent does not satisfy a threshold capability level, that a capability level of a particular 3P agent of the plurality of 3P agents is a greatest of the determined capability levels, and that the capability level of the particular 3P agent satisfies the threshold capability level, selecting the particular 3P agent to perform the task; and performing one or more actions determined by the selected agent to perform the task.

Claims (78)

1. A method comprising:

receiving, by a computational assistant executing at one or more processors, a representation of an utterance spoken at a computing device;

identifying, based on the utterance, a task to be performed, wherein the task to be performed is a multi-element task;

determining a capability level of a first party agent to perform the task;

determining whether the capability level of the first party agent satisfies a threshold capability level;

in response to determining that the capability level of the first party agent does not satisfy the threshold capability level:

determining capability levels of respective third party agents of a plurality of third party agents to perform the task,

wherein determining that the capability level of the first party agent does not satisfy the threshold capability level comprises determining that the capability level of the first party agent is associated with a metric that indicates the first party agent can perform a sub-set of elements of the multi-element task;

determining whether a particular capability level of a particular third party agent, of the plurality of third party agents, satisfies the threshold capability level;

in response to determining that the particular capability level of the particular third party agent satisfies the threshold capability level;

selecting the particular third party agent, from among the plurality of third party agents, to perform the task,

wherein determining that the particular capability level of the particular third party agent satisfies the threshold capability level comprises determining that the particular capability level of the particular third party agent is associated with a particular metric that indicates the particular third party agent can perform each element of the multi-element task; and

causing the selected agent to perform the task.

2. The method of claim 1 , wherein the plurality of third party agents comprise a plurality of pre-configured third party agents, the method further comprising:

in response to determining that the particular capability level of the particular third party agent, of the plurality of pre-configured third party agents, does not satisfy the threshold capability level:

determining additional capability levels of additional respective third party agents, of a plurality of non-configured third party agents to perform the task;

determining whether an additional particular capability level of a particular non-configured third party agent, of the plurality of non-configured third party agents, satisfies the threshold capability level; and

in response to determining that the additional particular capability level of the particular non-configured third party agent satisfies the threshold capability level;

selecting the particular non-configured third party agent, from among the plurality of non-configured third party agents to perform the task.

3. The method of claim 1 , wherein the one or more processors are included in the computing device.

4. The method of claim 1 , wherein the one or more processors are included in a computing system.

5. A computing device comprising:

at least one processor; and

at least one memory comprising instructions that when executed, cause the at least one processor to execute an assistant configured to:

receive a representation of an utterance spoken at one or more microphones operably connected to the computing device;

identify, based on the utterance, a task to be performed, wherein the task to be performed is a multi-element task;

determine a capability level of a first party agent to perform the task;

determine whether the capability level of the first party agent satisfies a threshold capability level;

in response to determining that the capability level of the first party agent does not satisfy the threshold capability level:

determine capability levels of respective third party agents of a plurality of third party agents to perform the task,

wherein, in determining that the capability level of the first party agent does not satisfy the threshold capability level, the assistant is configured to determine that the capability level of the first party agent is associated with a metric that indicates the first party agent can perform a sub-set of elements of the multi-element task;

determine whether a particular capability level of a particular third party agent, of the plurality of third party agents, satisfies the threshold capability level;

in response to determining that the particular capability level of the particular third party agent satisfies the threshold capability level:

select the particular third party agent, from among the plurality of third party agents, to perform the task,

wherein, in determining that the particular capability level of the particular third party agent satisfies the threshold capability level, the assistant is configured to determine that the particular capability level of the particular third party agent is associated with a particular metric that indicates the particular third party agent can perform each element of the multi-element task; and

cause the selected agent to perform the task.

6. The device of claim 5 , wherein the plurality of third party agents comprise a plurality of pre-configured third party agents, the assistant further configured to:

in response to determining that the particular capability level of the particular third party agent, of the plurality of pre-configured third party agents, does not satisfy the threshold capability level:

determine additional capability levels of additional respective third party agents of a plurality of non-configured third party agents to perform the task;

determine whether an additional particular capability level of a particular non-configured third party agent, of the plurality of non-configured third party agents, satisfies the threshold capability level; and

in response to determining that the additional particular capability level of the particular non-configured third party agent satisfies the threshold capability level:

select the particular non-configured third party agent, from among of the plurality of non-configured third party agents to perform the task.

7. The device of claim 5 , wherein the assistant is further configured to: select a first party agent of the one or more first party agents responsive to determining that satisfaction of the utterance needs special assistance.

8. A computing system comprising:

one or more communication units;

at least one processor; and

at least one memory comprising instructions that when executed, cause the at least one processor to execute an assistant configured to:

receive, from a computing device and via the one or more communication units, a representation of an utterance spoken at the computing device;

identify, based on the utterance, a task to be performed, wherein the task to be performed is a multi-element task;

determine a capability level of a first party agent to perform the task;

determine whether the capability level of the first party agent satisfies a threshold capability level;

in response to determining that the capability level of the first party agent does not satisfy the threshold capability level:

determine capability levels of respective third party agents of a plurality of third party agents to perform the task,

wherein, in determining that the capability level of the first party agent does not satisfy the threshold capability level, the assistant is configured to determine that the capability level of the first party agent is associated with a metric that indicates the first party agent can perform a sub-set of elements of the multi-element task;

determine whether a particular capability level of a particular third party agent, of the plurality of third party agents, satisfies the threshold capability level;

in response to determining that the particular capability level of the particular third party agent satisfies the threshold capability level:

select the particular third party agent, from among the plurality of third party agents, to perform the task,

wherein, in determining that the particular capability level of the particular third party agent satisfies the threshold capability level, the assistant is configured to determine that the particular capability level of the particular third party agent is associated with a particular metric that indicates the particular third party agent can perform each element of the multi-element task; and

cause the selected agent to perform the task.

9. The system of claim 8 , wherein the plurality of third party agents comprise a plurality of pre-configured third party agents, the assistant further configured to:

in response to determining that the particular capability level of the particular third party agent, of the plurality of pre-configured third party agents, does not satisfy the threshold capability level:

determine additional capability levels of additional respective third party agents of a plurality of non-configured third party agents to perform the task;

determine whether an additional particular capability level of a particular non-configured third party agent, of the plurality of non-configured third party agents, satisfies the threshold capability level; and

in response to determining that the additional particular capability level of the particular non-configured third party agent satisfies the threshold capability level:

select the particular non-configured third party agent, from among of the plurality of non-configured third party agents to perform the task.

10. The system of claim 8 , wherein the assistant is further configured to: select a first party agent of the one or more first party agents responsive to determining that satisfaction of the utterance needs special assistance.

11. A non-transitory computer-readable storage medium storing instructions that, when executed, cause one or more processors to execute an assistant configured to:

identify, based on the utterance, a task to be performed, wherein the task to be performed is a multi-element task;

determine a capability level of a first party agent to perform the task;

determine whether the capability level of the first party agent satisfies a threshold capability level;

in response to determining that the capability level of the first party agent does not satisfy the threshold capability level:

determine capability levels of respective third party agents of a plurality of third party agents to perform the task,

wherein, in determining that the capability level of the first party agent does not satisfy the threshold capability level, the assistant is configured to determine that the capability level of the first party agent is associated with a metric that indicates the first party agent can perform a sub-set of elements of the multi-element task;

determine whether a particular capability level of a particular third party agent, of the plurality of third party agents, satisfies the threshold capability level;

in response to determining that the particular capability level of the particular third party agent satisfies the threshold capability level:

select the particular third party agent, from among the plurality of third party agents, to perform the task,

wherein, in determining that the particular capability level of the particular third party agent satisfies the threshold capability level, the assistant is configured to determine that the particular capability level of the particular third party agent is associated with a particular metric that indicates the particular third party agent can perform each element of the multi-element task; and

cause the selected agent to perform the task.

Assignments (2)
CHANGE OF NAME Recorded Apr 15, 2019
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 050268/0589 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 1, 2017
From: WANG, BO; ZHONG, LEI; JAMES, BARNABY JOHN; KRISHNAKUMARAN, SAISURESH; STETS, ROBERT; CAPRITA, BOGDAN; NYGAARD, VALERIE
To: GOOGLE INC.
Reel/Frame 044273/0050 →
Continuity (3)
Continuation PCTUS2017054456 · Sep 29, 2017
Provisional Application 62403665 · Oct 3, 2016
Related Publication 20180096283A1 · Apr 5, 2018