Automated assistants with conference capabilities
Techniques are described related to enabling automated assistants to enter into a “conference mode” in which they can “participate” in meetings between multiple human participants and perform various functions described herein. In various implementations, an automated assistant implemented at least in part on conference computing device(s) may be set to a conference mode in which the automated assistant performs speech-to-text processing on multiple distinct spoken utterances, provided by multiple meeting participants, without requiring explicit invocation prior to each utterance. The automated assistant may perform semantic processing on first text generated from the speech-to-text processing of one or more of the spoken utterances, and generate, based on the semantic processing, data that is pertinent to the first text. The data may be output to the participants at conference computing device(s). The automated assistant may later determine that the meeting has concluded, and may be set to a non-conference mode.
1 . A method implemented using one or more processors, comprising:
during a first conference dialog session between multiple participants, setting an automated assistant implemented at least in part on one or more conference computing devices to a conference mode in which the automated assistant performs semantic processing on multiple distinct spoken utterances without requiring explicit invocation of the automated assistant prior to each of the multiple distinct spoken utterances;
detecting, by the automated assistant based on the semantic processing of the multiple distinct utterances, one or more topics discussed by one or more of the multiple participants during the first conference dialog session;
storing one or more of the topics discussed by one or more of the multiple participants during the first conference dialog session for use during one or more subsequent conference dialog sessions, wherein the first conference dialog session and the one or more subsequent conference dialog sessions are distinct from one other;
subsequent to the storing, transitioning the automated assistant out of the conference mode;
determining that a second conference dialog session, in which the automated assistant is transitioned back into the conference mode, subsequent to the first conference dialog session is a follow up conference dialog session to the first conference dialog session, wherein the determining comprises analyzing one or more electronic calendar entries associated with the first and second conference dialog sessions to detect whether there is an overlap between the multiple participants of the first conference dialog session and at least one participant in the second conference dialog session that occurs subsequent to the first conference dialog session;
in response to detecting an overlap between the multiple participants of the first conference dialog session and at least one participant of the second conference dialog session, carrying over content of the first conference dialog session to the second conference dialog session, wherein the carrying over includes, and during the second conference dialog session that occurs subsequent to the first conference dialog session, retrieving one or more of the stored topics; and
generating, by the automated assistant, data indicative of the one or more retrieved topics; and
outputting, by the automated assistant at one or more of the conference computing devices, the data indicative of the one or more retrieved topics.
2 . The method of claim 1 , wherein the determining is based on titles assigned to the first and second conference dialog sessions.
3 . The method of claim 1 , wherein the determining is based on one or more documents associated with the first and second conference dialog sessions.
4 . The method of claim 3 , further comprising generating an initial dialog context for the second conference dialog session based on one or more of the documents.
5 . The method of claim 1 , wherein the one or more of the topics discussed by one or more of the multiple participants comprise a plurality of action items, and the method further comprises:
determining that a given action item of the plurality of action items was addressed during a time interval between the first and second conference dialog sessions; and
refraining from including the given action item in the data indicative of the one or more retrieved topics.
6 . The method of claim 1 , wherein the one or more of the topics discussed by one or more of the multiple participants comprise a plurality of action items, and the method further comprises, in response to a determination that a given action item of the plurality of action items was not addressed during the first conference dialog session or during a time interval between the first and second conference dialog sessions, including the given action item in the data indicative of the one or more retrieved topics.
7 . A non-transitory computer-readable medium comprising instructions that, in response to execution of the instructions by a processor, cause the processor to:
during a first conference dialog session between multiple participants, set an automated assistant implemented at least in part on one or more conference computing devices to a conference mode in which the automated assistant performs semantic processing on multiple distinct spoken utterances without requiring explicit invocation of the automated assistant prior to each of the multiple distinct spoken utterances;
detect, by the automated assistant based on the semantic processing of the multiple distinct utterances, one or more topics discussed by one or more of the multiple participants during the first conference dialog session;
store one or more of the topics discussed by one or more of the multiple participants during the first conference dialog session for use during one or more subsequent conference dialog sessions, wherein the first conference dialog session and the one or more subsequent conference dialog sessions are distinct from one other;
subsequent to the storing, transitioning the automated assistant out of the conference mode;
determine that a second conference dialog session, in which the automated assistant is transitioned back into the conference mode, subsequent to the first conference dialog session is a follow up conference dialog session to the first conference dialog session, wherein the instructions to determine comprise instructions to analyze one or more electronic calendar entries associated with the first and second conference dialog sessions to detect whether there is an overlap between the multiple participants of the first conference dialog session and at least one participant in the second conference dialog session that occurs subsequent to the first conference dialog session;
in response to detecting an overlap between the multiple participants of the first conference dialog session and at least one participant of the second conference dialog session, carry over content of the first conference dialog session to the second conference dialog session, wherein the instructions to carry over comprise instructions to, during the second conference dialog session that occurs subsequent to the first conference dialog session, retrieve one or more of the stored topics; and
generate, by the automated assistant, data indicative of the one or more retrieved topics; and
output, by the automated assistant at one or more of the conference computing devices, the data indicative of the one or more retrieved topics.
8 . The non-transitory computer-readable medium of claim 7 , wherein the determination that the second conference dialog session is related to the first conference dialog session is based on titles assigned to the first and second conference dialog sessions.
9 . The non-transitory computer-readable medium of claim 7 , wherein the determination that the second conference dialog session is related to the first conference dialog session is based on one or more documents associated with the first and second conference dialog sessions.
10 . A system comprising one or more processors and memory storing instructions that, in response to execution of the instructions, cause the one or more processors to:
during a first conference dialog session between multiple participants, set an automated assistant implemented at least in part on one or more conference computing devices to a conference mode in which the automated assistant performs semantic processing on multiple distinct spoken utterances without requiring explicit invocation of the automated assistant prior to each of the multiple distinct spoken utterances;
detect, by the automated assistant based on the semantic processing of the multiple distinct utterances, one or more topics discussed by one or more of the multiple participants during the first conference dialog session;
store one or more of the topics discussed by one or more of the multiple participants during the first conference dialog session for use during one or more subsequent conference dialog sessions, wherein the first conference dialog session and the one or more subsequent conference dialog sessions are distinct from one other;
subsequent to the storing, transition the automated assistant out of the conference mode;
determine that a second conference dialog session, in which the automated assistant is transitioned back into the conference mode, subsequent to the first conference dialog session is a follow up conference dialog session to the first conference dialog session, wherein the instructions to determine comprise instructions to analyze one or more electronic calendar entries associated with the first and second conference dialog sessions to detect whether there is an overlap between the multiple participants of the first conference dialog session and at least one participant in the second conference dialog session that occurs subsequent to the first conference dialog session;
in response to detecting an overlap between the multiple participants of the first conference dialog session and at least one participant of the second conference dialog session, carry over content of the first conference dialog session to the second conference dialog session, wherein the instructions to carry over comprise instructions to, during the second conference dialog session that occurs subsequent to the first conference dialog session, retrieve one or more of the stored topics; and
generate, by the automated assistant, data indicative of the one or more retrieved topics; and
output, by the automated assistant at one or more of the conference computing devices, the data indicative of the one or more retrieved topics.
11 . The system of claim 10 , wherein the determination that the second conference dialog session is related to the first conference dialog session is based on titles assigned to the first and second conference dialog sessions.
12 . The system of claim 10 , wherein the determination that the second conference dialog session is related to the first conference dialog session is based on one or more documents associated with the first and second conference dialog sessions.
13 . The system of claim 12 , further comprising instructions to generate an initial dialog context for the second conference dialog session based on one or more of the documents.
14 . The system of claim 10 , wherein the one or more of the topics discussed by one or more of the multiple participants comprise a plurality of action items, and the system further comprises instructions to:
determine that a given action item of the plurality of action items was addressed during a time interval between the first and second conference dialog sessions; and
refrain from including the given action item in the data indicative of the one or more retrieved topics.
15 . The system of claim 10 , wherein the one or more of the topics discussed by one or more of the multiple participants comprise a plurality of action items, and the system further comprises instructions to, in response to a determination that a given action item of the plurality of action items was not addressed during the first conference dialog session or during a time interval between the first and second conference dialog sessions, include the given action item in the data indicative of the one or more retrieved topics.