Generating a summary of a conversation between users for an additional user in response to determining the additional user is joining the conversation
A communication system actively monitors a conversation between users in an environment. Once a threshold condition occurs (e.g., client device within a threshold distance of a room where the conversation is occurring, the user enters a chatroom via the client device), the client device receives a summary from the communication system. The communication system may customize the summary in accordance with user preferences. User preferences may include level of detail in summary or time range associated with summary. The client device then presents the summary to the user. The summary may be audio or text. In some embodiments, the summary may be historical audio that is overlaid over current audio at faster speed and higher volume, and stops once the historical audio reaches real time.
1 . A method comprising:
capturing, at a communication system, content from a conversation between a plurality of users via a plurality of artificial-reality (AR) devices communicatively coupled to the communication system;
generating, by the communication system, a summary of the conversation by:
identifying for respective ones of the plurality of users, a respective sentiment associated with a respective user, wherein each of the respective sentiments is determined based on data captured from the respective users during the conversation at a time when the respective sentiment is determined;
identifying one of the respective sentiments as a shared sentiment associated with at least a threshold percentage of the respective users; and
including, in the summary, an identification of the shared sentiment as being associated with at least the threshold percentage of the respective users;
after a start of the conversation, receiving, by the communication system, information indicating that an additional user is joining the conversation via another AR device communicatively coupled to the communication system; and
responsive to receiving the information indicating that the additional user is joining the conversation, automatically transmitting, by the communication system, a version of the summary to the other AR device, wherein the version of the summary comprises audio or text data based on the identification of the shared sentiment.
2 . The method of claim 1 , wherein the captured content comprises a plurality of messages, and generating the summary further comprises:
identifying messages from the plurality of messages sent within a threshold amount of time from a time at which the additional user joined the conversation;
wherein generating the summary is based on the identified messages.
3 . The method of claim 2 , wherein the threshold amount of time is set based on a preference specified by the additional user.
4 . The method of claim 2 , further comprising:
determining a type of the conversation based on the captured content; and
determining the threshold amount of time based on the determined type of the conversation.
5 . The method of claim 1 , wherein the generating of the summary is responsive to:
determining that a conversation identifier, included in a request received by the communication system from the other AR device, matches a conversation identifier of the conversation.
6 . The method of claim 1 , wherein the summary comprises text data configured to be displayed by a display element of the other AR device.
7 . The method of claim 1 , wherein the summary comprises audio data configured to be played by one or more speakers of the other AR device.
8 . The method of claim 7 , wherein the audio data is configured to be automatically played by the other AR device at a faster than real-time speed.
9 . The method of claim 7 , wherein the audio data is configured to be automatically played at a higher volume than a volume at which it was recorded.
10 . The method of claim 1 , wherein generating the summary further comprises:
identifying video data or audio data in the captured content; and
applying one or more machine learning models, trained to determine sentiment, to the identified video data or audio data to determine the respective sentiment of a respective user.
11 . The method of claim 1 , wherein the summary includes identifiers of users having identified sentiments different from the shared sentiment.
12 . A computer-readable storage medium storing instructions that, when executed by one or more processors of a communication system, cause the communication system to:
capture content from a conversation between a plurality of users via a plurality of artificial-reality (AR) devices communicatively coupled to the communication system;
generate a summary of the conversation by:
identifying, for respective ones of the plurality of users, a respective sentiment associated with a respective user, wherein each of the respective sentiments is determined based on data captured from the respective users during the conversation at a time when the respective sentiment is determined;
identifying one of the respective sentiments as a shared sentiment associated with at least a threshold percentage of the respective users; and
including, in the summary, an identification of the shared sentiment as being associated with at least the threshold percentage of the respective users;
after a start of the conversation, receive information indicating that an additional user is joining the conversation via another AR device communicatively coupled to the communication system; and
responsive to receiving the information indicating that the additional user is joining the conversation, automatically transmit a version of the summary to the other AR device, wherein the version of the summary comprises audio or text data based on the identification of the shared sentiment.
13 . The computer-readable storage medium of claim 12 , wherein the captured content comprises a plurality of messages, and generating the summary further comprises:
identifying messages from the plurality of messages sent within a threshold amount of time from a time at which the additional user joined the conversation;
wherein generating the summary is based on the identified messages.
14 . The computer-readable storage medium of claim 13 , wherein the threshold amount of time is set based on a preference specified by the additional user.
15 . The computer-readable storage medium of claim 13 , wherein the instructions further cause the communication system to:
determine a type of the conversation based on the captured content; and
determine the threshold amount of time based on the determined type of the conversation.
16 . The computer-readable storage medium of claim 12 , wherein the summary comprises text data configured to be displayed by a display element of the other AR device.
17 . A communication system comprising:
one or more processors; and
one or more memories storing instructions that, when executed by the one or more processors, cause the communication system to:
capture content from a conversation between a plurality of users via a plurality of artificial-reality (AR) devices communicatively coupled to the communication system;
generate a summary of the conversation by:
identifying for respective ones of the plurality of users, a respective sentiment associated with a respective user; wherein each of the respective sentiments is determined based on data captured from the respective users during the conversation at a time when the respective sentiment is determined;
identifying one of the respective sentiments as a shared sentiment associated with at least a threshold percentage of the respective users, and
including, in the summary, an identification of the shared sentiment as being associated with at least the threshold percentage of the respective users;
after a start of the conversation, receive information indicating that an additional user is joining the conversation via another AR device communicatively coupled to the communication system; and
responsive to receiving the information indicating that the additional user is joining the conversation, automatically transmit a version of the summary to the other AR device, wherein the version of the summary comprises audio or text data based on the identification of the shared sentiment.
18 . The communication system of claim 17 , wherein:
the summary comprises audio data configured to be played by one or more speakers of the other AR device, and
the audio data is configured to be automatically played by the other AR device at a faster than real-time speed.
19 . The communication system of claim 17 , wherein:
the summary comprises audio data configured to be played by one or more speakers of the other AR device, and
the audio data is configured to be automatically played at a higher volume than a volume at which it was recorded.
20 . The communication system of claim 17 , wherein generating the summary comprises:
identifying video or audio data in the captured content; and
applying one or more machine learning models, trained to determine sentiment, to the identified video data or audio data to determine the respective sentiment of a respective user.