Generating a summary of a conversation between users for an additional user in response to determining the additional user is joining the conversation
A communication system actively monitors a conversation between users in an environment. Once a threshold condition occurs (e.g., client device within a threshold distance of a room where the conversation is occurring, the user enters a chatroom via the client device), the client device receives a summary from the communication system. The communication system may customize the summary in accordance with user preferences. User preferences may include level of detail in summary or time range associated with summary. The client device then presents the summary to the user. The summary may be audio or text. In some embodiments, the summary may be historical audio that is overlaid over current audio at faster speed and higher volume, and stops once the historical audio reaches real time.
1. A method comprising:
capturing a conversation between a plurality of users at a communication system, the conversation including one or more messages exchanged between the plurality of users that are located within a same room;
determining, from one or more position sensors included in a client device of an additional user other than any of the plurality of users, that a location of the client device is within a threshold distance of a location associated with the conversation, wherein the location of the client device and the location associated with the conversation are within the same room;
determining, by the communication system based at least in part on the location of the client device being within the threshold distance of the location associated with the conversation, that characteristics of the client device satisfy one or more conditions for joining the conversation;
when the communication system determines that the characteristics of the client device satisfy the one or more conditions for joining the conversation, generating, by the communication system, a summary of the conversation for the additional user based on messages included in the conversation occurring before the determining that the characteristics of the client device satisfy the one or more conditions for joining the conversation, wherein generating the summary includes (a) identifying, by the communication system, for respective ones of the plurality of users participating in the conversation, a respective emotion associated with a respective user, wherein each of the respective emotions is determined based on data captured from the respective users during the conversation at a time when the respective emotion is determined, and (b) identifying, by the communication system, one of the respective emotions as a shared emotion associated with at least a threshold percentage of the respective users, and including, in the summary, an identification of the shared emotion as being associated with at least the threshold percentage of the respective users; and
transmitting the summary from the communication system to the client device for presentation to the additional user via the client device.
2. The method of claim 1 , wherein generating the summary comprises:
identifying messages included in the conversation within a threshold amount of time from a time when the characteristics of the client device were determined to satisfy the one or more conditions for joining the conversation; and
generating the summary from content included in the identified messages.
3. The method of claim 2 , wherein the threshold amount of time is included in a preference maintained by the communication system for the additional user.
4. The method of claim 2 , wherein the threshold amount of time is determined based on a type of the conversation.
5. The method of claim 1 , wherein the determining that the characteristics of the client device satisfy one or more conditions for joining the conversation further comprises:
determining that a conversation identifier included in a request received by the communication system from the client device matches a conversation identifier of the conversation.
6. The method of claim 1 , wherein the summary comprises text data configured to be displayed by a display element of the client device.
7. The method of claim 1 , wherein the summary comprises audio data configured to be played by one or more speakers of the client device.
8. The method of claim 7 , wherein the audio data comprising the summary is configured to be played by the client device at a faster speed than audio data included in messages of the conversation captured after the determining that the characteristics of the client device satisfy one or more conditions for joining the conversation.
9. The method of claim 7 , wherein the audio data comprising the summary is configured to be played by the client device at a higher volume than audio data included in messages of the conversation captured after the determining that the characteristics of the client device satisfy one or more conditions for joining the conversation.
10. The method of claim 1 , wherein generating, by the communication system, the summary comprises:
identifying video data or audio data in the captured data and corresponding to the one user; and
applying one or more models trained to determine emotion to the identified video data or audio data to determine the respective emotion of a respective user.
11. The method of claim 1 , wherein the summary includes (1) identifiers of ones of the respective users having a respective emotions different from the shared emotion and (2) the respective different emotions of the identified respective users.
12. The method of claim 1 , wherein generating, by the communication system, the summary includes the communication system storing (1) a respective identifier of each of the plurality of users for which a respective emotion is determined, (2) the respective determined emotion in association with the respective identifier, and (3) a respective time at which the respective emotion was determined.
13. A computer program product comprising a non-transitory computer-readable storage medium having instructions encoded thereon that, when executed by a processor, causes the processor to:
capture a conversation between a plurality of users at a communication system, the conversation including one or more messages exchanged between the plurality of users that are located within a same room;
determine, from one or more position sensors included in a client device of an additional user other than any of the plurality of users, that a location of the client device is within a threshold distance of a location associated with the conversation, wherein the location of the client device and the location associated with the conversation are within the same room;
determine, by the communication system based at least in part on the location of the client device being within the threshold distance of the location associated with the conversation, that characteristics of the client device satisfy one or more conditions for joining the conversation;
when the communication system determines that the characteristics of the client device satisfy the one or more conditions for joining the conversation, generate, by the communication system, a summary of the conversation for the additional user based on messages included in the conversation occurring before the determining that the characteristics of the client device satisfy the one or more conditions for joining the conversation, wherein generating the summary includes (a) identifying, by the communication system, for respective ones of the plurality of users participating in the conversation, a respective emotion associated with a respective user, wherein each of the respective emotions is determined based on data captured from the respective users during the conversation at a time when the respective emotion is determined, and (b) identifying, by the communication system, one of the respective emotions as a shared emotion associated with at least a threshold percentage of the respective users, and including, in the summary, an identification of the shared emotion as being associated with at least the threshold percentage of the respective users; and
transmitting the summary from the communication system to the client device for presentation to the additional user via the client device.
14. The computer program product of claim 13 , wherein an operation by the communication system to generate the summary comprises:
identifying messages included in the conversation within a threshold amount of time from a time when the characteristics of the client device were determined to satisfy the one or more conditions for joining the conversation; and
generating the summary from content included in the identified messages.
15. The computer program product of claim 14 , wherein the threshold amount of time is included in a preference maintained by the communication system for the additional user.
16. The computer program product of claim 14 , wherein the threshold amount of time is determined based on a type of the conversation.
17. The computer program product of claim 14 , wherein an operation by the communication system to determine that the characteristics of the client device satisfy one or more conditions for joining the conversation further comprises:
determining that a conversation identifier included in a request received by the communication system from the client device matches a conversation identifier of the conversation.
18. The computer program product of claim 13 , wherein the summary comprises text data configured to be displayed by a display element of the client device.
19. The computer program product of claim 13 , wherein the summary comprises audio data configured to be played by one or more speakers of the client device.
20. The computer program product of claim 19 , wherein the audio data comprising the summary is configured to be played by the client device at a faster speed than audio data included in messages of the conversation captured after the determining that the characteristics of the client device satisfies one or more conditions for joining the conversation.
21. The computer program product of claim 19 , wherein the audio data comprising the summary is configured to be played by the client device at a higher volume than audio data included in messages of the conversation captured after the determining that the characteristics of the client device satisfies one or more conditions for joining the conversation.
22. The computer program product of claim 13 , wherein an operation to generate, by the communication system, the summary comprises:
identifying video data or audio data in the captured data and corresponding to the one user; and
applying one or more models trained to determine emotion to the identified video data or audio data to determine the respective emotion of a respective user.
23. The computer program product of claim 1 , wherein the summary includes (1) identifiers of ones of the respective users having respective emotions different from the shared emotion and (2) the respective different emotions of the identified respective users.
24. The computer program product of claim 13 , wherein generating, by the communication system, the summary includes the communication system storing (1) a respective identifier of each of the plurality of users for which a respective emotion is determined, (2) the respective determined emotion in association with the respective identifier, and (3) a respective time at which the respective emotion was determined.
25. The method of claim 1 , wherein the threshold percentage is a majority of the respective users.
26. A computing system comprising:
one or more processors; and
one or more memories storing instructions that, when executed by the one or more processors, cause the computing system to perform a process comprising:
capturing a conversation between a plurality of users at a communication system, the conversation including one or more messages exchanged between the plurality of users that are located within a same room;
determining, from one or more position sensors included in a client device of an additional user other than any of the plurality of users, that a location of the client device is within a threshold distance of a location associated with the conversation, wherein the location of the client device and the location associated with the conversation are within the same room;
determining, by the communication system based at least in part on the location of the client device being within the threshold distance of the location associated with the conversation, that characteristics of the client device satisfy one or more conditions for joining the conversation;
generating, by the communication system, a summary of the conversation for the additional user based on messages included in the conversation occurring before the determining that the characteristics of the client device satisfy the one or more conditions for joining the conversation, wherein generating the summary includes (a) identifying, by the communication system, for respective ones of the plurality of users participating in the conversation, a respective emotion associated with a respective user, wherein each of the respective emotions is determined based on data captured from the respective users during the conversation at a time when the respective emotion is determined, and (b) identifying, by the communication system, one of the respective emotions as a shared emotion associated with at least a threshold percentage of the respective users, and including, in the summary, an identification of the shared emotion as being associated with at least the threshold percentage of the respective users; and
transmitting the summary from the communication system to the client device for presentation to the additional user via the client device.