METHOD AND APPARATUS FOR IDENTIFYING AN EPISODE IN A MULTI-PARTY MULTIMEDIA COMMUNICATION
An apparatus for identifying an episode in a multi-party multimedia communication includes a processor, and a memory storing instructions that, when executed by the processor, configure the apparatus to perform a method. The method includes receiving multi-modal data including video data and audio data for each of multiple participants. Vision information and at least one of tonal information or text information is used to determine a representative participation score (RPS) for one or more participants. An episode is identified during or proximate to pronounced RPS movement, and sent for display.
1 . A computer implemented method for identifying an episode in a multi-party multimedia communication, the method comprising:
receiving, at an analytics server, from a plurality of multimedia devices in a multimedia communication between a plurality of participants, multi-modal data comprising video data and audio data for at least one of the plurality of participants, wherein the plurality of multimedia devices are remote to the analytics server;
extracting, for each of the plurality of participants, vision information from the video data, at least one of tonal information or text information from the audio data;
determining a representative participation score (RPS) based on the vision information and the at least one of the tonal information or the text information, for each of the plurality of participants, for a plurality of consecutive time intervals;
determining a pronounced RPS movement
for at least one of the plurality of participants during a first plurality of time intervals,
for at least two of the plurality of participants during a second plurality of time intervals, or
for at least a predefined duration for at least one of the plurality of participants during a third plurality of time intervals;
identifying an episode associated with the pronounced RPS movement, the episode defined by at least two speech turns related to a topic; and
sending the episode for display.
2 . The method of claim 1 , wherein each of the at least two speech turns comprise at least one of a common entity or a common intent.
3 . The method of claim 2 , wherein the identifying the episode comprises:
analyzing conversation between the plurality of participants during the pronounced RPS movement; and
analyzing conversation between the plurality of participants in at least one of
a duration or a number of speaker turns before the pronounced RPS movement, or
a duration or a number of speaker turns after the pronounced RPS movement.
4 . The method of claim 3 , wherein the duration of the episode is different from the duration of the pronounced RPS movement.
5 . The method of claim 3 , further comprising:
identifying a hyper-relevant text keyphrase mentioned during the episode; and
sending the identified hyper-relevant text keyphrase for display.
6 . A computing apparatus comprising:
a processor; and
a memory storing instructions that, when executed by the processor, configure the apparatus to:
receive, at an analytics server, from a plurality of multimedia devices in a multimedia communication between a plurality of participants, multi-modal data comprising video data and audio data for at least one of the plurality of participants, wherein the plurality of multimedia devices are remote to the analytics server;
extract, for each of the plurality of participants, vision information from the video data, at least one of tonal information or text information from the audio data;
determine a representative participation score (RPS) based on the vision information and the at least one of the tonal information or the text information, for each of the plurality of participants, for a plurality of consecutive time intervals;
determine a pronounced RPS movement
for at least one of the plurality of participants during a first plurality of time intervals,
for at least two of the plurality of participants during a second plurality of time intervals, or
for at least a predefined duration for at least one of the plurality of participants during a third plurality of time intervals;
identify an episode corresponding to the pronounced RPS movement, the episode defined by at least two speech turns related to a topic; and
send the episode for display.
7 . The computing apparatus of claim 6 , wherein each of the at least two speech turns comprise at least one of a common entity or a common intent.
8 . The computing apparatus of claim 7 , wherein the identifying the episode comprises:
analyze conversation between the plurality of participants during the pronounced RPS movement; and
analyze conversation between the plurality of participants in at least one of
a duration or a number of speaker turns before the pronounced RPS movement, or
a duration or a number of speaker turns after the pronounced RPS movement.
9 . The computing apparatus of claim 8 , wherein the duration of the episode is different from the duration of the pronounced RPS movement.
10 . The computing apparatus of claim 8 , wherein the instructions further configure the apparatus to:
identify a hyper-relevant text keyphrase mentioned during the episode; and
send the identified hyper-relevant text keyphrase for display.
11 . A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to:
receive, at an analytics server, from a plurality of multimedia devices in a multimedia communication between a plurality of participants, multi-modal data comprising video data and audio data for at least one of the plurality of participants, wherein the plurality of multimedia devices are remote to the analytics server;
extract, for each of the plurality of participants, vision information from the video data, at least one of tonal information or text information from the audio data;
determine a representative participation score (RPS) based on the vision information and the at least one of the tonal information or the text information, for each of the plurality of participants, for a plurality of consecutive time intervals;
determine a pronounced RPS movement
for at least one of the plurality of participants during a first plurality of time intervals,
for at least two of the plurality of participants during a second plurality of time intervals, or
for at least a predefined duration for at least one of the plurality of participants during a third plurality of time intervals;
identify an episode corresponding to the pronounced RPS movement, the episode defined by at least two speech turns related to a topic; and
send the episode for display.
12 . The computer-readable storage medium of claim 11 , wherein each of the at least two speech turns comprise at least one of a common entity or a common intent.
13 . The computer-readable storage medium of claim 12 , wherein the identifying the episode comprises:
analyze conversation between the plurality of participants during the pronounced RPS movement; and
analyze conversation between the plurality of participants in at least one of
a duration or a number of speaker turns before the pronounced RPS movement, or
a duration or a number of speaker turns after the pronounced RPS movement.
14 . The computer-readable storage medium of claim 13 , wherein the duration of the episode is different from the duration of the pronounced RPS movement.
15 . The computer-readable storage medium of claim 13 , wherein the instructions further configure the computer to:
identify a hyper-relevant text keyphrase mentioned during the episode; and
send the identified hyper-relevant text keyphrase for display.