Systems and methods for providing real time assistance to voice over internet protocol (VOIP) users
Systems and methods are provided that may be implemented in a real time manner by an information handling system (the “client system”) to monitor one or more characteristics of a voice over internet protocol (VOIP) discussion, to use these monitored VOIP characteristics to identify one or more condition/s in real time as they are identified to occur during the current VOIP discussion, and to determine to take one or more automatic actions based on the identified VOIP condition/s so as to inform and/or alert a current human user of the client system to the occurrence of the identified VOIP condition/s as they occur.
1. A method, comprising:
providing in a first information handling system a voice over internet protocol (VOIP) audio output stream including incoming VOIP speech data of two or more separate and different VOIP human speakers received in the first information handling system from across a network, the two or more separate and different VOIP human speakers being respective different human users of two or more separate and different information handling systems that are different from the first information handling system and that are coupled across the network to the first information handling system;
separating the VOIP audio output stream into two or more separate and different VOIP speaker streams that each correspond to a respective separate and different one of the two or more VOIP human speakers, and transcribing each separate and different VOIP speaker stream to produce a corresponding separate and different speaker text channel for each given VOIP speaker stream that includes the one or more VOIP text strings spoken by the respective separate and different one of the two or more VOIP human speakers that corresponds to the given VOIP speaker stream;
comparing each of the separate and different speaker text channels in real time to one or more predefined keywords to identify a predefined keyword present within a current text string of one of the separate and different speaker text channels spoken by a first one of the two or more separate and different VOIP human speakers;
determining a predicted sequential correlation between the current text string containing the predefined keyword that is spoken by the first one of the two or more separate and different VOIP human speakers and one or more other text strings previously spoken by any of the two or more separate and different VOIP human speakers;
determining a current context of a human user of the first information handling system, the human user of the first information handling system being a separate and different human user from the two or more separate and different VOIP human speakers; and
selecting a predefined modality corresponding to the current context of the human user of the first information handling system, and taking an action corresponding to the determined modality to alert the human user of the first information handling system to the occurrence of the predicted sequential correlation between the current text string spoken by the first one of the two or more separate and different VOIP human speakers and the one or more other text strings previously spoken by any of the two or more separate and different VOIP human speakers;
where each of the respective different users of the two or more separate and different information handling systems are participating as one of the respective two or more VOIP human speakers with a human user of the first information handling system in a current VOIP discussion session held across the network between the first information handling system and the two or more separate and different information handling systems; and
where the respective incoming VOIP speech data of each given one of the two or more separate and different VOIP human speakers includes one or more VOIP text strings spoken during the current VOIP discussion session by the given VOIP human speaker, each of the two or more separate and different information handling systems providing the VOIP speech data of its corresponding human user to the current VOIP discussion session.
2. The method of claim 1 , further comprising maintaining a sliding window buffer containing a buffer portion of the previously spoken separate and different speaker text channels; and determining the predicted sequential correlation between the current text string containing the predefined keyword and the one or more other text strings previously spoken by any of the two or more separate and different VOIP human speakers by sequentially correlating the buffer portion of previously spoken separate and different speaker text channels to the current text string containing the predefined keyword.
3. The method of claim 2 , where the step of sequentially correlating the buffer portion of previously spoken separate and different speaker text channels to the current text string containing the predefined keyword comprises using semantic analysis to sequentially correlate the other one or more previously spoken text strings to the current text string containing the predefined keyword.
4. The method of claim 3 , where the step of using semantic analysis to sequentially correlate the other one or more previously spoken text strings to the current text string containing the predefined keyword comprises:
assigning a time stamp, semantic coefficient and speaker label to each of the other one or more previously spoken text strings;
determining a weight for each of the previously spoken text strings using a combination of time stamp, semantic coefficient and speaker label assigned to each of the other one or more previously spoken text strings; and
sequentially correlating the other one or more previously spoken text strings to the current text string containing the predefined keyword based on the determined weight of each of the previously spoken text strings.
5. The method of claim 2 , where the step of sequentially correlating the buffer portion of previously spoken separate speaker text channels to the current text string containing the predefined keyword comprises comparing each of the previously spoken separate speaker text channels in the buffer to one or more pre-defined main words tuples to identify at least one text string in the buffer portion of previously spoken separate speaker text channels that corresponds to one of the pre-defined main words tuples, and determining a predicted sequential correlation between the at least one text string containing the pre-defined main words tuples and the current text string containing the predefined keyword.
6. The method of claim 1 , where the determining the current context of the human user of the first information handling system comprises determining an identity of an active user application that is currently in focus on the user desktop of the first information handling system and that is different from a current VOIP application hosting the current VOIP discussion session on the first information handling system; and where the taking the action corresponding to the determined modality comprises one of:
automatically displaying an alert to the user superimposed or over the displayed window of the determined current in-focus active application on a display device of the first information handling system only upon determining the identity of the active user application that is currently in focus on the user desktop of the first information handling system and that is different from a current VOIP application hosting the current VOIP discussion session on the first information handling system; or
automatically changing the current in-focus application on the first information handling system to the current VOIP application from the determined current in-focus active application only upon determining the identity of the active user application that is currently in focus on the user desktop of the first information handling system and that is different from a current VOIP application hosting the current VOIP discussion session on the first information handling system.
7. The method of claim 1 , where the determining current context of the human user of the first information handling system comprises determining that a monitored current activity of the human user is currently occurring on a device that is different than the first information handling system; and where the taking the action corresponding to the determined modality comprises automatically providing a visual and/or audio alert to the user on the other device that is different from the first information handling system only upon the determining that the current context of the human user of the first information handling system comprises a monitored current activity of the human user of the first information handling system that is currently occurring on the other device that is different than the first information handling system.
8. The method of claim 1 , further comprising receiving and storing the predefined keyword corresponding to a current text string spoken by a first one of the separate and different VOIP human speakers prior to providing in the first information handling system the VOIP audio output stream including incoming VOIP speech data of the two or more separate and different VOIP human speakers received from across the network.
9. The method of claim 1 , further comprising receiving and storing the predefined modality corresponding to the current context of the human user of the first information handling system from the human user of the first information handling system prior to providing in the first information handling system the VOIP audio output stream including incoming VOIP speech data of two or more separate and different VOIP human speakers received from across the network.
10. The method of claim 9 , further comprising receiving and storing the predefined modality corresponding to the current context of the human user of the first information handling system prior to providing in the first information handling system the VOIP audio output stream including incoming VOIP speech data of the two or more separate and different VOIP human speakers received from across the network.
11. A system, comprising a first information handling system coupled to a network and comprising at least one programmable integrated circuit receiving a voice over internet protocol (VOIP) audio output stream including incoming VOIP speech data of two or more separate and different VOIP human speakers received from across the network, the two or more separate and different VOIP human speakers being respective different human users of two or more separate and different information handling systems that are different from the first information handling system and that are coupled across the network to the first information handling system; where the at least one programmable integrated circuit is programmed to:
separate the VOIP audio output stream into two or more separate and different VOIP speaker streams that each correspond to a respective separate and different one of the two or more VOIP human speakers, and transcribe each separate and different VOIP speaker stream to produce a corresponding separate speaker and different text channel that includes the one or more VOIP text strings spoken by the respective separate and different one of the two or more VOIP human speakers that corresponds to the given VOIP speaker stream;
compare each of the separate and different speaker text channels in real time to one or more predefined keywords to identify a predefined keyword present within a current text string of one of the separate and different speaker text channels spoken by a first one of the two or more separate and different VOIP human speakers;
determine a predicted sequential correlation between the current text string containing the predefined keyword that is spoken by the first one of the two or more separate and different VOIP human speakers and one or more other text strings previously spoken by any of the two or more separate and different VOIP human speakers;
determine a current context of a human user of the first information handling system, the human user of the first information handling system being a separate and different human user from the two or more separate and different VOIP human speakers;
select a predefined modality corresponding to the current context of the human user of the first information handling system; and
take an action corresponding to the determined modality to alert the human user of the first information handling system to the occurrence of the predicted sequential correlation between the current text string spoken by the first one of the two or more separate and different VOIP human speakers and the one or more other text strings previously spoken by any of the two or more separate and different VOIP human speakers;
where each of the respective different users of the two or more separate and different information handling systems are participating as one of the respective two or more VOIP human speakers with a human user of the first information handling system in a current VOIP discussion session held across the network between the first information handling system and the two or more separate and different information handling systems; and
where the respective incoming VOIP speech data of each given one of the two or more separate and different VOIP human speakers includes one or more VOIP text strings spoken during the current VOIP discussion session by the given VOIP human speaker, each of the two or more separate and different information handling systems providing the VOIP speech data of its corresponding human user to the current VOIP discussion session.
12. The system of claim 11 , where the at least one programmable integrated circuit is further programmed to:
maintain a sliding window buffer containing a buffer portion of the previously spoken separate and different speaker text channels; and
determine the predicted sequential correlation between the current text string containing the predefined keyword and the one or more other text strings previously spoken by any of the two or more separate and different VOIP human speakers by sequentially correlating the buffer portion of previously spoken separate and different speaker text channels to the current text string containing the predefined keyword.
13. The system of claim 12 , where the at least one programmable integrated circuit is further programmed to sequentially correlate the buffer portion of previously spoken separate and different speaker text channels to the current text string containing the predefined keyword using semantic analysis to sequentially correlate the other one or more previously spoken text strings to the current text string containing the predefined keyword.
14. The system of claim 13 , where the at least one programmable integrated circuit is further programmed to use semantic analysis to sequentially correlate the other one or more previously spoken text strings to the current text string containing the predefined keyword by:
assigning a time stamp, semantic coefficient and speaker label to each of the other one or more previously spoken text strings;
determining a weight for each of the previously spoken text strings using a combination of time stamp, semantic coefficient and speaker label assigned to each of the other one or more previously spoken text strings; and
sequentially correlating the other one or more previously spoken text strings to the current text string containing the predefined keyword based on the determined weight of each of the previously spoken text strings.
15. The system of claim 12 , where the at least one programmable integrated circuit is further programmed to sequentially correlate the buffer portion of previously spoken separate speaker text channels to the current text string containing the predefined keyword by:
comparing each of the previously spoken separate speaker text channels in the buffer to one or more pre-defined main words tuples to identify at least one text string in the buffer portion of previously spoken separate speaker text channels that corresponds to one of the pre-defined main words tuples; and
determining a sequential correlation between the at least one text string containing the pre-defined main words tuples and he current text string containing the predefined keyword.
16. The system of claim 15 , further comprising a second information handling system comprising a programmable integrated circuit and being coupled to the first information handling system across the network, the programmable integrated circuit of the second information handling system being programmed to:
receive and store in a backend database of the second information handling system the pre-defined main words tuples received across the network from the human user of the first information handling system; and
then provide from the backend database the pre-defined main words tuples across the network to the first information handling system.
17. The system of claim 11 , where the at least one programmable integrated circuit is programmed to:
determine the current context of the human user of the first information handling system by determining an identity of an active user application that is currently in focus on the user desktop of the first information handling system and that is different from a current VOIP application hosting the current VOIP discussion session on the first information handling system; and
where the action taken corresponding to the determined modality comprises one of:
automatically displaying an alert to the user superimposed or over the displayed window of the determined current in-focus active application on a display device of the first information handling system only upon determining the identity of the active user application that is currently in focus on the user desktop of the first information handling system and that is different from a current VOIP application hosting the current VOIP discussion session on the first information handling system, or
automatically changing the current in-focus application on the first information handling system to the current VOIP application from the determined current in-focus active application only upon the determining the identity of the active user application that is currently in focus on the user desktop of the first information handling system and that is different from a current VOIP application hosting the current VOIP discussion session on the first information handling system.
18. The system of claim 11 , where the at least one programmable integrated circuit is programmed to:
determine the current context of the human user of the first information handling system by determining that a monitored current activity of the human user is currently occurring on a device that is different than the first information handling system; and
where the action taken corresponding to the determined modality comprises automatically providing a visual and/or audio alert to the user on the other device that is different from the first information handling system only upon the determining that the current context of the human user of the first information handling system comprises a monitored current activity of the human user of the first information handling system that is currently occurring on the other device that is different than the first information handling system.
19. The system of claim 11 , further comprising a second information handling system comprising a programmable integrated circuit and being coupled to the first information handling system across the network, the programmable integrated circuit of the second information handling system being programmed to:
receive and store in a backend database of the second information handling system the predefined keyword corresponding to a current text string spoken by a first one of the two or more separate and different VOIP human speakers across the network from the human user of the first information handling system; and
then provide from the backend database the predefined keyword corresponding to the current text string spoken by the first one of the two or more separate and different VOIP human speakers across the network to the first information handling system.
20. The system of claim 11 , further comprising a second information handling system comprising a programmable integrated circuit and being coupled to the first information handling system across the network, the programmable integrated circuit of the second information handling system being programmed to:
receive and store in a backend database of the second information handling system the predefined modality corresponding to the current context of the human user of the first information handling system from the human user of the first information handling system; and
then provide from the backend database of the second information handling system the predefined modality corresponding to the current context of the human user of the first information handling system across the network to the first information handling system.