Generating summary and next actions in real-time for multiple users from interaction records in natural language
A system receives messaging, video and/or audio input streams including dialogue spoken by users at a group meeting. From these inputs, the system obtains single or multiple interaction records including natural language text memorializing content spoken by each speaker at a meeting, analyzes the content, and identifies single or multiple action item tasks in the interaction records. The system then generates summaries indicating the action item tasks for the users. From the dialogue content, the system further detects whether each action item is addressed, and whether the action item for a user has a solution, or not. The system further detects whether one action item is a precondition for resolving another action item by the user or in conjunction with another user. Using a pre-configured template, the system generates action item summaries, any associated solution, and any relationship or precondition between action items and presents the summary to a user.
1. A computer-implemented method comprising:
receiving, at one or more processors, multiple interaction records of natural language text relating to a group of users attending a meeting;
analyzing, using one or more classifier models, said text of said multiple interaction records to identify one or more action items from said text, an action item relating to a respective task to be performed;
detecting, from said interaction records, a respective user assigned to perform the respective task;
detecting, from said interaction records, whether each action item from the identified one or more action items is addressed with a solution for carrying out said task or not, said detecting, from said interaction records, whether each action item from the identified one or more action items is addressed with a solution for carrying out said task comprising:
converting a detected action item into a question form;
forming a question-answering task; and
running a trained question-answering model for extracting spans from said interaction records, and determining whether an answer to the question exists from an extracted span;
identifying, using a natural language processing (NLP) entailment model, whether any action item from the identified one or more action items is dependent upon another action item from the identified one or more action items, or whether an action item relating to a task is a precondition to performing another action item task from the identified one or more action items; and
generating a text summary of a solution that addresses each action item for the user based upon the identifying and the determining that the answer to the question exists from the extracted span.
2. The computer-implemented method as claimed in claim 1 , further comprising:
receiving a real-time audio signal stream or audio/video signal stream capturing users spoken content at said meeting; and
converting said real-time audio signal stream or audio/video signal stream into a textual representation to form said multiple interaction records.
3. The computer-implemented method as claimed in claim 1 , further comprising:
receiving, at said one or more processors, one of: an activation signal to initiate said receiving of interaction records, said analyzing, said action item detecting and said summary generating, at a beginning of or during said meeting, or a deactivation signal to terminate said receiving of said interaction records, said analyzing, said action item detecting and said summary generating during said meeting.
4. The computer-implemented method as claimed in claim 1 , wherein said generating a summary of a solution that addresses each action item for the user further comprises:
obtaining a template having pre-defined fields for receiving information related to said action items detected in said interaction records; and
populating said template with action item information including a corresponding user owning that action item, and for each action item: specifying in said template information comprising: the solution to said action item and/or any precondition action item upon which an action item depends.
5. The computer-implemented method as claimed in claim 4 , further comprising:
receiving, from a user, based on said populated template summary information, a feedback information relating to an action item task, its detected corresponding user, a solution to said action item; and
using said feedback information to refine tuning one or more of: said one or more classifier models, the trained question-answering model, and said NLP entailment model.
6. A computer-implemented system comprising:
a memory storage device for storing a computer-readable program, and at least one processor adapted to run said computer-readable program to configure the at least one processor to:
receive multiple interaction records of natural language text relating to a group of users attending a meeting;
analyze, using one or more classifier models, said text of said multiple interaction records to identify one or more action items from said text, an action item relating to a respective task to be performed;
detect, from said interaction records, a respective user assigned to perform the respective task;
detect, from said interaction records, whether each action item from the identified one or more action items is addressed with a solution for carrying out said task or not, wherein to detect, from said interaction records, whether each action item from the identified one or more action items is addressed with a solution for carrying out said task, said at least one processor is further configured to:
convert a detected action item into a question form;
form a question-answering task; and
run a trained question-answering model for extracting spans from said interaction records, and determine whether an answer to the question exists from an extracted span;
identify, using a natural language processing (NLP) entailment model, whether any action item is dependent upon another action item or whether an action item relating to a task is a precondition to performing another action item task; and
generate a text summary of a solution that addresses each action item for the user based upon the identifying and the determining that the answer to the question exists from the extracted span.
7. The computer-implemented system as claimed in claim 6 , wherein said at least one processor is further configured to:
receive a real-time audio signal stream or audio/video signal stream capturing users spoken content at said meeting; and
convert said real-time audio signal stream or audio/video signal stream into a textual representation to form said multiple interaction records.
8. The computer-implemented system as claimed in claim 6 , wherein said at least one processor is further configured to:
receive one of: an activation signal to initiate said receiving of interaction records, said analyzing, said action item detecting and said summary generating, at a beginning of or during said meeting, or a deactivation signal to terminate said receiving of said interaction records, said analyzing, said action item detecting and said summary generating during said meeting.
9. The computer-implemented system as claimed in claim 6 , wherein to generate a summary of a solution that addresses each action item for the user, said at least one processor is further configured to:
obtain a template having pre-defined fields for receiving information related to said action items detected in said interaction records; and
populate said template with action item information including a corresponding user owning that action item, and for each action item: specifying in said template information comprising: the solution to said action item and/or any precondition action item upon which an action item depends.
10. The computer-implemented system as claimed in claim 9 , wherein said at least one processor is further configured to:
receive, from a user, based on said populated template summary information, a feedback information relating to an action item task, its detected corresponding user, a solution to said action item; and
use said feedback information to refine tuning of one or more of: said one or more classifier models, the trained question-answering model, and said NLP entailment model.
11. A computer program product, the computer program product comprising a computer-readable storage medium having a computer-readable program stored therein, wherein the computer-readable program, when executed on a computer including at least one processor, causes the at least one processor to:
receive multiple interaction records of natural language text relating to a group of users attending a meeting;
analyze, using one or more classifier models, said text of said multiple interaction records to identify one or more action items from said text, an action item relating to a respective task to be performed;
detect, from said interaction records, a respective user assigned to perform the respective task;
detect, from said interaction records, whether each action item from the identified one or more action items is addressed with a solution for carrying out said task or not, wherein to detect, from said interaction records, whether each action item from the identified one or more action items is addressed with a solution for carrying out said task, the computer-readable program further configuring the at least one processor to:
convert a detected action item into a question form;
form a question-answering task; and
run a trained question-answering model for extracting spans from said interaction records, and determine whether an answer to the question exists from an extracted span;
identify, using a natural language processing (NLP) entailment model, whether any action item is dependent upon another action item or whether an action item relating to a task is a precondition to performing another action item task; and
generate a text summary of a solution that addresses each action item for the user based upon the identifying and the determining that the answer to the question exists from the extracted span.
12. The computer program product as claimed in claim 11 , wherein the computer-readable program further configures the at least one processor to:
receive one of: an activation signal to initiate said receiving of interaction records, said analyzing, said action item detecting and said summary generating, at a beginning of or during said meeting, or a deactivation signal to terminate said receiving of said interaction records, said analyzing, said action item detecting and said summary generating during said meeting.
13. The computer program product as claimed in claim 11 , wherein to generate a summary of a solution that addresses each action item for the user, the computer-readable program further configures the at least one processor to:
obtain a template having pre-defined fields for receiving information related to said action items detected in said interaction records; and
populate said template with action item information including a corresponding user owning that action item, and for each action item: specifying in said template information comprising: the solution to said action item and/or any precondition action item upon which an action item depends.
14. The computer program product as claimed in claim 13 , wherein the computer-readable program further configures the at least one processor to:
receive, from a user, based on said populated template summary information, a feedback information relating to an action item task, its detected corresponding user, a solution to said action item; and
use said feedback information to refine tuning of one or more of: said one or more classifier models, the trained question-answering model, and said NLP entailment model, use the explanation or reason as a recommendation to prospectively change a condition of an existing policy in anticipation of a future occurring event.