IP Library Granted Patent US 10,659,399
Granted Patent B2
US 10,659,399 · App. 15/853,189 · Granted May 19, 2020

Message analysis using a machine learning model

Inventors: Jakob Nicolaus Foerster (Oxford, GB); Matthew Sharifi (Kilchberg, CH)
Assignee: Google LLC
H04L51/02G06F17/2705G06F17/276G06N20/00G06Q10/107H04L51/063H04L51/04H04L51/16
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,659,399
App. No.
15/853,189
Granted
May 19, 2020
Kind
B2
Abstract

A method includes receiving a received message and a draft reply message to the received message, the received message including a received message component, the received message component representing one or more of a question, a request, and a subject. The method also includes parsing the received message to detect the received message component and parsing the draft reply message into one or more reply message components, identifying, using one or more machine learning models, informational components associated with the received message component and the one or more reply message components by at least applying the machine learning module to the received message component and each reply message component of the one or more reply message, and identifying, based on the one or more informational components, one or more deficient components in the subject draft reply message, and outputting, for display, information about the one or more deficient components.

Claims (78)

1. A method comprising:

receiving, by at least one processor, a subject received message including a received message component, the received message component representing one or more of a question, a request, and a subject included in content of the subject received message;

receiving, by the at least one processor, an indication of user input composing a subject draft reply message to the subject received message;

parsing, by the at least one processor, the content of the subject received message to identify the received message component;

parsing, by the at least one processor, the subject draft reply message into one or more reply message components;

identifying, by the at least one processor, one or more informational components associated with the received message component and the one or more reply message components by at least applying one or more machine learning models to the received message component and each reply message component of the one or more reply message components;

identifying, by the at least one processor, one or more deficient components from the one or more informational components in the subject draft reply message, wherein each of the one or more deficient components is a respective one of the one or more informational components that is missing or incomplete in the draft reply message; and

outputting, for display, information about the one or more deficient components.

2. The method of claim 1 , wherein identifying one or more informational components further comprises:

determining, by the at least one processor and using a first machine learning model of the one or more machine learning models, a first information type associated with the received message component, the first information type indicating a type of information expected in a response to the received message component; and

determining, by the at least one processor and using a second machine learning model of the one or more machine learning models, a second information type associated with a first draft message component, the second information type indicating a type of information associated with the first draft message component, and

wherein identifying one or more deficient components further comprises:

determining that the first information type does not match the second information type; and

identifying, based on determining that the first information type does not match the second information type, that the received message component is a deficient component of the one or more deficient components.

3. The method of claim 1 , further comprising:

training, by the at least one processor, a first machine learning model of the one or more machine learning models with a plurality of response tuples, each response tuple of the plurality of response tuples includes a question and an associated answer, the first machine learning model being configured to identify an information type associated with an input question, the information type indicating a type of information expected in a response to the input question.

4. The method of claim 1 , further comprising:

training, by at least one processor, a first machine learning model of the one or more machine learning models with a corpus of received messages and associated reply messages, the reply messages identifying one or more missing components from the associated received message not addressed in the reply message, missing components representing one or more of questions, requests, and subjects presented in a received message not addressed in the associated reply message, the first machine learning model being configured to identify missing components in an input draft reply message based on an input received message and the input draft reply message;

wherein identifying one or more informational components further comprises applying the first machine learning model to a first seed subject for each draft reply component of the one or more draft reply components to generate a draft reply component score for each draft reply component, and

wherein identifying one or more deficient components comprises:

determining, for each seed subject, whether any of the draft reply component scores associated with the seed subject satisfies a threshold; and

for each seed subject and responsive to determining that all of the draft reply component scores associated with the corresponding seed subject do not satisfy the threshold, identifying the corresponding seed subject as being one of the one or more deficient components.

5. The method of claim 1 , further comprising:

identifying a suggested reply associated with a first deficient component from the one or more deficient components;

inserting, by the at least one processor, the suggested reply into the subject draft reply message; and

outputting, for display, the updated subject draft reply message including the suggested reply.

6. The method of claim 1 , wherein outputting information about the one or more deficient components comprises highlighting a first seed item associated with a first deficient component of the one or more deficient components within the subject received message.

7. The method of claim 1 , wherein outputting information about the one or more deficient components comprises prompting a user to provide additional information associated with the one or more deficient components.

8. A computing system comprising:

a storage device that stores one or more modules; and

at least one processor that executes the one or more modules to:

receive a subject received message including a received message component, the received message component representing one or more of a question, a request, and a subject presented in the subject received message;

receive an indication of user input composing a subject draft reply message to the subject received message;

parse the subject received message to identify the received message component;

parse the subject draft reply message into one or more reply message components;

identifying one or more informational components associated with the received message component and the one or more reply message components by at least applying one or more machine learning models to the received message component and each reply message component of the one or more reply message components;

identify one or more deficient components from the one or more informational components in the subject draft reply message, wherein each of the one or more deficient components is a respective one of the one or more informational components that is missing or incomplete in the draft reply message; and

output, for display, information about the one or more deficient components.

9. The system of claim 8 , wherein identifying one or more informational components further comprises:

determining, by the at least one processor and using the one or more machine learning models, a first information type associated with the received message component, the first information type indicating a type of information expected in a response to the received message component; and

determining, by the at least one processor and using a second machine learning model of the one or more machine learning models, a second information type associated with a first draft message component, the second information type indicating a type of information associated with the first draft message component,

wherein identifying one or more deficient components further comprises:

determining that the first information type does not match the second information type; and

identifying, based on determining that the first information type does not match the second information type, that the received message component is a deficient component of the one or more deficient components.

10. The system of claim 8 , the at least one processor is further configured to: train a first machine learning model of the one or more machine learning models with a plurality of response tuples, each response tuple of the plurality of response tuples includes a question and an associated answer, the first machine learning model being configured to identify an information type associated with an input question, the information type indicating a type of information expected in a response to the input question.

11. The system of claim 8 , the at least one processor is further configured to: train a first machine learning model of the one or more machine learning models with a corpus of received messages and associated reply messages, the reply messages identifying one or more missing components from the associated received message not addressed in the reply message, missing components representing one or more of questions, requests, and subjects presented in a received message not addressed in the associated reply message, the first machine learning model being configured to identify missing components in an input draft reply message based on an input received message and the input draft reply message;

wherein identifying one or more informational components further comprises applying the first machine learning model to a first seed subject for each draft reply component of the one or more draft reply components to generate a draft reply component score for each draft reply component, and

wherein identifying one or more deficient components comprises:

determining, for each seed subject, whether any of the draft reply component scores associated with the seed subject satisfies a threshold; and

for each seed subject and responsive to determining that all of the draft reply component scores associated with the corresponding seed subject do not satisfy the threshold, identifying the corresponding seed subject as being one of the one or more deficient components.

12. The system of claim 8 , the at least one processor is further configured to:

identify a suggested reply associated with a first deficient component, the suggested reply being identified to address the first deficient component;

insert the suggested reply into the subject draft reply message; and

output, for display, the updated subject draft reply message including the suggested reply.

13. The system of claim 8 , wherein outputting information about the one or more deficient components comprises highlighting a first seed item associated with a first deficient component of the one or more deficient components within the subject received message.

14. The system of claim 8 , wherein outputting information about the one or more deficient components comprises prompting a user to provide additional information associated with the one or more deficient components.

15. A non-transitory computer-readable storage medium having stored thereon instructions that, when executed, cause a processor of a computing system to:

receive a subject received message including a received message component, the received message component representing one or more of a question, a request, and a subject presented in the subject received message;

receive an indication of user input composing a subject draft reply message to the subject received message;

parse the subject received message to identify the received message component;

parse the subject draft reply message into one or more reply message components;

identify one or more informational components associated with the received message component and the one or more reply message components by at least applying one or more machine learning models to the received message component and each reply message component of the one or more reply message components;

identify one or more deficient components from the one or more informational components in the subject draft reply message, wherein each of the one or more deficient components is a respective one of the one or more informational components that is missing or incomplete in the draft reply message; and

output, for display, information about the one or more deficient components.

16. The non-transitory computer-readable storage medium of claim 15 , wherein the instructions further cause the processor to:

determine, using the one or more machine learning models, a first information type associated with the received message component, the first information type indicating a type of information expected in a response to the received message component; and

determine, using a second machine learning model of the one or more machine learning models, a second information type associated with a first draft message component, the second information type indicating a type of information associated with the first draft message component,

wherein the instructions that cause the processor to identify identifying one or more deficient components further include instructions that cause the processor to:

determine that the first information type does not match the second information type; and

identify, based on determining that the first information type does not match the second information type, that the received message component is a deficient component of the one or more deficient components.

17. The non-transitory computer-readable storage medium of claim 15 , the instructions further cause the processor to:

train a first machine learning model of the one or more machine learning models with a plurality of response tuples, each response tuple of the plurality of response tuples includes a question and an associated answer, the first machine learning model being configured to identify an information type associated with an input question, the information type indicating a type of information expected in a response to the input question.

18. The non-transitory computer-readable storage medium of claim 15 , the instructions further cause the processor to:

identify a suggested reply associated with a first deficient component from the one or more deficient components;

insert the suggested reply into the subject draft reply message; and

output, for display, the updated subject draft reply message including the suggested reply.

19. The non-transitory computer-readable storage medium of claim 15 , wherein outputting information about the one or more deficient components comprises highlighting a first seed item associated with a first deficient component of the one or more deficient components within the subject received message.

20. The non-transitory computer-readable storage medium of claim 15 wherein outputting information about the one or more deficient components comprises prompting a user to provide additional information associated with the one or more deficient components.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 22, 2017
From: FOERSTER, JAKOB NICOLAUS; SHARIFI, MATTHEW
To: GOOGLE LLC
Reel/Frame 044473/0611 →
Continuity (1)
Related Publication 20190199656A1 · Jun 27, 2019
Cited By (5)
US 12,282,731 US 12,315,604 US 12,567,482 US 12,695,754 US 12,705,418