IP Library Granted Patent US 11,630,952
Granted Patent B2
US 11,630,952 · App. 16/518,894 · Granted Apr 18, 2023

Classifying terms from source texts using implicit and explicit class-recognition-machine-learning models

Inventors: Sean MacAvaney (Washington, DC); Franck Dernoncourt (Sunnyvale, CA); Walter Chang (San Jose, CA); Seokhwan Kim (San Jose, CA); Doo Soon Kim (San Jose, CA); Chen Fang (Hanover, NH)
Assignee: Adobe Inc.
G06F40/279G06F17/15G06F17/16G06F18/2431G06N3/045G06V10/764G06V10/768G06V10/809G06V10/82
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,630,952
App. No.
16/518,894
Granted
Apr 18, 2023
Kind
B2
Abstract

This disclosure relates to methods, non-transitory computer readable media, and systems that can classify term sequences within a source text based on textual features analyzed by both an implicit-class-recognition model and an explicit-class-recognition model. For example, by applying machine-learning models for both implicit and explicit class recognition, the disclosed systems can determine a class corresponding to a particular term sequence within a source text and identify the particular term sequence reflecting the class. The dual-model architecture can equip the disclosed systems to apply (i) the implicit-class-recognition model to recognize implicit references to a class in source texts and (ii) the explicit-class-recognition model to recognize explicit references to the same class in source texts.

Claims (71)

1. A non-transitory computer readable medium storing instructions thereon that, when executed by at least one processor, cause the at least one processor to:

generate a first plurality of n-gram class scores corresponding to a plurality of classes for term sequences from a source text utilizing an implicit-class-recognition model;

generate a second plurality of n-gram class scores corresponding to the plurality of classes for the term sequences from the source text utilizing an explicit-class-recognition model,

wherein the first plurality of n-gram class scores and the second plurality of n-gram class scores indicate measures of likelihood that the term sequences from the source text correspond to the plurality of classes; and

determine a class from the plurality of classes corresponding to the source text and a term sequence from the source text reflecting the class based on the first plurality of n-gram class scores and the second plurality of n-gram class scores.

2. The non-transitory computer readable medium of claim 1 , wherein the instructions, when executed by the at least one processor, cause the at least one processor to:

generate the first plurality of n-gram class scores by generating a first set of unigram class scores, a first set of bigram class scores, and a first set of trigram class scores utilizing the implicit-class-recognition model; and

generate the second plurality of n-gram class scores by generating a second set of unigram class scores, a second set of bigram class scores, and a second set of trigram class scores utilizing the explicit-class-recognition model.

3. The non-transitory computer readable medium of claim 1 , wherein the instructions, when executed by the at least one processor, cause the at least one processor to:

apply a max pooling layer to the first plurality of n-gram class scores and the second plurality of n-gram class scores to generate consolidated-class scores for the term sequences; and

identify the class and the term sequence based on the consolidated-class scores.

4. The non-transitory computer readable medium of claim 1 , wherein the instructions, when executed by the at least one processor, cause the at least one processor to generate the first plurality of n-gram class scores for the term sequences from the source text utilizing the implicit-class-recognition model by:

generating a plurality of feature vectors based on terms from the source text utilizing a plurality of long-short-term memory (“LSTM”) layers from the implicit-class-recognition model; and

generating the first plurality of n-gram class scores for the term sequences based on the plurality of feature vectors utilizing a convolutional neural network from the implicit-class-recognition model.

5. The non-transitory computer readable medium of claim 4 , wherein the instructions, when executed by the at least one processor, cause the at least one processor to:

generate the plurality of feature vectors by:

generating a first feature vector for a first term embedding from a first term of the source text utilizing the plurality of LSTM layers; and

generating a second feature vector for a second term embedding from a second term of the source text utilizing the plurality of LSTM layers; and

generate the first plurality of n-gram class scores for the term sequences by utilizing the convolutional neural network to generate a first unigram class score based on the first feature vector, generate a second unigram class score based on the second feature vector, and generate a bigram class score based on the first feature vector and the second feature vector.

6. The non-transitory computer readable medium of claim 1 , wherein the instructions, when executed by the at least one processor, cause the at least one processor to generate the second plurality of n-gram class scores for the term sequences from the source text utilizing the explicit-class-recognition model by:

generating similarity matrices based on terms from the source text and a plurality of labels corresponding to a plurality of classes; and

analyzing the similarity matrices utilizing a convolutional neural network of the explicit-class-recognition model to generate the second plurality of n-gram class scores.

7. The non-transitory computer readable medium of claim 6 , wherein the instructions, when executed by the at least one processor, cause the at least one processor to generate the similarity matrices by:

generating a first similarity matrix comprising similarity scores between source-term-feature vectors for the terms from the source text and label-feature vectors for a first label corresponding to a first class; and

generating a second similarity matrix comprising similarity scores between the source-term-feature vectors for the terms from the source text and label-feature vectors for a second label corresponding to a second class.

8. The non-transitory computer readable medium of claim 1 , wherein the instructions, when executed by the at least one processor, cause the at least one processor to provide, for display within a graphical user interface of a computing device, the source text and a visual indicator identifying the term sequence in the source text as corresponding to the class.

9. The non-transitory computer readable medium of claim 1 , wherein:

the term sequences from the source text comprise unigrams, bigrams, and trigrams; and

the term sequence comprises one of a unigram, a bigram, or a trigram.

10. A system comprising:

at least one processor; and

at least one non-transitory computer readable medium comprising instructions that, when executed by the at least one processor, cause the system to:

generate a first plurality of class scores for term sequences from a source text utilizing an implicit-class-recognition model by:

generating a plurality of feature vectors based on terms from the source text utilizing a plurality of LSTM layers; and

generating the first plurality of class scores for the term sequences based on the plurality of feature vectors utilizing a first convolutional neural network;

generate a second plurality of class scores for the term sequences from the source text utilizing an explicit-class-recognition model by:

generating similarity matrices based on the terms from the source text and a plurality of labels corresponding to the plurality of classes; and

analyzing the similarity matrices utilizing a second convolutional neural network to generate the second plurality of class scores; and

determine a class corresponding to a term sequence from the source text reflecting the class based on the first plurality of class scores and the second plurality of class scores.

11. The system of claim 10 , further comprising instructions that, when executed by the at least one processor, cause the system to:

generate the first plurality of class scores by generating a first set of unigram class scores, a first set of bigram class scores, and a first set of trigram class scores utilizing the implicit-class-recognition model; and

generate the second plurality of class scores by generating a second set of unigram class scores, a second set of bigram class scores, and a second set of trigram class scores utilizing the explicit-class-recognition model.

12. The system of claim 10 , further comprising instructions that, when executed by the at least one processor, cause the system to:

generate the plurality of feature vectors by:

generating a first feature vector for a first term embedding from a first term of the source text utilizing the plurality of LSTM layers; and

generating a second feature vector for a second term embedding from a second term of the source text utilizing the plurality of LSTM layers; and

generate the first plurality of class scores for the term sequences by utilizing the first convolutional neural network to generate a first unigram class score based on the first feature vector, generate a second unigram class score based on the second feature vector, and generate a bigram class score based on the first feature vector and the second feature vector.

13. The system of claim 10 , further comprising instructions that, when executed by the at least one processor, cause the system to generate the similarity matrices by:

generating a first similarity matrix comprising similarity scores between source-term-feature vectors for the terms from the source text and label-feature vectors for a first label corresponding to a first class; and

generating a second similarity matrix comprising similarity scores between the source-term-feature vectors for the terms from the source text and label-feature vectors for a second label corresponding to a second class.

14. The system of claim 10 , further comprising instructions that, when executed by the at least one processor, cause the system to generate the second plurality of class scores for the term sequences from the source text utilizing the explicit-class-recognition model by:

generating term frequency measures for the terms within the source text and within the plurality of labels; and

analyzing the term frequency measures and the similarity matrices utilizing the second convolutional neural network to generate the second plurality of class scores.

15. The system of claim 10 , further comprising instructions that, when executed by the at least one processor, cause the system to apply a max pooling layer to the first plurality of class scores and the second plurality of class scores to generate consolidated-class scores for the term sequences.

16. The system of claim 15 , further comprising instructions that, when executed by the at least one processor, cause the system to train the implicit-class-recognition model and the explicit-class-recognition model by:

applying an additional max pooling layer to the consolidated-class scores to determine the class; and

comparing the class to a ground-truth class to modify internal parameters of the implicit-class-recognition model and the explicit-class-recognition model.

17. The system of claim 10 , further comprising instructions that, when executed by the at least one processor, cause the system to provide, for display within a graphical user interface of a computing device, the source text and a visual indicator identifying the term sequence in the source text as corresponding to the class.

18. The system of claim 10 , wherein:

the term sequences from the source text comprise unigrams, bigrams, and trigrams; and

the term sequence comprises one of a unigram, a bigram, or a trigram.

19. A computer-implemented method comprising:

identifying a source text from a digital document and a plurality of classes;

generating a first plurality of n-gram class scores corresponding to a plurality of classes for term sequences from a source text utilizing an implicit-class-recognition model;

generating a second plurality of n-gram class scores corresponding to the plurality of classes for the term sequences from the source text utilizing an explicit-class-recognition model,

wherein the first plurality of n-gram class scores and the second plurality of n-gram class scores indicate measures of likelihood that the term sequences from the source text correspond to the plurality of classes;

determining a class from the plurality of classes corresponding to the source text and a term sequence from the source text reflecting the class based on the first plurality of n-gram class scores and the second plurality of n-gram class scores; and

providing the digital document for display within a graphical user interface comprising the source text and an indication of the term sequence corresponding to the class within the source text.

20. The method of claim 19 , wherein:

the implicit-class-recognition model comprises a plurality of long-short-term memory layers and a first convolutional neural network; and

the explicit-class-recognition model comprises a second convolutional neural network for multiple channels.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 22, 2019
From: MACAVANEY, SEAN; DERNONCOURT, FRANCK; CHANG, WALTER; KIM, SEOKHWAN; KIM, DOO SOON; FANG, CHEN
To: ADOBE INC.
Reel/Frame 049824/0945 →
Continuity (1)
Related Publication 20210027141A1 · Jan 28, 2021
Cited By (1)
US 12,254,049