IP Library Granted Patent US 11,520,762
Granted Patent B2
US 11,520,762 · App. 16/713,776 · Granted Dec 6, 2022

Performing fine-grained question type classification

Inventors: Sarthak Dash (Jersey City, NJ); Gaetano Rossiello (Brooklyn, NY); Alfio Massimiliano Gliozzo (Brooklyn, NY); Robert G. Farrell (Cornwall, NY); Bassem Makni (Bellevue, WA); Avirup Sil (Hopewell Junction, NY); Vittorio Castelli (Croton-on-Hudson, NY); Radu Florian (Danbury, CT)
Assignee: International Business Machines Corporation
G06F16/2237G06F16/24578G06F16/285G06K9/6256G06N3/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,520,762
App. No.
16/713,776
Granted
Dec 6, 2022
Kind
B2
Abstract

A computer-implemented method according to one embodiment includes converting an input question into a vector form using trained word embeddings; constructing a type similarity matrix using a predetermined ontology; and determining a score for all possible types for the input question, based on the input question in vector form and the type similarity matrix.

Claims (59)

1. A computer-implemented method, comprising:

converting, by a neural network, an input question into a vector form using trained word embeddings;

constructing, by the neural network, a type similarity matrix using a predetermined ontology;

performing, by the neural network, matrix multiplication between the input question in vector form and the type similarity matrix to determine a plurality of possible types for the input question; and

scoring, by the neural network, each of the plurality of possible types for the input question to create a ranked list of possible types for the input question,

wherein the predetermined ontology includes a plurality of labeled nodes representing entities and their associated types,

wherein the labeled nodes within the predetermined ontology are restructured and reformatted into a series of vectors in Euclidean vector space to construct the type similarity matrix,

wherein each vector within the series of vectors conveys a predetermined type within the predetermined ontology,

wherein a location of each vector within the Euclidean vector space is used to determine a similarity score between the associated types of the labeled nodes within the predetermined ontology,

wherein the neural network performs matrix multiplication between the input question in vector form and the type similarity matrix to determine a plurality of possible types for the input question, and

wherein the plurality of possible types for the input question is refined utilizing a list of potential types for training data derived from the predetermined ontology.

2. The computer-implemented method of claim 1 , wherein the input question is converted into a vector by a bidirectional long short-term memory (LSTM) and a combination layer.

3. The computer-implemented method of claim 1 , wherein:

the input question includes a text string, and

the input question is converted into the vector form by a recurrent neural network (RNN).

4. The computer-implemented method of claim 1 , wherein each of a plurality of vectors are created within a Euclidean vector space to represent an associated type within the predetermined ontology, where a proximity of the vectors within the Euclidean vector space is associated with a similarity of the vectors.

5. The computer-implemented method of claim 1 , wherein the type similarity matrix is created utilizing a low rank decomposition method such as SVD on top of a co-occurrence matrix.

6. The computer-implemented method of claim 1 , wherein:

the neural network is trained utilizing a plurality of question/answer pairs, wherein each of the question/answer pairs includes a question string and an associated answer string, and

for each of the plurality of question/answer pairs, a type of the associated answer string is determined from the predetermined ontology and is assigned to the question string as its label to create a list of question/type pairs.

7. The computer-implemented method of claim 1 , wherein:

the input question includes a text string that is converted into the vector form by a recurrent neural network (RNN).

8. The computer-implemented method of claim 1 , further comprising:

encoding a difference between a vector of possible types for the input question as predicted by a system and a list of potential types as provided by training data by utilizing a loss function such as a weighted negative log-likelihood (NLL) operation; and

minimizing the difference during a training phase of a neural network which modifies parameters of the neural network.

9. The computer-implemented method of claim 8 , further comprising back-propagating a loss value for the input question through the neural network.

10. A computer program product comprising one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions comprising instructions configured to cause one or more processors to perform a method comprising:

converting, by a neural network utilizing the one or more processors, an input question into a vector form using trained word embeddings;

constructing, by the neural network utilizing the one or more processors, a type similarity matrix using a predetermined ontology;

performing, by the neural network utilizing the one or more processors, matrix multiplication between the input question in vector form and the type similarity matrix to determine a plurality of possible types for the input question; and

scoring, by the neural network utilizing the one or more processors, each of the plurality of possible types for the input question to create a ranked list of possible types for the input question,

wherein the predetermined ontology includes a plurality of labeled nodes representing entities and their associated types,

wherein the labeled nodes within the predetermined ontology are restructured and reformatted into a series of vectors in Euclidean vector space to construct the type similarity matrix,

wherein each vector within the series of vectors conveys a predetermined type within the predetermined ontology,

wherein a location of each vector within the Euclidean vector space is used to determine a similarity score between the associated types of the labeled nodes within the predetermined ontology,

wherein the neural network performs matrix multiplication between the input question in vector form and the type similarity matrix to determine a plurality of possible types for the input question, and

wherein the plurality of possible types for the input question is refined utilizing a list of potential types for training data derived from the predetermined ontology.

11. The computer program product of claim 10 , wherein the input question is converted into a vector by a bidirectional long short-term memory (LSTM) and a combination layer.

12. The computer program product of claim 10 , wherein:

the input question includes a text string, and

the input question is converted into the vector form by a recurrent neural network (RNN).

13. The computer program product of claim 10 , wherein each of a plurality of vectors are created within a Euclidean vector space to represent an associated type within the predetermined ontology, where a proximity of the vectors within the Euclidean vector space is associated with a similarity of the vectors.

14. The computer program product of claim 10 , wherein the type similarity matrix is created utilizing a low rank decomposition method such as SVD on top of a co-occurrence matrix.

15. The computer program product of claim 12 , wherein:

the neural network is trained utilizing a plurality of question/answer pairs, wherein each of the question/answer pairs includes a question string and an associated answer string, and

for each of the plurality of question/answer pairs, a type of the associated answer string is determined from the predetermined ontology and is assigned to the question string as its label to create a list of question/type pairs.

16. A system, comprising:

a processor; and

logic integrated with the processor, executable by the processor, or integrated with and executable by the processor, the logic being configured to:

convert, by a neural network, an input question into a vector form using trained word embeddings;

construct, by the neural network, a type similarity matrix using a predetermined ontology;

perform, by the neural network, matrix multiplication between the input question in vector form and the type similarity matrix to determine a plurality of possible types for the input question; and

score, by the neural network, each of the plurality of possible types for the input question to create a ranked list of possible types for the input question,

wherein the predetermined ontology includes a plurality of labeled nodes representing entities and their associated types,

wherein the labeled nodes within the predetermined ontology are restructured and reformatted into a series of vectors in Euclidean vector space to construct the type similarity matrix,

wherein each vector within the series of vectors conveys a predetermined type within the predetermined ontology,

wherein a location of each vector within the Euclidean vector space is used to determine a similarity score between the associated types of the labeled nodes within the predetermined ontology,

wherein the neural network performs matrix multiplication between the input question in vector form and the type similarity matrix to determine a plurality of possible types for the input question, and

wherein the plurality of possible types for the input question is refined utilizing a list of potential types for training data derived from the predetermined ontology.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 26, 2020
From: DASH, SARTHAK; ROSSIELLO, GAETANO; GLIOZZO, ALFIO MASSIMILIANO; FARRELL, ROBERT G.; MAKNI, BASSEM; SIL, AVIRUP; CASTELLI, VITTORIO; FLORIAN, RADU
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 051622/0161 →
Continuity (1)
Related Publication 20210182258A1 · Jun 17, 2021
Cited By (2)
US 12,579,365 US 12,639,337