System and method of identifying a variation of a phrase in a textual passage
A system and method of identifying occurrence of a semantic variation of a phrase in a passage by at least one processor may include calculating a phrase embedding vector, representing a semantic meaning of the phrase; extracting, from a textual representation of the passage, at least one hierarchical set of nested sequences of words; for each sequence, calculating a corresponding sequence embedding vector, representing a semantic meaning of the sequence; for one or more sequence embedding vectors, calculating a corresponding vector similarity value, representing similarity of the sequence embedding vectors to the phrase embedding vector, identifying a sequence corresponding to a maximal vector similarity value of the one or more vector similarity values; and determining the identified sequence as a semantic variation of the phrase, based on the maximal vector similarity value.
1 . A method of identifying occurrence of a semantic variation of a phrase in a passage by at least one processor, the method comprising:
based on a textual representation of the phrase, calculating a phrase embedding vector, representing a semantic meaning of the phrase;
obtaining a textual representation of the passage, comprising a plurality of words;
extracting, from the textual representation of the passage, at least one hierarchical set of nested sequences of words, wherein each sequence of a hierarchical set forms a subset of words of a subsequent sequence in the hierarchical set;
for each of the at least one hierarchical set of nested sequences, calculating a corresponding sequence embedding vector, representing a semantic meaning of the hierarchical set of nested sequences;
for one or more sequence embedding vectors, calculating a corresponding vector similarity value, representing similarity with the phrase embedding vector;
identifying a sequence corresponding to a maximal vector similarity value of the one or more vector similarity values; and
determining the identified sequence as a semantic variation of the phrase, based on the maximal vector similarity value.
2 . The method of claim 1 , wherein the at least one hierarchical set comprises a plurality of hierarchical sets.
3 . The method of claim 2 , wherein extracting the plurality of hierarchical sets comprises:
selecting a plurality of kernel sequences, each comprising one or more words of the passage; and
for each kernel sequence, producing a respective hierarchical set of nested sequences, each comprising the kernel sequence and subsequent, monotonically increasing, nesting sequences of words.
4 . The method of claim 3 , wherein the textual representation of the passage is a transcript of a conversation, and wherein selecting the plurality of kernel sequences comprises:
selecting a section of the transcript, associated with a specific speaker in the conversation; and
selecting the plurality of kernel sequences to comprise one or more words of the selected section.
5 . The method of claim 3 , wherein selecting the plurality of kernel sequences comprises:
calculating, for one or more words of the passage, one or more respective labels representing parts of speech (POS); and
selecting the plurality of kernel sequences to comprise one or more words of the passage based on the calculated one or more respective labels representing POS.
6 . The method of claim 3 , wherein selecting the plurality of kernel sequences comprises:
calculating, for one or more words of the passage, one or more respective metrics of term relevance; and
selecting the plurality of kernel sequences to comprise one or more words of the passage based on the calculated one or more respective metrics of term relevance.
7 . The method of claim 1 , wherein calculating a corresponding sequence embedding vector of a specific sequence comprises:
obtaining a machine-learning (ML) based model, pretrained to map between textual representations of words and corresponding word embedding vectors;
inferring the ML based model on one or more words of the specific sequence, to produce one or more corresponding word embedding vectors, based on said pre-training; and
calculating the corresponding sequence embedding vector of the specific sequence as a function of the one or more corresponding word embedding vectors.
8 . The method of claim 7 , wherein training of the obtained ML based model comprises:
inferring the ML based model on a first word, to produce a first interim word embedding vector, representing a semantic meaning of the first word;
inferring the ML based model on a second word, to produce a second interim word embedding vector, representing a semantic meaning of the second word;
receiving a first annotation data element, representing semantic similarity between the first word and the second word;
calculating a vector similarity value, representing similarity between the first interim word embedding vector and the second interim word embedding vector;
calculating a first loss function value, representing a difference between the vector similarity value and the semantic similarity, as represented by the first annotation data element; and
training the ML based model so as to minimize the first loss function value.
9 . The method of claim 7 , wherein training of the obtained ML based model comprises:
inferring the ML based model on a first sequence, to produce a first interim sequence embedding vector;
inferring the ML based model on a second sequence, to produce a second interim sequence embedding vector;
receiving a second annotation data element, representing sequence semantic similarity between the first sequence and the second sequence;
calculating a second vector similarity value, representing similarity between the first interim sequence embedding vector and the second interim sequence embedding vector;
calculating a second loss function value, representing a difference between the second vector similarity value and the sequence semantic similarity, as represented by the second annotation data element; and
training the ML based model so as to minimize the second loss function value.
10 . The method of claim 7 , wherein training of the obtained ML based model further comprises:
receiving a textual representation of the phrase;
receiving a passage annotation data element, indicating existence of a variation of the phrase in the passage;
inferring the ML based model on a hierarchical set of sequences obtained from said passage, to calculate an interim maximal vector similarity value; and
training the ML based model such that a value of the interim maximal vector similarity value corresponds to occurrence of a variation of the phrase in the passage, as represented by the passage annotation data element.
11 . The method of claim 10 , wherein the passage annotation data element is devoid of information indicating a location of the occurrence of the variation of the phrase in the passage.
12 . A system for identifying occurrence of a semantic variation of a phrase in a passage, the system comprising: a non-transitory memory device, wherein modules of instruction code are stored, and at least one processor associated with the memory device, and configured to execute the modules of instruction code, whereupon execution of said modules of instruction code, the at least one processor is configured to:
based on a textual representation of the phrase, calculate a phrase embedding vector, representing a semantic meaning of the phrase;
obtain a textual representation of the passage, comprising a plurality of words;
extract, from the textual representation of the passage, at least one hierarchical set of nested sequences of words, wherein each sequence of a hierarchical set forms a subset of words of a subsequent sequence in the hierarchical set;
for each of the at least one hierarchical set of nested sequences, calculate a corresponding sequence embedding vector, representing a semantic meaning of the hierarchical set of nested sequences;
for one or more sequence embedding vectors, calculate a corresponding vector similarity value, representing similarity with the phrase embedding vector;
identify a sequence corresponding to a maximal vector similarity value of the one or more vector similarity values; and
determine the identified sequence as a semantic variation of the phrase, based on the maximal vector similarity value.
13 . The system of claim 12 , wherein the at least one hierarchical set comprises a plurality of hierarchical sets.
14 . The system of claim 13 , wherein the at least one processor is further configured to extract the plurality of hierarchical sets by:
selecting a plurality of kernel sequences, each comprising one or more words of the passage; and
for each kernel sequence, producing a respective hierarchical set of nested sequences, each comprising the kernel sequence and subsequent, monotonically increasing, nesting sequences of words.
15 . The system of claim 14 , wherein the textual representation of the passage is a transcript of a conversation, and wherein the at least one processor is further configured to select the plurality of kernel sequences by:
selecting a section of the transcript, associated with a specific speaker in the conversation; and
selecting the plurality of kernel sequences to comprise one or more words of the selected section.
16 . The system of claim 14 , wherein the at least one processor is further configured to select the plurality of kernel sequences by:
calculating, for one or more words of the passage, one or more respective labels representing parts of speech (POS); and
selecting the plurality of kernel sequences to comprise one or more words of the passage based on the calculated one or more respective labels representing POS.
17 . The system of claim 14 , wherein the at least one processor is further configured to select the plurality of kernel sequences by:
calculating, for one or more words of the passage, one or more respective metrics of term relevance; and
selecting the plurality of kernel sequences to comprise one or more words of the passage based on the calculated one or more respective metrics of term relevance.
18 . The system of claim 12 , wherein the at least one processor is further configured to calculate a corresponding sequence embedding vector of a specific sequence by:
obtaining a machine-learning (ML) based model, pretrained to map between textual representations of words and corresponding word embedding vectors;
inferring the ML based model on one or more words of the specific sequence, to produce one or more corresponding word embedding vectors, based on said pre-training; and
calculating the corresponding sequence embedding vector of the specific sequence as a function of the one or more corresponding word embedding vectors.
19 . The system of claim 18 , wherein the at least one processor is further configured to train the obtained ML based model further by:
receiving a textual representation of the phrase;
receiving a passage annotation data element, indicating existence of a variation of the phrase in the passage;
inferring the ML based model on a hierarchical set of sequences obtained from said passage, to calculate an interim maximal vector similarity value; and
training the ML based model such that a value of the interim maximal vector similarity value corresponds to occurrence of a variation of the phrase in the passage, as represented by the passage annotation data element.
20 . A method of identifying occurrence of a semantic variation of a phrase in a passage by at least one processor, the method comprising:
based on a textual representation of the phrase, calculating a phrase embedding vector, representing a semantic meaning of the phrase;
receiving a textual representation of the passage, comprising a plurality of n-grams,
extracting, from the textual representation of the passage, a plurality of sequences of n-grams;
for each sequence, calculating a corresponding sequence embedding vector, representing a semantic meaning of the sequence;
for one or more sequence embedding vectors, calculating a similarity with the phrase embedding vector; and
determining an identified sequence as a semantic variation of the phrase, based on the one or more calculated similarities.