Method for querying long-form speech

1 . A method of applying a search engine query against a transcript data source, the method comprising:

parsing the search engine query into a tree of operations, the tree of operations comprising literals and operators, wherein each of the literals comprises at least one word and wherein each of the operators is one of a binary operator and a unary operator;

generating a query matrix from the literals;

generating a transcript matrix from a document of the transcript data source, wherein the transcript data source comprises multiple documents storing metadata-enriched text generated from recorded or streamed audio communications;

wherein the metadata-enriched text includes word embeddings and embedded non-speech data,

wherein the word embeddings represent individual words using real-valued vectors in a predefined vector space and wherein words that have a same meaning have a similarity score indicating related vector representations in the predefined vector space, and wherein the similarity score is determined by a cross-correlation of the real-valued vectors representing the words in the predefined vector space, and

wherein the embedded non-speech data stores non-speech information detected in the recorded or streamed audio communications, including at least one of: information regarding the meaning of words in the metadata-enriched text, a speaker role of a speaker in the recorded or streamed audio communications, and an emotion detected for the speaker from the recorded or streamed audio communications;

determining a cross-correlation of the query matrix and the metadata-enriched text in the transcript matrix;

applying a softness map to the cross-correlation to determine one or more matches for each of the literals, the softness map generated from a thresholding softness assigned to each of the literals as part of the search engine query, wherein the thresholding softness assigned to each of the literals determines how much semantic difference is permitted between the literal and the one or more cross-correlation matches for that literal, and wherein the one or more matches are identified from the cross-correlations that fall above the thresholding softness assigned to the literal;

combining the one or more matches utilizing the tree of operations and the operators to generate search results, each of the operators corresponding to a combination operation for the matches; and

reducing a number of documents in the transcript data source by applying the search results, thereby reducing a memory and computational overhead of a subsequent query.

2 . The method of claim 1 , wherein determining the cross-correlation of the query matrix and the transcript matrix comprises:

transforming the query matrix and the transcript matrix into a query word embedding matrix and transcript word embedding matrix utilizing a multi-dimensional word embedding for each word in the query matrix and the transcript matrix; and

determining a sum of dot products of the query matrix and the transcript matrix.

3 . The method of claim 2 , wherein query word-transcript word pairs from the query matrix and the transcript matrix and the sum of the dot products for each of the query word-transcript word pairs is stored in a quantitative thesaurus matrix.

4 . The method of claim 1 , wherein determining the cross-correlation of the query matrix and the transcript matrix comprises:

transforming the query matrix and the transcript matrix into a query word embedding matrix and transcript word embedding matrix utilizing a multi-dimensional word embedding for each word in the query matrix and the transcript matrix;

performing a Fourier transformation on the transcript matrix and the query matrix to generate a transformed transcript matrix and a transformed query matrix;

determining a point-wise product of the transformed transcript matrix and the transformed query matrix; and

performing an inverse Fourier transform on the point-wise product to determine the cross-correlation.

5 . The method of claim 1 , further comprising:

transforming the query matrix and the transcript matrix into a query word embedding matrix and transcript word embedding matrix utilizing a multi-dimensional word embedding for each word in the query matrix and the transcript matrix;

determining at least one of the literals comprises two or more words; and

applying a Gaussian kernel to the transcript word embedding matrix.

6 . The method of claim 1 , wherein determining the cross-correlation of the query matrix and the transcript matrix comprises:

determining query word-transcript word pairs; and

sending a control to a quantitative thesaurus matrix to return a similarity score for each of the query word-transcript word pairs; and

setting the cross-correlation equal to the similarity score.

7 . The method of claim 1 , wherein the transcript comprises one or more documents, further comprising:

sending a control to a quantitative thesaurus matrix to return similar words, the similar words having a similarity score above a thresholding softness for each of the literals;

determining a set of the one or more documents having at least one of the similar words; and

utilizing the set of the one or more documents as the transcript.

8 . The method of claim 1 , wherein the query further comprises non-speech information, further comprising adding a non-speech information dimension to the query matrix and the transcript matrix.

9 . The method of claim 8 , wherein the non-speech information is an emotion model:

the non-speech information dimension of the query matrix being a query flag; and

the non-speech information dimension of the transcript matrix being an output of the emotion model.

10 . The method of claim 8 , wherein the non-speech information is speaker metadata:

the non-speech information dimension of the query matrix being a query flag; and

the non-speech information dimension of the transcript matrix being an indication of a speaker.

11 . A computing apparatus, the computing apparatus comprising:

a processor; and

a memory storing instructions that, when executed by the processor, configure the apparatus to:

parse a search engine query into a tree of operations, the tree of operations comprising literals and operators, wherein each of the literals comprises at least one word and wherein each of the operators is one of a binary operator and a unary operator;

generate a query matrix from the literals;

generate a transcript matrix from a document of a transcript data source, wherein the transcript data source comprises multiple documents storing metadata-enriched text generated from recorded or streamed audio communications;

wherein the metadata-enriched text includes word embeddings and embedded non-speech data,

wherein the word embeddings represent individual words using real-valued vectors in a predefined vector space and wherein words that have a same meaning have a similarity score indicating related vector representations in the predefined vector space, and wherein the similarity score is determined by a cross-correlation of the real-valued vectors representing the words in the predefined vector space, and

wherein the embedded non-speech data stores non-speech information detected in the recorded or streamed audio communications, including at least one of: information regarding the meaning of words in the metadata-enriched text, a speaker role of a speaker in the recorded or streamed audio communications, and an emotion detected for the speaker from the recorded or streamed audio communications;

determine a cross-correlation of the query matrix and the metadata-enriched text in the transcript matrix;

apply a softness map to the cross-correlation to determine one or more matches for each of the literals, the softness map generated from a thresholding softness assigned to each of the literals as part of the search engine query, wherein the thresholding softness assigned to each of the literals determines how much semantic difference is permitted between the literal and the one or more cross-correlation matches for that literal, and wherein the one or more matches are identified from the cross-correlations that fall above the thresholding softness assigned to the literal;

combine the one or more matches utilizing the tree of operations and the operators to generate search results, each of the operators corresponding to a combination operation for the matches; and

reduce a number of documents in the transcript data source by applying the search results, thereby reducing a memory and computational overhead of a subsequent query.

12 . The computing apparatus of claim 11 , wherein determining the cross-correlation of the query matrix and the transcript matrix comprises:

transform the query matrix and the transcript matrix into a query word embedding matrix and transcript word embedding matrix utilizing a multi-dimensional word embedding for each word in the query matrix and the transcript matrix; and

determine a sum of dot products of the query matrix and the transcript matrix.

13 . The computing apparatus of claim 12 , wherein query word-transcript word pairs from the query matrix and the transcript matrix and the sum of the dot products for each of the query word-transcript word pairs is stored in a quantitative thesaurus matrix.

14 . The computing apparatus of claim 11 , wherein determining the cross-correlation of the query matrix and the transcript matrix comprises:

transform the query matrix and the transcript matrix into a query word embedding matrix and transcript word embedding matrix utilizing a multi-dimensional word embedding for each word in the query matrix and the transcript matrix;

perform a Fourier transformation on the transcript matrix and the query matrix to generate a transformed transcript matrix and a transformed query matrix;

determine a point-wise product of the transformed transcript matrix and the transformed query matrix; and

perform an inverse Fourier transform on the point-wise product to determine the cross-correlation.

15 . The computing apparatus of claim 11 , wherein the instructions further configure the apparatus to:

transform the query matrix and the transcript matrix into a query word embedding matrix and transcript word embedding matrix utilizing a multi-dimensional word embedding for each word in the query matrix and the transcript matrix;

determine at least one of the literals comprises two or more words; and

apply a Gaussian kernel to the transcript word embedding matrix.

16 . The computing apparatus of claim 11 , wherein determining the cross-correlation of the query matrix and the transcript matrix comprises:

determine query word-transcript word pairs; and

send a control to a quantitative thesaurus matrix to return a similarity score for each of the query word-transcript word pairs; and

set the cross-correlation equal to the similarity score.

17 . The computing apparatus of claim 11 , wherein the transcript comprises one or more documents, wherein the instructions further configure the apparatus to:

send a control to a quantitative thesaurus matrix to return similar words, the similar words having a similarity score above a thresholding softness for each of the literals;

determine a set of the one or more documents having at least one of the similar words; and

utilize the set of the one or more documents as the transcript.

18 . The computing apparatus of claim 11 , wherein the query further comprises non-speech information, wherein the instructions further configure the apparatus to add a non-speech information dimension to the query matrix and the transcript matrix.

19 . The computing apparatus of claim 18 , wherein the non-speech information is an emotion model:

the non-speech information dimension of the query matrix being a query flag; and

the non-speech information dimension of the transcript matrix being an output of the emotion model.

20 . The computing apparatus of claim 18 , wherein the non-speech information is speaker metadata:

the non-speech information dimension of the query matrix being a query flag; and

the non-speech information dimension of the transcript matrix being an indication of a speaker.