System and method for text categorization and sentiment analysis

1. A system for categorization and sentiment analysis, comprising:

a chunk parser for

receiving input text; and

breaking the input text into chunks of text comprising words and phrases;

a chunk sentiment analyzer which receives the chunks of text from the chunk parser, assigns a sentiment to each chunk of text, and passes each chunk with its assigned sentiment to a deterministic rules engine;

the deterministic rules engine which:

categorizes each chunk of text into a first set of semantic categories using regular expression rules; and

for chunks of text where no regular expression rule is found for categorization into the first set of semantic categories, passes those chunks of text to a semantic similarity engine;

the semantic similarity engine which:

adds a vector to each chunk of text received from the deterministic rules engine representing the semantic characteristics of that chunk of text;

categorizes the chunks of text into a second set of semantic categories based on a threshold semantic distance from one or more category anchor vectors; and

for chunks of text where no match is found for categorization into the second set of semantic categories, passes those chunks of text to a semantic cluster discovery engine;

a semantic cluster discovery engine for categorizing chunks of text received from the semantic similarity engine into a third set of semantic categories based on their clustering relative to one another, for those chunks of text which do not fall within the threshold distance from any of the one or more category anchor vectors; and

a category comparator and integrator for

comparing the first, second, and third sets of semantic categories to identify contextual associations between the chunks of text in each semantic category; and

calculating a sentiment for the input text based on the contextual associations.

2. The system of claim 1 , further comprising a sequence reducer and embedder for

after sentiment has been calculated on each chunk of text, reducing each chunk of text further into a sequence of words which preserves the order of words from the input text;

embedding each input word sequence into a vector according to a chosen sequence embedding model.

3. The system of claim 1 , further comprising a trend analyzer for

as additional input texts are received, analyzing and displaying:

the number of and proportion of texts in each category;

the growth or decline of categories over time; and

an automated management alert when an emerging category grows at or above a threshold rate.

4. The system of claim 1 , further comprising a supervised machine learning algorithm for

analyzing the attributes of the input text, the categories, and calculated sentiment; and

predicting a combination of attributes which will result in a given sentiment.

5. A method for categorization and sentiment analysis, comprising the steps of:

using a chunk parser operating on a computing device comprising a memory and a processor to perform the steps of:

receiving an input text;

breaking the input text into chunks of text comprising words and phrases;

using a chunk sentiment analyzer operating on the computing device to perform the steps of:

receiving the chunks of text from the chunk parser;

assigning a sentiment to each chunk of text; and

passing each chunk with its assigned sentiment to a deterministic rules engine;

using a deterministic rules engine operating on the computing device to perform the steps of:

categorizing each chunk of text into a first set of semantic categories using regular expression rules; and

for chunks of text where no regular expression rule is found for categorization into the first set of semantic categories, passing those chunks of text to a semantic similarity engine;

using a semantic similarity engine operating on the computing device to perform the steps of:

adding a vector to each chunk of text received from the deterministic rules engine representing the semantic characteristics of that chunk of text;

categorizing the chunks of text into a second set of semantic categories based on a threshold semantic distance from one or more category anchor vectors; and

for chunks of text where no match is found for categorization into the second set of semantic categories, passing those chunks of text to a semantic cluster discovery engine;

using a semantic cluster discovery engine operating on the computing device to perform the steps of:

categorizing those chunks of text received from the semantic similarity engine into a third set of semantic categories based on their clustering relative to one another, for chunks of text which do not fall within the threshold distance from any of the one or more category anchor vectors;

using a category comparator and integrator operating on the computing device to perform the steps of:

comparing the first, second, and third sets of semantic categories to identify contextual associations between the chunks of text in each semantic category; and

calculating a sentiment for the input text based on the contextual associations.

6. The method of claim 5 , further comprising the steps of:

after sentiment has been calculated on each chunk of text, reducing each chunk of text further into a sequence of words which preserves the order of words from the input text; and

embedding each input word sequence into a vector according to a chosen sequence embedding model.

7. The method of claim 5 , further comprising the steps of:

as additional input texts are received, analyzing and displaying:

the number of and proportion of texts in each category;

the growth or decline of categories over time; and

an automated management alert when an emerging category grows at or above a threshold rate.

8. The system of claim 5 , further comprising the steps of:

analyzing the attributes of the input text, the categories, and calculated sentiment using a machine learning algorithm; and

predicting a combination of attributes which will result in a given sentiment.