Systems and methods for dynamically generating groups of received textual data for collective labeling

1 . A system for dynamically generating textual data groups in real time such that the textual data groups preserve semantic and syntactic information for collective labeling, the system comprising:

one or more processors; and

one or more non-transitory, computer-readable media comprising instructions that, when executed by the one or more processors, cause operations comprising:

storing within a label record database, at a first timestamp, first textual data representing a first text string from chatbot conversations received from a first device in a computer network;

querying the label record database to retrieve a plurality of datasets respectively corresponding to a plurality of receipt ranges for the first textual data based on the first timestamp and each of the plurality of datasets comprises chatbot data labeled with respective pre-existing labels;

inputting, the first textual data and a dataset corresponding to a receipt range for the first textual data based on the first timestamp from the plurality of datasets into a natural language processing model to determine a similarity metric indicating a measure of text similarity between the first textual data and data within the dataset;

assigning and storing, in the label record database, in response to determining that the similarity metric is greater than or equal to a threshold similarity metric, a first pre-existing label of second textual data from the dataset representing a second text string and having a second timestamp within the receipt range, to the first textual data to reduce labeling tasks;

inputting, (i) the first textual data, (ii) the second textual data, and (iii) third textual data from the dataset representing a third text string from the chatbot conversations and having a third timestamp at which the third textual data was received within the receipt range, into a the natural language processing model to calculate a representativeness metric for the third textual data based on the first textual data, the second textual data and the third textual data, wherein the representativeness metric indicates an extent to which the second textual data represents a set comprising the first textual data, the second textual data, and the third textual data;

preventing the first pre-existing label assigned to the first textual data from being updated, in the label record database, based on a second pre-existing label for the third textual data, responsive to (i) the similarity metric being greater than or equal to the threshold similarity metric and (ii) determining that the third textual data fails to be representative of at least the first textual data and the second textual data from the dataset based on the representativeness metric being less than a threshold representative metric; and

based on the representativeness metric being less than the threshold representative metric, providing, to the first device, a user interface comprising a warning to enable tracking and filtering of the first textual data.

2 . A method comprising:

storing, within a label record database, first textual data representing a first text string received from a first device in a computer network at a first timestamp;

querying the label record database to retrieve a plurality of datasets respectively corresponding to a plurality of receipt ranges for the first textual data based on the first timestamp and each of the plurality of datasets comprises chatbot data labeled with respective pre-existing labels;

inputting the first textual data and a dataset corresponding to a receipt range for the first textual data based on the first timestamp into a natural language processing model to determine a similarity metric indicating a similarity of the first textual data and the dataset;

assigning and storing, in the label record database, based on the similarity metric being greater than or equal to a threshold similarity metric, a first pre-existing label for the of second textual data from the dataset having a second timestamp within the receipt range as a label for the first textual data to reduce labeling tasks;

inputting (i) the first textual data, (ii) the second textual data, and (iii) third textual data from the dataset having a third timestamp within the receipt range, into the natural language processing model to calculate a first representativeness metric for the third textual data based on the first textual data, the second textual data, and the third textual data;

preventing the first pre-existing label assigned to the first textual data from being updated, in the label record database, based on a second pre-existing label for the third textual data, responsive to (i) the similarity metric being greater than or equal to the threshold similarity metric and (ii) the first representativeness metric being less than a threshold representative metric; and

based on the first representativeness metric being less than the threshold representative metric, providing, to the first device, a user interface comprising a warning to enable tracking and filtering of the first textual data.

3 . The method of claim 2 , wherein

the first representativeness metric for the second textual data indicates an extent to which the second textual data represents a set comprising the first textual data, the second textual data, and the third textual data.

4 . The method of claim 2 , further comprises:

inputting, (i) the first textual data, (ii) the second textual data, and (iii) fourth textual data from the dataset having a fourth timestamp within the receipt range into the natural language processing model to calculate (i) a second representativeness metric for the second textual data based on the first textual data, the second textual data, and the fourth textual data, and (ii) a third representativeness metric for the fourth textual data; and

updating and storing, in the label record database, based on comparing the second representativeness metric and the third representativeness metric, the label for the first textual data based on a third pre-existing label for the fourth textual data.

5 . The method of claim 2 , wherein storing the first textual data comprises:

retrieving a system clock setting at a time of receipt for the first textual data from the first device in the computer network, wherein the system clock setting comprises a standardized setting for recording times across a computer network; and

generating the first timestamp based on the system clock setting.

6 . The method of claim 2 , wherein inputting the first textual data and the dataset corresponding to the receipt range comprises:

based on the first textual data, generating a relevance window comprising a length of time and indicates a period of time before which information is not to be considered for labeling decisions; and

based on the relevance window, determining a relevance timestamp after which the information is considered to determine the receipt range.

7 . The method of claim 2 , wherein inputting the first textual data and the dataset into the natural language processing model comprises:

calculating an average receipt time for the dataset; and

comparing the average receipt time with the receipt range, wherein the dataset is selected based on that the average receipt time being within the receipt range.

8 . The method of claim 2 , wherein inputting the first textual data and the dataset into the natural language processing model to determine the similarity metric comprises:

generating a vector representation of the first textual data;

generating a plurality of vector representations respectively corresponding to a plurality of text strings in the dataset;

calculating a plurality of similarity metrics respectively corresponding to a measure of similarity between the vector representation and each vector representation in the plurality of vector representations; and

calculating an average of the plurality of similarity metrics, wherein the similarity metric represents the average.

9 . The method of claim 8 , wherein calculating the plurality of similarity metrics comprises:

generating a plurality of text distances between the vector representation and each vector representation in the plurality of vector representations; and

determining the plurality of similarity metrics based on the plurality of text distances.

10 . The method of claim 8 , wherein calculating the plurality of similarity metrics comprises:

inputting the vector representation and the plurality of vector representations into the natural language processing model; and

based on inputting the vector representation and the plurality of vector representations into the natural language processing model, generating the plurality of similarity metrics.

11 . The method of claim 2 , further comprising:

generating a feature input for training a machine learning model based on the first textual data and the label for the first textual data, wherein the machine learning model uses a sentiment analysis algorithm; and

generating an output for the machine learning model based on the first textual data and the label.

12 . The method of claim 11 , further comprising:

based on training the machine learning model, generating a prediction for a sentiment characterizing sample textual data; and

based on comparing the sentiment to a training sentiment, determining a performance metric for the machine learning model, wherein the training sentiment indicates a ground-truth sentiment.

13 . The method of claim 2 , wherein the similarity metric comprises a first similarity metric, the method further comprises:

inputting fourth textual data having a fourth timestamp within the receipt range and the dataset into the natural language processing model;

to determine a second similarity metric indicating a similarity of the fourth textual data and the dataset; and

based on the second similarity metric being less than the threshold similarity metric, providing, to the first device, a user interface comprising a warning to enable tracking and filtering of the first textual data.

14 . The method of claim 2 , wherein the similarity metric comprises a first similarity metric, the method further comprises:

inputting fourth textual data having a fourth timestamp within the receipt range and the dataset into the natural language processing model to determine a second similarity metric indicating a similarity of the fourth textual data and the dataset; and

based on the second similarity metric being less than the threshold similarity metric, selecting a second dataset from the plurality of datasets based on the second dataset corresponding to the receipt range.

15 . One or more non-transitory, computer-readable media comprising instructions that, when executed by one or more processors, causes operations comprising:

storing, within a label record database, first textual data representing a first text string received from a first device of a computer network at a first timestamp;

querying the label record database to retrieve a plurality of datasets, respectively corresponding to a plurality of receipt ranges for the first textual data based on the first timestamp and each of the plurality of datasets comprises chatbot data labeled with respective pre-existing labels;

inputting, the first textual data and a dataset corresponding to a receipt range for the first textual data based on the first timestamp from the plurality of datasets into an artificial intelligence model to determine a similarity metric indicating a similarity of the first textual data and the dataset;

assigning and storing, in the label record database, based on the similarity metric being greater than or equal to a threshold similarity metric, a first pre-existing label of second textual data having a second timestamp within the receipt range as a label for the first textual data;

inputting (i) the first textual data, (ii) the second textual data, and (iii) third textual data from the dataset having a third timestamp within the receipt range, into the artificial intelligence model to calculate a first representativeness metric for the third textual data;

preventing the first pre-existing label assigned to the first textual data from being updated, in the label record database, based on a second pre-existing label for the third textual data; and

based on the first representativeness metric being less than a threshold representative metric, providing, to the first device, a user interface comprising a warning to enable tracking and filtering of the first textual data.

16 . The one or more non-transitory, computer-readable media of claim 15 , wherein the operations comprising:

inputting (i) the first textual data, (ii) the second textual data, and (iii) fourth textual data from the dataset having a fourth timestamp within the receipt range into the artificial intelligence model to calculate (i) a second representativeness metric for the second textual data, and (ii) a third representativeness metric for the fourth textual data; and

updating and storing, in the label record database, based on comparing the second representativeness metric and the third representativeness metric, the label for the first textual data based on a third pre-existing label for the fourth textual data.

17 . The one or more non-transitory, computer-readable media of claim 15 , wherein inputting the first textual data and the dataset into the artificial intelligence model to determine the similarity metric between the first textual data and the dataset comprises:

generating a vector representation of the first textual data;

generating a plurality of vector representations, respectively corresponding to a plurality of text strings in the dataset;

calculating a plurality of similarity metrics, respectively corresponding to a measure of similarity between the vector representation and each vector representation in the plurality of vector representations; and

calculating an average of the plurality of similarity metrics, wherein the similarity metric represents the average.