System and method for determining open-ended response quality

1. A method of filtering an original set of user-provided text responses across a network, comprising:

receiving, from one or more processors on one or more user computers, multiple user-provided text responses to a question, the multiple user-provided text responses forming the original set;

identifying text responses that are no-value text responses and removing the no-value text responses from the original set;

removing text responses from the original set where a length of a response does not meet a threshold length;

identifying text responses that are gibberish responses and removing gibberish responses from the original set;

sending remaining responses as a filtered set of responses to a machine learning system, the machine learning system to:

perform clustering on the filtered set to identify one or more clusters of text responses that are similar to each other; and

identifying text responses outside the one or more clusters as noise responses;

scoring the noise responses;

and

removing text responses having a score equal to or below a threshold score from the filtered set of responses to produce a final set; and

outputting the final set of responses to an information gatherer.

2. The method as claimed in claim 1 , wherein identifying no-value responses comprises:

using a sentence transformer to convert the text responses in the set to vector representations;

using the vector representations of the text responses to calculate cosine similarity between the text responses and a predefined list of no-value responses; and

identifying no-value responses in the text responses of the set when the cosine similarity of the vector representation for that text response is within a predetermined threshold to no-value responses on the predefined list.

3. The method as claimed in claim 2 , wherein the predetermined threshold has 0.9 cosine or higher similarity to the no-value responses on the predefined list.

4. The method as claimed in claim 1 , wherein removing text responses from the original set where the length of the response does not meet a threshold length, comprises:

using lengths of all responses in the set to determine a 25 th percentile, a 75 th percentile, and an inter-quartile range between the 75 th percentile and the 25 th percentile; and

setting the threshold length equal to 25 th percentile minus one and one half times the interquartile range.

5. The method as claimed in claim 1 , wherein identifying gibberish responses comprises:

identifying character groupings that are not words as gibberish;

identifying character groupings that are incomplete words; and

identifying text responses that match text of a corresponding question.

6. The method as claimed in claim 1 , wherein scoring the noise responses comprises:

identifying the text responses that lie within a cluster as valid responses;

building a corpus of words from the valid responses; and

scoring each noise response based upon a frequency of words from the noise response appearing in the corpus.

7. The method as claimed in claim 6 , wherein scoring each noise response comprises determining a frequency of appearance of a word from the noise response in the corpus.

8. The method as claimed in claim 6 , wherein scoring each noise response comprises determining a frequency of appearance of one or more of antonyms or synonyms of a word from the noise response in the corpus.

9. The method as claimed in claim 6 , wherein scoring each noise response comprises determining a frequency of appearance of a word from the noise response, and one or more of antonyms and synonyms of the word, in the corpus.

10. The method as claimed in claim 6 , wherein building a corpus comprises:

expanding any contractions used in the valid responses;

performing lemmatization of words used in the valid responses;

removing any stopwords;

calculating a term frequency for each word remaining in the corpus; and

calculating a sentence presence frequency for each word remaining in the corpus.