Multi-stage adaptive system for content moderation

1. A toxicity moderation system, the system comprising

an input configured to receive speech from a speaker;

a multi-stage toxicity machine learning system including a first stage and a second stage, wherein the first stage is trained to analyze the received speech to determine whether a toxicity level of the speech meets a toxicity threshold,

the first stage configured to filter-through, to the second stage, speech that meets the toxicity threshold, and further configured to filter-out speech that does not meet the toxicity threshold.

2. The toxicity moderation system of claim 1 , wherein the first stage is trained using a database having training data with positive and/or negative examples of training content for the first stage.

3. The toxicity moderation system of claim 2 , wherein the first stage is trained using a feedback process comprising:

receiving speech content;

analyzing the speech content using the first stage to categorize the speech content as having first-stage positive speech content and/or first-stage negative speech content;

analyzing the first-stage positive speech content using the second stage to categorize the first-stage positive speech content as having second-stage positive speech content and/or second-stage negative speech content; and

updating the database using the second-stage positive speech content and/or the second-stage negative speech content.

4. The toxicity moderation system of claim 3 , wherein the first stage discards at least a portion of the first-stage negative speech content.

5. The toxicity moderation system of claim 3 , wherein the first stage is trained using the feedback process further comprising:

analyzing less than all of the first-stage negative speech content using the second stage to categorize the first-stage negative speech content as having second-stage positive speech content and/or second-stage negative speech content,

further updating the database using the second-stage positive speech content and/or the second-stage negative speech content.

6. The toxicity moderation system of claim 1 , further comprising a random uploaded configured to upload portions of the speech that did not meet the toxicity threshold to the subsequent stage or a human moderator.

7. The toxicity moderation system of claim 1 , further comprising a session context flagger configured to receive an indication that the speaker previously met the toxicity threshold within a pre-determined amount of time, and to: (a) adjust the toxicity threshold, or (b) upload portions of the speech that did not meet the toxicity threshold to the subsequent stage or a human moderator.

8. The toxicity moderation system of claim 1 , further comprising a user context analyzer, the user context analyzer configured to adjust the toxicity threshold and/or the toxicity confidence based on the speaker's age, a listener's age, the speaker's geographic region, the speaker's friends list, history of recently interacted listeners, speaker's gameplay time, length of speaker's game, time at beginning of game and end of game, and/or gameplay history.

9. The toxicity moderation system of claim 1 , further comprising an emotion analyzer trained to determine an emotion of the speaker.

10. The toxicity moderation system of claim 1 , further comprising an age analyzer trained to determine an age of the speaker.

11. The toxicity moderation system of claim 1 , further comprising a temporal receptive field configured to divide speech into time segments that can be received by at least one stage.

12. The toxicity moderation system of claim 1 , further comprising a speech segmenter configured to divide speech into time segments that can be analyzed by at least one stage.

13. The toxicity moderation system of claim 1 , wherein the first stage is more efficient than the second stage.

14. A multi-stage content analysis system comprising:

a first stage trained using a database having training data with positive and/or negative examples of training content for the first stage,

the first stage configured to:

receive speech content,

analyze the speech content to categorize the speech content as having first-stage positive speech content and/or first-stage negative speech content;

a second stage configured to receive at least a portion, but less than all, of the first-stage negative speech content,

the second stage further configured to analyze the first-stage positive speech content to categorize the first-stage positive speech content as having second-stage positive speech content and/or second-stage negative speech content, the second stage further configured to update the database using the second-stage positive speech content and/or the second-stage negative speech content.

15. The multi-stage content analysis system of claim 14 , wherein:

the second stage is configured to analyze the received first-stage negative speech content to categorize the first-stage negative speech content as having second-stage positive speech content and/or second-stage negative speech content.

16. The multi-stage content analysis system of claim 15 , wherein:

the second stage is configured to update the database using the second-stage positive speech content and/or the second-stage negative speech content.

17. A method of training a multi-stage content analysis system, the method comprising:

providing a multi-stage content analysis system, the system having a first stage and a second stage;

training the first stage using a database having training data with positive and/or negative examples of training content for the first stage;

receiving speech content;

analyzing the speech content using the first stage to categorize the speech content as having first-stage positive speech content and/or first-stage negative speech content;

analyzing the first-stage positive speech content using the second stage to categorize the first-stage positive speech content as having second-stage positive speech content and/or second-stage negative speech content;

updating the database using the second-stage positive speech content and/or the second-stage negative speech content;

discarding at least a portion of the first-stage negative speech content.

18. The method of claim 17 , the method comprising:

analyzing less than all of the first-stage negative speech content using the second stage to categorize the first-stage negative speech content as having second-stage positive speech content and/or second-stage negative speech content,

further updating the database using the second-stage positive speech content and/or the second-stage negative speech content.

19. The method of claim 18 , further comprising:

using a database having training data with positive and/or negative examples of training content for the first stage;

producing first-stage positive determinations (“S1-positive determinations”) associated with a portion of the speech content, and/or first-stage negative determinations (“S1-negative determinations”);

analyzing the speech associated with the S1-positive determinations.

20. The method of claim 19 , wherein the positive and/or negative examples relate to particular categories of toxicity.