System and method for validating natural language content using crowdsourced validation jobs

1. A computer-implemented method, the method being implemented in a computer system having one or more physical processors programmed with computer program instructions that, when executed by the one or more physical processors, program the computer system to perform the method, the method comprising:

causing, by the computer system, a first crowdsourced validation job to be provided to one or more first validation devices, the first crowdsourced validation job comprising first instructions for a crowd user to provide an indication of an accuracy of a transcription of natural language content;

receiving, by the computer system, a plurality of responses from the one or more first validation devices, wherein the plurality of responses include at least a first response from at least a first validation device from among the one or more first validation devices, the first response including a first indication of an accuracy of the transcription of the natural language content; and

determining, by the computer system, a first confidence score of the first validation device based, at least in part, on the plurality of responses received from the one or more first validation devices and the first response received from the first validation device.

2. The method of claim 1 , the method further comprising: assigning, by the computer system, a validation score to the transcription of natural language content based, at least in part, on the plurality of responses received from the one or more first validation devices and the first confidence score of the first validation device.

3. The method of claim 1 , the method further comprising: receiving, by the computer system, a threshold confidence score; and determining, by the computer system, whether the first confidence score of the first validation device is below the threshold confidence score.

4. The method of claim 3 , wherein the first crowdsourced validation job is part of a crowdsourced campaign comprising the first crowdsourced validation job and a second crowdsourced validation job, the method further comprising: preventing, by the computer system, the first validation device from participating in the second crowdsourced validation job responsive to a determination that the first confidence score is below the threshold confidence score.

5. The method of claim 3 , wherein if the first confidence score of the first validation device is below the threshold confidence score, the method further comprises:

causing, by the computer system, a second crowdsourced validation job to be provided to one or more second validation devices, the second crowdsourced validation job comprising second instructions for a crowd user to provide an indication of an accuracy of the transcription of natural language content;

receiving, by the computer system, at least a second response from at least a second validation device from among the one or more second validation devices, wherein the second response includes a second indication of an accuracy of the transcription of the natural language content; and assigning, by the computer system, a validation score to the transcription of natural language content based, at least in part, on the second response received from the second validation device.

6. The method of claim 5 , the method further comprising: identifying, by the computer system, the one or more second validation devices based, at least in part, on the first confidence score of the first validation device.