METHODS AND SYSTEMS FOR CROWDSOURCING A TASK
The disclosed embodiments illustrate methods and systems for crowdsourcing a task. The method comprises presenting the task to one or more crowdworkers based on their reputation scores and a redundancy degree associated with the task. The task comprises one or more sub-tasks. A plurality of responses to the task are received from the one or more crowdworkers. Each response further comprises one or more sub-responses each of which in turn corresponds to a response for a sub-task of the one or more sub-tasks. A first set of sub-tasks is identified from the one or more sub-tasks based on a first set of similar sub-responses received for the first set of sub-tasks from the plurality of responses. A first set of crowdworkers from the one or more crowdworkers who provided the first set of similar sub-responses for the first set of sub-tasks are remunerated.
1 . A method for crowdsourcing a task, the method comprising:
presenting the task to one or more crowdworkers based on a reputation score associated with each of the one or more crowdworkers and a degree of redundancy associated with the task, wherein the task comprises one or more sub-tasks;
receiving a plurality of responses for the task from the one or more crowdworkers, wherein each response from the plurality of responses comprises one or more sub-responses, wherein each of the one or more sub-responses corresponds to a response for a sub-task of the one or more sub-tasks;
identifying a first set of sub-tasks from the one or more sub-tasks based on a first set of similar sub-responses received for the first set of sub-tasks from the plurality of responses and a first predetermined number; and
remunerating a first set of crowdworkers from the one or more crowdworkers, wherein the first set of crowdworkers have provided the first set of similar sub-responses for the first set of sub-tasks,
wherein the method is performed by a processor on a computing device.
2 . The method of claim 1 , wherein the degree of redundancy associated with the task corresponds to a maximum number of crowdworkers who are presented the task.
3 . The method of claim 1 , wherein the first predetermined number corresponds to a minimum number of similar sub-responses required for the identification of the first set of sub-tasks.
4 . The method of claim 1 further comprising storing sub-responses received for a second set of sub-tasks, wherein the second set of sub-tasks corresponds to the one or more sub-tasks excluding the identified first set of sub-tasks.
5 . The method of claim 4 further comprising presenting the task to a second set of crowdworkers from the one or more crowdworkers based on a second predetermined number and the redundancy degree associated with the task, such that a second set of sub-responses is received from the second set of crowdworkers for the second set of sub-tasks, wherein the second predetermined number corresponds to a maximum number of permissible re-assignments of the task.
6 . The method of claim 5 further comprising identifying at least one sub-task from the second set of sub-tasks based on a second set of similar sub-responses for the at least one sub-task from the second set of sub-responses, and a third set of similar sub-responses for the at least one sub-task from the stored sub-responses.
7 . The method of claim 6 , wherein a sum of the second set of similar sub-responses and the third set of similar sub-responses is at least the first predetermined number.
8 . The method of claim 6 further comprising remunerating a third set of crowdworkers from the one or more crowdworkers, wherein the third set of crowdworkers have provided the similar sub-responses for the at least one sub-task in the second set sub-tasks.
9 . The method of claim 6 further comprising modifying the reputation score associated with each of the one or more crowdworkers based on the identification of the first set of sub-tasks and the at least one sub-task.
10 . A crowdsourcing management system for managing crowdsourcing of a task, the crowdsourcing management system comprising:
a processor; and
a memory comprising:
a task manager configured to present the task to one or more crowdworkers based on a reputation score associated with each of the one or more crowdworkers and a degree of redundancy associated with the task, wherein the task comprises one or more sub-tasks;
a communication manager configured to receive a plurality of responses for the task from the one or more crowdworkers, wherein each response from the plurality of responses comprises one or more sub-responses, wherein each of the one or more sub-responses corresponds to a response for a sub-task of the one or more sub-tasks;
a redundancy handler configured to identify a first set of sub-tasks from the one or more sub-tasks based on a first set of similar sub-responses received for the first set of sub-tasks from the plurality of responses and a first predetermined number; and
a remuneration manager configured to remunerate a first set of crowdworkers from the one or more crowdworkers, wherein the first set of crowdworkers have provided the first set of similar sub-responses for the first set of sub-tasks.
11 . The crowdsourcing management system of claim 10 , wherein the degree of redundancy associated with the task corresponds to a maximum number of crowdworkers who are presented the task.
12 . The crowdsourcing management system of claim 10 , wherein the first predetermined number corresponds to a minimum number of similar sub-responses required for the identification of the first set of sub-tasks.
13 . The crowdsourcing management system of claim 10 , wherein the redundancy handler is further configured to store sub-responses received for a second set of sub-tasks, wherein the second set of sub-tasks corresponds to the one or more sub-tasks excluding the identified first set of sub-tasks.
14 . The crowdsourcing management system of claim 13 , wherein the task manager is further configured to present the task to a second set of crowdworkers from the one or more crowdworkers based on a second predetermined number and the redundancy degree associated with the task, such that a second set of sub-responses is received from the second set of crowdworkers for the second set of sub-tasks, wherein the second predetermined number corresponds to a maximum number of permissible re-assignments of the task.
15 . The crowdsourcing management system of claim 14 , wherein the redundancy handler is further configured to identify at least one sub-task from the second set of sub-tasks based on a second set of similar sub-responses for the at least one sub-task from the second set of sub-responses, and a third set of similar sub-responses for the at least one sub-task from the stored sub-responses.
16 . The crowdsourcing management system of claim 15 , wherein a sum of the second set of similar sub-responses and the third set of similar sub-responses is at least the first predetermined number.
17 . The crowdsourcing management system of claim 15 , wherein the remuneration manager is further configured to remunerate a third set of crowdworkers from the one or more crowdworkers, wherein the third set of crowdworkers have provided the similar sub-responses for the at least one sub-task in the second set sub-tasks.
18 . The crowdsourcing management system of claim 15 further comprising a crowdworker manager configured to modify the reputation score associated with each of the one or more crowdworkers based on the identification of the first set of sub-tasks and the at least one sub-task.
19 . A computer program product for use with a computing device, the computer program product comprising a non-transitory computer-usable medium having a computer readable program code embodied therein for crowdsourcing a task, the computer readable program code executable by a processor in the computing device for:
presenting the task to one or more crowdworkers based on a reputation score associated with each of the one or more crowdworkers and a degree of redundancy associated with the task, wherein the task comprises one or more sub-tasks, wherein the degree of redundancy associated with the task corresponds to a maximum number of crowdworkers who are presented the task;
receiving a plurality of responses for the task from the one or more crowdworkers, wherein each response from the plurality of responses comprises one or more sub-responses, wherein each of the one or more sub-responses corresponds to a response for a sub-task of the one or more sub-tasks;
identifying a first set of sub-tasks from the one or more sub-tasks based on a first set of similar sub-responses received for the first set of sub-tasks from the plurality of responses and a first predetermined number; and
remunerating a first set of crowdworkers from the one or more crowdworkers, wherein the first set of crowdworkers have provided the first set of similar sub-responses for the first set of sub-tasks.
20 . The computer program product 19 , wherein the first predetermined number corresponds to a minimum number of the similar sub-responses required for the identification of the first set of sub-tasks.