Method and system for receiving label for digital task executed within crowd-sourced environment

1. A computer-implemented method for receiving a label for a digital task executed within a computer-implemented crowd-sourced environment, the digital task corresponding to a search engine result page (SERP) relevancy labelling task, the method being executed by a server, the server being coupled communicatively to:

a SERP log database, the SERP log database comprising a plurality of SERPs previously generated;

the method comprising:

receiving, by the server, an indication of the digital task to be processed in the computer-implemented crowd-sourced environment;

retrieving, from the SERP log database, the plurality of SERPs;

transmitting the plurality of SERPs to a plurality of crowd-sourced worker in the computer-implemented crowd-sourced environment for evaluating a pertinence of each of the plurality of SERPs;

receiving, from the plurality of crowd-sourced worker, a plurality of labels for each of the plurality of SERPs;

inputting, the plurality of labels into a first machine learning algorithm (MLA) configured to generate a set of unbiased labels corresponding to the plurality of labels with a bias perception of the plurality of crowd-sourced worker removed;

determining, by the server, a subset of examples comprising a subset of the plurality of SERPs and associated unbiased labels, each of the subset of examples being similar to the digital task within a pre-determined similarity threshold;

mapping, by the server, the plurality of SERPs into a scale of n-dimensional space based on the associated unbiased labels;

inputting, by the server, the n-dimensional scale into a second MLA configured to generate one or more clusters, each of the one or more clusters providing maximum benchmark coverage of the n-dimensional scale, each cluster comprising a minimum number of SERP, wherein the second MLA has previously been trained based on a training set, the training set including a pair of digital tasks and a training label indicative of usefulness of a first of the pair of digital tasks as an illustrative example for a second one of the pair of digital tasks;

associating, by the server, the subset of examples to the digital task to be presented in the computer-implemented crowd-sourced environment;

causing the digital task to be presented on a computing device of at least one crowd-sourced worker in the computer-implemented crowd-sourced environment to solicit the label for the digital task;

determining, by the server, a step corresponding to a distance between each of the one or more clusters; and

generating, by the server, a sliding labelling scale, the sliding labelling scale having label indicators based on the subset of examples, selection of the label being done by the crowd-sourced worker executing an interaction with the label indicator, the interaction of a first type.

2. The method of claim 1 , wherein the sliding scale is configured to enable the crowd-sourced worker to select one of the label indicators.

3. The method of claim 1 , wherein the method further comprises generating one or more links each associated with a respective label indicator, and wherein providing an associated example of the past digital tasks with the label indicator in response to the crowd-sourced worker interacting with the associated one of the links.

4. The method of claim 1 , wherein the training label is assigned by training crowd-sourced workers in the computer-implemented crowd-sourced environment.

5. The method of claim 1 , wherein the determining the step is executed by the second MLA.

6. The method of claim 1 , wherein the step is visually represented in the sliding labelling scale as distance between samples in the subset of examples.

7. The method of claim 1 , wherein the step is indicative of illustrative closeness between the subset of examples.

8. A system for receiving a label for a digital task executed within a computer-implemented crowd-sourced environment, the digital task corresponding to a search engine result page (SERP) relevancy labelling task, the system comprising a server coupled communicatively to:

a SERP log database, the SERP log database comprising a plurality of SERPs previously generated;

the server comprising a processor configured to:

receive, an indication of the digital task to be processed in the computer-implemented crowd-sourced environment;

retrieve, from the SERP log database, the plurality of SERPs;

transmit the plurality of SERPs to a plurality of crowd-sourced worker in the computer-implemented crowd-sourced environment for evaluating a pertinence of each of the plurality of SERPs;

receive, from the plurality of crowd-sourced worker, a plurality of labels for each of the plurality of SERPs;

input, the plurality of labels into a first machine learning algorithm (MLA), configured to generate a set of unbiased labels corresponding to the plurality of labels with a bias perception of the plurality of crowd-sourced worker removed;

determine, a subset of examples comprising a subset of the plurality of SERPs and associated unbiased labels, each of the subset of examples being similar to the digital task within a pre-determined similarity threshold;

map, the plurality of SERPs into a scale of n-dimensional space based on the associated unbiased labels;

input, the n-dimensional scale into a second MLA configured to generate one or more clusters, each of the one or more clusters providing maximum benchmark coverage of the n-dimensional scale, each cluster comprising a minimum number of SERP, wherein the second MLA has previously been trained based on a training set, the training set including a pair of digital tasks and a training label indicative of usefulness of a first of the pair of digital tasks as an illustrative example for a second one of the pair of digital tasks;

associate the subset of examples to the digital task to be presented in the computer-implemented crowd-sourced environment;

cause the digital task to be presented on a computing device of at least one crowd-sourced worker in the computer-implemented crowd-sourced environment to solicit the label for the digital task;

determine a step corresponding to a distance between each of the one or more clusters; and

generate a sliding labelling scale, the sliding labelling scale having label indicators based on the subset of examples, selection of the label being done by the crowd-sourced worker executing an interaction with the label indicator, the interaction of a first type.