Providing a result with a requested accuracy using individuals previously acting with a consensus
View Patent ↗A result for a task may be provided in response to receiving a request from, for example, a user for the result. The request specifies a desired accuracy level for the result. The accuracy of the result is determined using the individual accuracies of one or more persons that have selected the result. Each person's individual accuracy is determined based on results for prior tasks previously performed by that person. The person's individual accuracy is proportional to the number of that person's prior results that are with a consensus of other persons that have performed the same prior task. The result is provided if the accuracy of the result is equal to or greater than the desired accuracy level.
1. A computer-implemented method to provide a result for a task, the method comprising:
receiving, via a computing device, a request for the result, the request specifying a desired accuracy level;
determining, via the computing device, an accuracy for the result using an individual accuracy for each of one or more persons selecting the result, wherein each respective individual accuracy is determined based on prior results to prior tasks previously performed by the respective person, each result for a respective prior task being either with or against a consensus, and the individual accuracy being proportional to the number of prior results that are with the consensus; and
providing, via the computing device, the result when the accuracy of the result is equal to or greater than the desired accuracy level.
2. The method of claim 1 , wherein the task is the selection of one of a plurality of choices, and the result is a selected one of the choices.
3. The method of claim 2 , wherein the selection comprises reviewing titles presented on a display of a user device, the choices are a list of potential titles for a content item, and the result is a single title selected from the list of titles.
4. The method of claim 1 , further comprising paying a fee to each of the plurality of persons for selecting the result.
5. The method of claim 1 , wherein the task is the selection of a keyword associated with text, and the result is the selected keyword.
6. The method of claim 5 , wherein the text is a title for a content item.
7. The method of claim 1 , wherein determining the accuracy comprises determining a first accuracy using the individual accuracy for each of the one or more persons, and the method further comprising, when the first accuracy is below the desired accuracy level, determining a second accuracy using the individual accuracy of each of the one or more persons along with an individual accuracy of an additional person.
8. The method of claim 1 , further comprising repeatedly improving the accuracy by using an individual accuracy of at least one additional person until the accuracy is equal to or greater than the desired accuracy level.
9. The method of claim 8 , wherein repeatedly improving the accuracy is stopped when a predetermined maximum number of persons is reached.
10. The method of claim 8 , further comprising:
associating a cost with a selection of a result by each of the one or more persons and each at least one additional person, a sum of the costs for the one or more persons and the at least one additional person being a total cost; and
stopping the repeatedly improving the accuracy when the total cost reaches a predetermined maximum.
11. The method of claim 8 , wherein the task is the selection of one of a number of choices, and the result is a selected one of the choices, the method further comprising reducing the number of choices presented to the at least one additional person, the reduced number of choices having been previously selected by the one or more persons.
12. The method of claim 1 , further comprising storing the individual accuracy of each of the one or more persons as a plurality of records in at least one storage device.
13. The method of claim 12 , further comprising creating the plurality of records prior to receiving the request, the creating comprising:
sending a survey to a plurality of feedback devices, the survey including a plurality of data items and each feedback device being operated by a respective one of the one or more persons;
receiving feedback from each respective feedback device, the feedback associating an activity at each respective feedback device with at least one of the data items;
determining, based on the feedback, a consensus for the survey; and
creating the plurality of records, each record corresponding to a unique one of the one or more persons, and each record indicating whether an activity of a feedback device operated by that person is with the consensus.
14. The method of claim 13 , wherein each data item is a title, and the feedback is a selected title.
15. The method of claim 1 , further comprising receiving a fee that corresponds to the desired accuracy level, the fee being greater for a higher desired accuracy level.
16. The method of claim 1 , further comprising ceasing to use the individual accuracy of a first person of the one or more persons in any further accuracy determination when the individual accuracy of the first person falls below a minimum individual accuracy threshold.
17. The method of claim 2 , further comprising, when the accuracy of the result is less than the desired accuracy level, discarding the plurality of choices and determining an accuracy for a result selected from another plurality of choices.
18. The method of claim 17 , wherein the plurality of choices is an initial list of potential titles for a content item, and the another plurality of choices is a new list of potential titles for the content item, the new list being different by at least one title from the initial list.
19. A computer readable medium having stored thereon a set of instructions, which when executed by a computer having a processor and a memory, cause the computer to perform operations to provide a result for a task, the operations comprising:
receiving a request for the result, the request specifying a desired accuracy level;
determining, via the computer, an accuracy for the result using an individual accuracy for each of one or more persons selecting the result, wherein each respective individual accuracy is determined based on prior results to prior tasks previously performed by the respective person, each result for a respective prior task being either with or against a consensus, and the individual accuracy being proportional to the number of prior results that are with the consensus; and
providing the result when the accuracy of the result is equal to or greater than the desired accuracy level.
20. A system to provide a result for a task, the system comprising:
a processor programmed to:
receive a request for the result, the request specifying a desired accuracy level,
determine an accuracy for the result using an individual accuracy for each of one or more persons selecting the result, wherein each respective individual accuracy is determined based on prior results to prior tasks previously performed by the respective person, each result for a respective prior task being either with or against a consensus, and the individual accuracy being proportional to the number of prior results that are with the consensus, and
provide the result when the accuracy of the result is equal to or greater than the desired accuracy level; and
a storage device to store the individual accuracy of each of the one or more persons.