Methods for prediction and rating aggregation
View Patent ↗A method determines forecaster biases based on the correlations between prediction errors made by a group of forecasters. The method measures forecaster skills in different areas of expertise based on the correlations between the absolute values of prediction errors made by a group of forecasters. The method uses bias and skill measurements to raise the precision of aggregate forecasts. The method also reduces bias in social network rating systems by linking the ratings with objectively verifiable predictions.
1. A computer implemented method, comprising: obtaining individual estimates for a set of numerical parameters from a plurality of information sources, wherein an information source may be a person, a group, an organization or any other entity that can provide information; comparing at least some of the estimates to the correct values of at least some of the parameters; decomposing estimate errors into generalized products of two vectors where the first vector measures the biases of the information sources and the second vector measures the sensitivity of the estimates to the biases of the information sources.
2. The method of claim 1 , further including: measuring sensitivity of the estimates to the biases of the information sources.
3. The method of claim 2 , further including: calculating aggregate estimates for the parameters based on the individual estimates, the biases of the information sources and the sensitivity of the estimates to the biases of the information sources.
4. The method of claim 3 , further including: measuring differences between the individual and aggregate estimates for parameters whose correct values are unknown; adjusting the bias values based on the measured differences.
5. The method of claim 3 , further including: measuring differences between the individual and aggregate estimates for parameters whose correct values are unknown; using the measured differences to score accuracy of the information sources.
6. The methods of claim 1 , further including: obtaining ratings for a plurality of items from a plurality of persons; calculating, based on the ratings and bias values of persons who provided the ratings, aggregate ratings for the items.
7. The method of claim 6 , further including: measuring differences between the individual and aggregate ratings; adjusting the bias values of the persons who provided the ratings based on the measured differences.
8. The methods of claim 1 , further including: obtaining ratings for a plurality of items from a plurality of persons; scoring person-item pairs based on the ratings and bias values of the persons who provided the ratings.