Systems and methods for calibrating user ratings

1. A non-transitory computer-readable storage medium comprising instructions to cause a computing device to perform a method for calibrating a rating weight assigned to a user, comprising:

assigning an expert rating to a content item;

receiving a rating of the content item submitted by the user;

adjusting a rating weight of the user based on a comparison between the rating submitted by the user and the expert rating; and

calculating an aggregate rating of the content item from a plurality of user-submitted ratings, wherein each of the user submitted ratings is submitted by a user having a respective rating weight, and wherein the aggregate rating is based upon the plurality of user submitted ratings and the respective rating weights;

wherein the aggregate rating is calculated using the following equation:

R

A

=

1

N

·

∑

i

=

1

N

⁢

⁢

W

i

·

∑

i

=

1

N

⁢

⁢

W

i

·

R

i

.

2. The computer-readable storage medium of claim 1 , wherein the rating weight of the user is increased if the rating submitted by the user is within a threshold of the expert rating.

3. The computer-readable storage medium of claim 1 , wherein the rating weight of the user is decreased if the rating submitted by the user differs from the expert rating by more than a threshold.

4. The computer-readable storage medium of claim 1 , wherein calculating the aggregate rating comprises weighting each of the user submitted ratings by a respective user rating weight.

5. The computer-readable storage medium of claim 4 , wherein the user submitted ratings are linearly scaled by the respective user rating weights.

6. The computer-readable storage medium of claim 4 , wherein the user submitted ratings are exponentially scaled by the respective user rating weights.

7. The computer-readable storage medium of claim 4 , wherein the user submitted ratings are logarithmically scaled by the respective user rating weights.

8. The computer-readable storage medium of claim 1 , wherein the expert rating assigned to the content item comprises a plurality of ratings in a plurality of different rating categories, wherein the rating submitted by the user comprises a plurality of user-submitted ratings in two or more of plurality of different rating categories, and wherein the adjusting the rating weight of the user comprises adjusting each of a plurality of rating weights for each of the two or more different rating categories rated by the user.

9. The computer-readable storage medium of claim 1 , further comprising providing an interface to display content submitted by the user, wherein the interface includes an indication of the rating weight of the user.

10. The computer-readable storage medium of claim 1 , further comprising providing an interface to display a user profile of the user, wherein the interface includes an indication of the rating weight of the user.

11. The computer-readable storage medium of claim 1 , further comprising providing an interface to display the content item, wherein the interface comprises a content item rating input, and wherein the interface is configured to display an aggregate rating of the content item after a rating of the content item has been submitted through the content rating input.

12. The computer-readable storage medium of claim 1 , further comprising providing an interface to display the content item, wherein the interface comprises a content item rating input and an indicator to display an aggregate rating of the content item.

13. The computer-readable storage medium of claim 12 , wherein the indicator is configured to display the aggregate rating when a user of the interface has submitted a rating of the content item, and wherein the indicator is configured to display the aggregate rating of the content item and the rating submitted by the user of the interface.

14. The computer-readable storage medium of claim 1 , further comprising designating one or more of a plurality of users as expert raters, and wherein assigning an expert rating to the content item comprises receiving a rating of the content item submitted by one or more of the expert raters.

15. The computer-readable storage medium of claim 14 , wherein two or more expert raters submit a rating of the content item, and wherein the expert rating of the content item is an average of the two or more ratings submitted by the two or more expert raters.

16. The computer-readable storage medium of claim 15 , wherein each of the users designated as expert raters has a respective expert rating weight, and wherein the expert rating of the content item is a weighted average of the two or more ratings submitted by the two or more expert raters.

17. The computer-readable storage medium of claim 1 , wherein the expert rating is assigned after the user submits the user rating.

18. A system for calibrating a rating weight assigned to a user, comprising:

a computing device comprising a processor;

a content manager operable on the processor and comprising a content item assigned an expert rating; and

a user manager operable on the processor and communicatively coupled to the content manager, wherein the user manager is configured to adjust a rating weight of a user based on a comparison between the expert rating and a rating of the content item submitted by the user;

wherein the content manager is configured to calculate an aggregate rating of the content item from a plurality of user-submitted ratings, wherein each of the user submitted ratings is submitted by a user having a respective rating weight, and wherein the aggregate rating is based on the plurality of user submitted ratings and the respective rating weights; and

wherein the aggregate rating is calculated using the following equation:

R

A

=

1

N

·

∑

i

=

1

N

⁢

⁢

W

i

·

∑

i

=

1

N

⁢

⁢

W

i

·

R

i

.

19. The system of claim 18 , wherein the rating weight of the user is increased if the rating submitted by the user is within a first threshold of the expert rating, is decreased if the rating submitted by the user differs from the expert rating by more than a second threshold, and is unchanged otherwise.

20. The system of claim 18 , wherein the rating weight is adjusted proportionally to a proximity of the user submitted rating to the expert rating.

21. The system of claim 18 , wherein the content manager is configured to provide an interface to display content submitted by the user, and wherein the interface includes an indication of the rating weight of the user.

22. A computer-implemented method for calibrating a rating weight assigned to a user, the method being performed by a processor executing instructions stored in a memory, the method comprising:

assigning an expert rating to a content item;

receiving a rating of the content item submitted by the user;

adjusting the rating weight of the user based upon a comparison of the expert rating and the rating submitted by the user, wherein the rating weight is incremented if the rating submitted by the user is within a first threshold of the expert rating, the rating weight is decremented if the rating submitted by the user differs from the expert rating by more than a second threshold, and is unchanged otherwise;

providing an interface to display content submitted by the user, the interface comprising an indication of the rating weight of the user; and

calculating an aggregate rating of the content item from a plurality of user-submitted ratings, wherein each of the user submitted ratings is submitted by a user having a respective rating weight, and wherein the aggregate rating is based on the plurality of user submitted ratings and the respective rating weights;

wherein the aggregate rating is calculated using the following equation:

R

A

=

1

N

·

∑

i

=

1

N

⁢

⁢

W

i

·

∑

i

=

1

N

⁢

⁢

W

i

·

R

i

.