Deduplication of media files

1. A method for identifying visually similar media content items from media content items that have been indicated as being candidate matches by being grouped together by identical first segment of a perceptual hash, in a key value database, the method comprising:

receiving a new media content item;

determining at least one perceptual hash for one or more video frames of the new media content item;

partitioning the at least one perceptual hash for the new media content item into segments including a first segment and a second segment;

identifying a group of potential match media content items for the new media content item by comparing the first segment of the at least one perceptual hash for the new media content item with the key value database;

comparing the perceptual hashes in the group of potential match media content items with the at least one perceptual hash for the new media content item; and

identifying a subset of the group of potential match media content items as matches for the new media content item, based on the comparison between the perceptual hashes of the group of potential match media content items with the at least one perceptual hash for the new media content item.

2. The method of claim 1 , wherein the comparing the perceptual hashes in the group of potential match media content items with the at least one perceptual hash for the new media content item comprises:

determining distances between the perceptual hashes in the group of potential match media content items and the at least one perceptual hash for the new media content item.

3. The method of claim 2 , further comprising:

based on a distance between the perceptual hash of a particular media content item in the group of potential match media content items and the at least one perceptual hash for the new media content item satisfying a distance threshold, including the particular media content item in the subset of the group of potential match media content items.

4. The method of claim 1 , wherein the media content items are stored within a media content item library.

5. The method of claim 1 , further comprising:

ranking the subset of the group of potential match media content items according to at least one factor.

6. The method of claim 1 , wherein media content items that have been matched as duplicates are marked as such within a search index, such that media content items marked as duplicates are not included in the subset of the group of potential match media content items.

7. The method of claim 5 , wherein the subset of the group of potential match media content items are ranked according to resolution or quality.

8. A non-transitory computer readable storage medium having computer readable program code stored thereon for causing a computer system to perform a method for identifying visually similar media content items from media content items that have been indicated as being candidate matches by being grouped together by identical first segment of a perceptual hash, in a key value database, the method comprising:

receiving a new media content item;

determining at least one perceptual hash for one or more video frames of the new media content item;

partitioning the at least one perceptual hash for the new media content item into segments including a first segment and a second segment;

identifying a group of potential match media content items for the new media content item by comparing the first segment of the at least one perceptual hash for the new media content item with the key value database;

comparing the perceptual hashes in the group of potential match media content items with the at least one perceptual hash for the new media content item; and

identifying a subset of the group of potential match media content items as matches for the new media content item, based on the comparison between the perceptual hashes of the group of potential match media content items with the at least one perceptual hash for the new media content item.

9. The non-transitory computer readable storage medium of claim 8 , wherein the comparing the perceptual hashes in the group of potential match media content items with the at least one perceptual hash for the new media content item comprises:

determining distances between the perceptual hashes in the group of potential match media content items and the at least one perceptual hash for the new media content item.

10. The non-transitory computer readable storage medium of claim 9 , wherein the method further comprises:

based on a distance between the perceptual hash of a particular media content item in the group of potential match media content items and the at least one perceptual hash for the new media content item satisfying a distance threshold, including the particular media content item in the subset of the group of potential match media content items.

11. The non-transitory computer readable storage medium of claim 8 , wherein the media content items are stored within a media content item library.

12. The non-transitory computer readable storage medium of claim 8 , the method further comprising:

ranking the subset of the group of potential match media content items according to at least one factor.

13. The non-transitory computer readable storage medium of claim 8 , wherein media content items that have been matched as duplicates are marked as such within a search index, such that media content items marked as duplicates are not included in the subset of the group of potential match media content items.

14. The non-transitory computer readable storage medium of claim 12 , wherein the subset of the group of potential match media content items are ranked according to resolution or quality.

15. A computer system comprising:

a data storage unit; and

a processor coupled with the data storage unit, the processor configured to perform a method for identifying visually similar media content items from media content items that have been indicated as being candidate matches by being grouped together by identical first segment of a perceptual hash, in a key value database, the method comprising:

receiving a new media content item;

determining at least one perceptual hash for one or more video frames of the new media content item;

partitioning the at least one perceptual hash for the new media content item into segments including a first segment and a second segment;

identifying a group of potential match media content items for the new media content item by comparing the first segment of the at least one perceptual hash for the new media content item with the key value database;

comparing the perceptual hashes in the group of potential match media content items with the at least one perceptual hash for the new media content item; and

identifying a subset of the group of potential match media content items as matches for the new media content item, based on the comparison between the perceptual hashes of the group of potential match media content items with the at least one perceptual hash for the new media content item.

16. The computer system of claim 15 , wherein the comparing the perceptual hashes in the group of potential match media content items with the at least one perceptual hash for the new media content item comprises:

determining distances between the perceptual hashes in the group of potential match media content items and the at least one perceptual hash for the new media content item.

17. The computer system of claim 16 , wherein the comparing the perceptual hashes in the group of potential match media content items with the at least one perceptual hash for the new media content item comprises:

determining distances between the perceptual hashes in the group of potential match media content items and the at least one perceptual hash for the new media content item.

18. The computer system of claim 15 , wherein media content items that have been matched as duplicates are marked as such within a search index, such that media content items marked as duplicates are not included in the subset of the group of potential match media content items.

19. The computer system of claim 15 , the method further comprising:

ranking the subset of the group of potential match media content items according to at least one factor.

20. The computer system of claim 19 , wherein the subset of the group of potential match media content items are ranked according to resolution or quality.