Efficient similarity detection
Embodiments disclosed herein generally relate to a method for uploading a file to a server. The server receives a request from a user to upload a file thereto. An upload client extracts file information from the file. The server generates a file signature for the file. The server accesses one or more existing file signatures for each of one or more existing files on the server. The server determines whether any of the one or more existing file signatures are similar to the file signature. Responsive to determining that there is an existing file signature that is similar to the file signature, the server accesses a first hash signature for the existing file. The upload client generates a second hash signature for the file. Responsive to determining that the first hash signature does not equal the second hash signature, the upload client uploads the file to the server.
1. A method for detecting similar files, the method comprising:
storing a plurality of existing file signatures in memory associated with a server, each of the existing file signatures being uniquely mapped to a corresponding media file;
receiving a request sent over a communication network from a user device to the server, the request concerning upload of a file to the server; and
executing instructions stored in memory of the server, wherein the instructions are executed by a processor of the server to:
generate a file signature for the file based on metadata associated with the file, wherein the file signature is different from a hash signature;
compare the generated file signature to one or more of the existing file signatures stored in the memory;
compile a list of one or more potential matches from among the one or more existing file signatures, wherein each of the potential matches is determined to be similar to the generated file signature;
perform a visual similarity analysis between the requested file and one or more media files associated with the potential matches; and
generate a curation suggestion for the requested file based on results of the visual similarity analysis, wherein the curation suggestion includes an upload location for the requested file based on respective locations of the media files associated with the potential matches.
2. The method of claim 1 , wherein the request is a curation request associated with a plurality of files presented for upload, and wherein the requested file is one of the plurality of presented files associated with the curation request.
3. The method of claim 1 , wherein the curation suggestion includes assignment of the requested file to one of a plurality of available web pages.
4. The method of claim 1 , further comprising transmitting an authorization to upload the requested file, wherein the authorization is sent to an upload client at the user device.
5. The method of claim 1 , further comprising identifying the potential matches by determining that each of the potential matches is within a predetermined deviation from the generated file signature.
6. The method of claim 1 , further comprising comparing a hash signature of the requested file to one or more hash signatures of the media files associated with the potential matches.
7. The method of claim 6 , wherein identifying that one of the hash signatures of one of the potential matches is similar to the hash signature of the requested file based on the potential match hash signature being within a predetermined deviation from the hash signature of the requested file.
8. The method of claim 1 , wherein the visual similarity analysis includes analyzing one or more of backdrops, subjects, locations, and pixels.
9. The method of claim 1 , wherein the curation suggestion includes generating a new online media gallery for the requested file.
10. The method of claim 1 , wherein the curation suggestion includes a category for the requested file, and further comprising uploading the file in accordance with the category.
11. A system for detecting similar files, the system comprising:
memory that stores a plurality of existing file signatures, each of the existing file signatures being uniquely mapped to a corresponding media file;
a communication interface that receives a request sent over a communication network from a user device, the request concerning upload of a file; and
a processor that executes instructions stored in memory, wherein the processor executes the instructions to:
generate a file signature for the file based on metadata associated with the file, wherein the file signature is different from a hash signature;
compare the generated file signature to one or more of the existing file signatures stored in the memory;
compile a list of one or more potential matches from among the one or more existing file signatures, wherein each of the potential matches is determined to be similar to the generated file signature;
perform a visual similarity analysis between the requested file and one or more media files associated with the potential matches; and
generate a curation suggestion for the requested file based on results of the visual similarity analysis, wherein the curation suggestion includes an upload location for the requested file based on respective locations of the media files associated with the potential matches.
12. The system of claim 11 , wherein the request is a curation request associated with a plurality of files presented for upload, and wherein the requested file is one of the plurality of presented files associated with the curation request.
13. The system of claim 11 , wherein the curation suggestion includes assignment of the requested file to one of a plurality of available web pages.
14. The system of claim 11 , wherein the communication interface further transmits an authorization to upload the requested file, wherein the authorization is sent to an upload client at the user device.
15. The system of claim 11 , wherein the processor executes further instructions to identify the potential matches by determining that each of the potential matches is within a predetermined deviation from the generated file signature.
16. The system of claim 11 , wherein the processor executes further instructions to compare a hash signature of the requested file to one or more hash signatures of the media files associated with the potential matches.
17. The system of claim 16 , wherein the processor identifies that one of the hash signatures of one of the potential matches is similar to the hash signature of the requested file based on the potential match hash signature being within a predetermined deviation from the hash signature of the requested file.
18. The system of claim 11 , wherein the visual similarity analysis includes analyzing one or more of backdrops, subjects, locations, and pixels.
19. The system of claim 11 , wherein the curation suggestion includes generating a new online media gallery for the requested file.
20. The system of claim 11 , wherein the curation suggestion includes a category for the requested file, and further comprising uploading the file in accordance with the category.
21. A non-transitory computer-readable storage medium having instructions stored thereon, the instructions executable by a processor, to perform a method of detecting similar files, the method comprising:
storing a plurality of existing file signatures in memory associated with a server, each of the existing file signatures being uniquely mapped to a corresponding media file;
receiving a request sent over a communication network from a user device to the server, the request concerning upload of a file to the server;
generating a file signature for the file based on metadata associated with the file, wherein the file signature is different from a hash signature;
comparing the generated file signature to one or more of the existing file signatures stored in the memory;
compiling a list of one or more potential matches from among the one or more existing file signatures, wherein each of the potential matches is determined to be similar to the generated file signature;
performing a visual similarity analysis between the requested file and one or more media files associated with the potential matches; and
generating a curation suggestion for the requested file based on results of the visual similarity analysis, wherein the curation suggestion includes an upload location for the requested file based on respective locations of the media files associated with the potential matches.