Near-duplicate filtering in search engine result page of an online shopping system

1. A computer-implemented method to reduce same merchant near-duplicate entries in online shopping system search results, comprising:

for each pair of entries in a set of entries from the same merchant, each entry characterizing a product in a data store of an online shopping system and each entry characterized by a set of quantified attributes, determining, by one or more computing devices, a distance between the entries in the pair in a vector space of the quantified attributes;

determining, by the one or more computing devices, clusters of entries as a function of the determined distance between each pair of entries;

receiving, by the one or more computing devices, a query directed to the data store; and

returning, by the one or more computing devices, an ordered list of results responsive to the query from the data store of an online shopping system, filtered as a function of the determined distance to reduce the number of near duplicate entries from the same entry in the search results, wherein filtering as a function of the determined distance comprises at least one of:

limiting the number of entries from a given cluster to a predetermined threshold number of entries from the given cluster; and

after including in the ordered list a first entry from a given cluster of entries, excluding entries within a predetermined first threshold distance of the first entry.

2. The method of claim 1 , wherein determining a distance between the entries in the pair based on the quantified attributes comprises determining a weighted sum of an edit distance between the quantified attributes of entries of the pair.

3. The method of claim 2 , wherein the edit distance is one of: a Hamming distance, a Levenshtein distance, a Damerau-Levenshtein distance, and a Jaro-Winkler distance.

4. The method of claim 1 , wherein determining clusters of entries comprises identifying as clusters groups of entries related to another entry by a determined distance of less than a predetermined first second threshold distance.

5. The method of claim 4 , wherein the distance is normalized on an interval from 0 to 1, and the predetermined second threshold distance is approximately 0.05.

6. The method of claim 1 , further comprising:

for at least one result in the ordered list, returning, by the one or more computing devices, a link: which, when selected, prompts as a response from the online shopping system an ordered list of products in the same cluster as the at least one result.

7. A computer program product, comprising:

a non-transitory computer-readable storage device having computer-executable program instructions embodied thereon that when executed by a computer cause the computer to reduce same merchant near-duplicate entries in online shopping system search results, the computer-executable program instructions comprising:

computer-executable program instructions to determine, for each pair of entries from the same merchant in a set of entries, each entry characterizing a product in a data store of an online shopping system and each entry characterized by a set of quantified attributes, a distance between the entries in the pair in a vector space of the quantified attributes;

computer-executable program instructions to determine clusters of entries from a graph formed with each determined distance as an edge between nodes representing the entries used to determine the corresponding distance, each entry cluster identified by a cluster identifier;

computer-executable program instructions to receive a query directed to the data store; and

computer-executable program instructions to return an ordered list of results responsive to the query from the data store of an online shopping system, filtered as a function of the determined distance to reduce the number of near duplicate entries from the same entry in the search results, wherein filtering as a function of the determined distance comprises at least one of:

limiting the number of entries from a given cluster to a predetermined threshold number of entries from the given cluster; and

after including in the ordered list a first entry from a given cluster of entries, excluding entries within a predetermined first threshold distance of the first entry.

8. The computer program product of claim 7 , wherein determining a distance between the entries in the pair based on the quantified attributes comprises determining a weighted sum of an edit distance between the quantified attributes of entries of the pair.

9. The computer program product of claim 8 , wherein the edit distance is one of: a Hamming distance, a Levenshtein distance, a Damerau-Levenshtein distance, and a Jaro-Winkler distance.

10. The computer program product of claim 7 , wherein determining clusters of entries comprises identifying as clusters groups of entries related to another entry by a determined distance of less than a predetermined second threshold distance.

11. The computer program product of claim 10 , wherein the distance is normalized on an interval from 0 to 1, and the predetermined second threshold distance is approximately 0.05.

12. The computer program product of claim 7 , further comprising:

for at least one result in the ordered list, returning, by the one or more computing devices, a link: which, when selected, prompts as a response from the online shopping system an ordered list of products in the same cluster as the at least one result.

13. A system to reduce same merchant near-duplicate entries in online shopping system search results, comprising:

a storage device; and

a processor communicatively coupled to the storage device, wherein the processor executes application code instructions that are stored in the storage device to cause the system to:

determine, for each pair of entries in a set of entries from the same merchant, each entry characterizing a product in a data store of an online shopping system and each entry characterized by a set of quantified attributes, a distance between the entries in the pair in a vector space of the quantified attributes;

determine clusters of entries from a graph formed with each determined distance as an edge between nodes representing the entries used to determine the corresponding distance, each entry cluster identified by a cluster identifier;

receive a query directed to the data store; and

return an ordered list of results responsive to the query from the data store of an online shopping system, filtered as a function of the determined distance to reduce the number of near duplicate entries from the same entry in the search results, wherein filtering as a function of the determined distance comprises at least one of:

limiting the number of entries from a given cluster to a predetermined threshold number of entries from the given cluster; and

after including in the ordered list a first entry from a given cluster of entries, excluding entries within a predetermined first threshold distance of the first entry.

14. The computer program product of claim 13 , wherein determining a distance between the entries in the pair based on the quantified attributes comprises determining a weighted sum of an edit distance between the quantified attributes of entries of the pair.

15. The system of claim 14 , wherein the edit distance is one of: a Hamming distance, a Levenshtein distance, a Damerau-Levenshtein distance, and a Jaro-Winkler distance.

16. The system of claim 13 , wherein determining clusters of entries comprises identifying as clusters groups of entries related to another entry by a determined distance of less than a predetermined second threshold distance.

17. The system of claim 16 , wherein the distance is normalized on an interval from 0 to 1, and the predetermined second threshold distance is approximately 0.05.

18. The system of claim 13 , further comprising:

for at least one result in the ordered list, returning, by one or more computing devices, a link which, when selected, prompts as a response from the online shopping system an ordered list of products in the same cluster as the at least one result.