Systems and methods for matching products in the absence of unique identifiers
Systems and methods including one or more processing modules and one or more non-transitory storage modules storing computing instructions configured to run on the one or more processing modules and perform acts of retrieving a first product offer from a first seller for a first product to offer for sale on a website of an online retailer, determining if the first product offer for the first product matches one or more existing candidate product offers in a catalog of the online retailer using one or more sets of rules, and, if the first product offer matches the one or more existing candidate product offers, coordinating displaying at least a portion of the first product offer on the website of the online retailer.
1. A system comprising:
one or more processors; and
one or more non-transitory storage devices storing computing instructions configured to run on the one or more processors and perform:
retrieving a first product offer from a first seller for a first product to offer for sale on a website of an online retailer, the first product offer comprising one or more of:
a first title of the first product;
a first price of the first product; or
a first image of the first product;
determining when the first product offer for the first product matches one or more existing candidate product offers in a catalog of the online retailer using:
each of a modified token edit distance algorithm, a term frequency-inverse document frequency (tf-idf) algorithm, and a word embedding algorithm to determine a respective title similarity between the first title of the first product from the first product offer and a respective existing candidate title of each respective existing candidate product offer of the one or more existing candidate product offers;
a second set of rules to determine when the first price of the first product from the first product offer is a respective price outlier relative to a respective existing candidate price of each respective existing candidate product offer of the one or more existing candidate product offers; and
a third set of rules to determine when a respective image similarity of the first image from the first product offer meets a respective predetermined image similarity threshold relative to a respective existing candidate image of each respective existing candidate product offer of the one or more existing candidate product offers;
when the respective title similarity comprises a first output of the modified token edit distance algorithm that is smaller than a second output of the word embedding algorithm and a third output of the tf-idf algorithm that is smaller than the first output of the modified token edit distance algorithm:
determining that the first product offer matches at least one respective existing candidate product offer of the one or more existing candidate product offers; and
in response to determining that the first product offer matches the at least one respective existing candidate product offer of the one or more existing candidate product offers, coordinating displaying at least a portion of the first product offer on the website of the online retailer; and
when the respective title similarity does not comprises the first output of the modified token edit distance algorithm that is smaller than the second output of the word embedding algorithm and the third output of the tf-idf algorithm that is smaller than the first output of the modified token edit distance algorithm:
rejecting the first product offer; and
prompting a first electronic device of the first seller to correct the first product offer, wherein the modified token edit distance algorithm comprises:
tokenizing the first title and the respective existing candidate title;
determining a sequence alignment of a smaller title of (1) the first title and (2) the respective existing candidate title within a larger title of (1) the first title and (2) the respective existing candidate title; and
determining, using the sequence alignment, a Levenshtein distance between the smaller title and a smallest sequence in the larger title.
2. The system of claim 1 , wherein the first product offer comprises a first new product offer that has not previously been offered for sale on the website of the online retailer.
3. The system of claim 1 , wherein the first product offer comprises a first existing product offer.
4. The system of claim 1 , wherein the modified token edit distance algorithm further comprises:
removing non-alphanumeric characters from the first title and the respective existing candidate title;
sorting one or more tokens, as tokenized for each of the first title and the respective existing candidate title; and
converting the Levenshtein distance to a similarity score between 0 and 1.
5. The system of claim 1 , wherein the second set of rules comprises at least one of a Dixon's Q test algorithm, a Grubbs' test algorithm, or a chi-square test algorithm.
6. The system of claim 1 , wherein the third set of rules comprises at least one of a VGG deep learning algorithm, an inception deep learning algorithm, or a Resnet deep learning algorithm.
7. The system of claim 1 , wherein determining when the first product offer for the first product matches the one or more existing candidate product offers comprises:
determining when the first product offer matches the one or more existing candidate product offers in the catalog of the online retailer using one or more of:
a fourth set of rules to determine an attribute similarity between (1) at least one attribute of the first product from the first product offer and (2) one or more existing candidate attributes of the one or more existing candidate product offers; or
a fifth set of rules to determine when a first product classification of the first product of the first product offer matches an existing product classification of the one or more existing candidate product offers.
8. The system of claim 1 , wherein:
the first product offer comprises at least one of (1) a first new product offer that has not previously been offered for sale on the website of the online retailer or (2) a first existing product offer;
the modified token edit distance algorithm further comprises:
removing non-alphanumeric characters from the first title and the respective existing candidate title;
sorting one or more tokens as tokenized for each of the first title and the respective existing candidate title; and
converting the Levenshtein distance to a similarity score between 0 and 1;
the second set of rules comprises at least one of a Dixon's Q test algorithm, a Grubbs' test algorithm, or a chi-square test algorithm;
the third set of rules comprises at least one of a VGG deep learning algorithm, an inception deep learning algorithm, or a Resnet deep learning algorithm; and
determining when the first product offer for the first product matches the one or more existing candidate product offers comprises:
determining when the first product offer matches the one or more existing candidate product offers in the catalog of the online retailer using:
a fourth set of rules to determine an attribute similarity between (1) at least one attribute of the first product from the first product offer and (2) one or more existing candidate attributes of the one or more existing candidate product offers; and
a fifth set of rules to determine when a first product classification of the first product of the first product offer matches an existing product classification of the one or more existing candidate product offers.
9. The system of claim 7 , wherein a system administrator chooses whether to use the fourth set of rules, the fifth set of rules, or both the fourth set of rules and the fifth set of rules.
10. The system of claim 1 , wherein the computing instructions are further configured to run on the one or more processors and perform:
when the respective title similarity comprises: (a) the first output of the modified token edit distance algorithm that is larger than the second output of the word embedding algorithm or (b) the third output of the tf-idf algorithm that is larger than the first output of the modified token edit distance algorithm, determining that the first product offer does not match the at least one respective existing candidate product offer of the one or more existing candidate product offers; and
removing the at least the portion of the first product offer from displaying on the website of the online retailer.
11. A method being implemented via execution of computing instructions configured to run at one or more processors and configured to be stored at non-transitory computer-readable media, the method comprising:
retrieving a first product offer from a first seller for a first product to offer for sale on a website of an online retailer, the first product offer comprising one or more of:
a first title of the first product;
a first price of the first product; or
a first image of the first product;
determining when the first product offer for the first product matches one or more existing candidate product offers in a catalog of the online retailer using:
each of a modified token edit distance algorithm, a term frequency-inverse document frequency (tf-idf) algorithm, and a word embedding algorithm to determine a respective title similarity between the first title of the first product from the first product offer and a respective existing candidate title of each respective existing candidate product offer of the one or more existing candidate product offers;
a second set of rules to determine when the first price of the first product from the first product offer is a respective price outlier relative to a respective existing candidate price of each respective existing candidate product offer of the one or more existing candidate product offers; and
a third set of rules to determine when a respective image similarity of the first image from the first product offer meets a respective predetermined image similarity threshold relative to a respective existing candidate image of each respective existing candidate product offer of the one or more existing candidate product offers;
when the respective title similarity comprises a first output of the modified token edit distance algorithm that is smaller than a second output of the word embedding algorithm and a third output of the tf-idf algorithm that is smaller than the first output of the modified token edit distance algorithm:
determining that the first product offer matches at least one respective existing candidate product offer of the one or more existing candidate product offers; and
in response to determining that the first product offer matches the at least one respective existing candidate product offer of the one or more existing candidate product offers, coordinating displaying at least a portion of the first product offer on the website of the online retailer; and
when the respective title similarity does not comprises the first output of the modified token edit distance algorithm that is smaller than the second output of the word embedding algorithm and the third output of the tf-idf algorithm that is smaller than the first output of the modified token edit distance algorithm:
rejecting the first product offer; and
prompting a first electronic device of the first seller to correct the first product offer, wherein the modified token edit distance algorithm comprises:
tokenizing the first title and the respective existing candidate title;
determining a sequence alignment of a smaller title of (1) the first title and (2) the respective existing candidate title within a larger title of (1) the first title and (2) the respective existing candidate title; and
determining, using the sequence alignment, a Levenshtein distance between the smaller title and a smallest sequence in the larger title.
12. The method of claim 11 , wherein the first product offer comprises a first new product offer that has not previously been offered for sale on the website of the online retailer.
13. The method of claim 11 , wherein the first product offer comprises a first existing product offer.
14. The method of claim 11 , wherein the modified token edit distance algorithm further comprises:
removing non-alphanumeric characters from the first title and the respective existing candidate title;
sorting one or more tokens, as tokenized for each of the first title and the respective existing candidate title; and
converting the Levenshtein distance to a similarity score between 0 and 1.
15. The method of claim 11 , wherein the second set of rules comprises at least one of a Dixon's Q test algorithm, a Grubbs' test algorithm, or a chi-square test algorithm.
16. The method of claim 11 , wherein the third set of rules comprises at least one of a VGG deep learning algorithm, an inception deep learning algorithm, or a Resnet deep learning algorithm.
17. The method of claim 11 , wherein determining when the first product offer for the first product matches the one or more existing candidate product offers comprises:
determining when the first product offer matches the one or more existing candidate product offers in the catalog of the online retailer using one or more of:
a fourth set of rules to determine an attribute similarity between (1) at least one attribute of the first product from the first product offer and (2) one or more existing candidate attributes of the one or more existing candidate product offers; or
a fifth set of rules to determine when a first product classification of the first product of the first product offer matches an existing product classification of the one or more existing candidate product offers.
18. The method of claim 11 , wherein:
the first product offer comprises at least one of (1) a first new product offer that has not previously been offered for sale on the website of the online retailer or (2) a first existing product offer;
the modified token edit distance algorithm further comprises:
removing non-alphanumeric characters from the first title and the respective existing candidate title;
sorting one or more tokens as tokenized for each of the first title and the respective existing candidate title; and
converting the Levenshtein distance to a similarity score between 0 and 1;
the second set of rules comprises at least one of a Dixon's Q test algorithm, a Grubbs' test algorithm, or a chi-square test algorithm;
the third set of rules comprises at least one of a VGG deep learning algorithm, an inception deep learning algorithm, or a Resnet deep learning algorithm; and
determining when the first product offer for the first product matches the one or more existing candidate product offers comprises:
determining when the first product offer matches the one or more existing candidate product offers in the catalog of the online retailer using:
a fourth set of rules to determine an attribute similarity between (1) at least one attribute of the first product from the first product offer and (2) one or more existing candidate attributes of the one or more existing candidate product offers; and
a fifth set of rules to determine when a first product classification of the first product of the first product offer matches an existing product classification of the one or more existing candidate product offers.
19. The method of claim 17 , wherein a system administrator chooses whether to use the fourth set of rules, the fifth set of rules, or both the fourth set of rules and the fifth set of rules.
20. The method of claim 11 further comprising:
when the respective title similarity comprises: (a) the first output of the modified token edit distance algorithm that is larger than the second output of the word embedding algorithm or (b) the third output of the tf-idf algorithm that is larger than the first output of the modified token edit distance algorithm, determining that the first product offer does not match the at least one respective existing candidate product offer of the one or more existing candidate product offers; and
removing the at least the portion of the first product offer from displaying on the website of the online retailer.