Systems and methods for modeling item similarity using converted image information
Systems and methods for correlating item data are disclosed. A system for correlating item data may include a memory storing instructions and at least one processor configured to execute instructions to perform operations including: receiving text and image data associated with a reference item from a remote device; converting, using a computer-modeled embedding layer, at least one image to an image embedding; comparing the image embedding to reference embeddings stored in a database; selecting a subset of the candidate item text as candidate text data based on the comparison; selecting a subset of the candidate item images as candidate image data based on the comparison; selecting a text correlation model; determining a first similarity score; selecting an image correlation model; determining a second similarity score; calculating a confidence score based on the first and second similarity scores; and performing a responsive action based on the calculated confidence score.
1. A system for correlating item data, the system comprising:
at least one processor; and
a non-transitory computer-readable medium containing a set of instructions that, when executed by the at least one processor, cause the processor to perform steps comprising:
receiving text data associated with a reference item;
receiving image data comprising at least one image associated with the reference item;
converting, using a computer-modeled embedding layer, the at least one image to an image embedding;
comparing the image embedding to reference embeddings, the stored reference embeddings being associated with candidate item images and candidate item text of candidate items;
selecting, based on the comparison, candidate text data, wherein the candidate text data is a subset of the candidate item text and includes a price;
selecting, based on the comparison, candidate image data, the candidate image data being a subset of the candidate item images;
selecting a text correlation model;
determining a first similarity score by applying the selected text correlation model to the received text data and the subset of the candidate item text;
selecting an image correlation model;
determining a second similarity score by applying the selected image correlation model to the received image data and the subset of the candidate item images;
calculating a confidence score based on the determined first and second similarity scores; and
performing a responsive action based on the calculated confidence score, wherein the responsive action comprises at least one of:
creating an association between the reference item and one of the candidate items;
monitoring a webpage associated with the reference item;
updating at least one of the reference embeddings;
modifying the selected text correlation model; or
modifying the selected image correlation model.
2. The system of claim 1 , wherein the stored reference embeddings are clustered in a Euclidean space.
3. The system of claim 1 , wherein the responsive action comprises updating at least one of the reference embeddings by performing at least one of:
adding a reference embedding to the stored reference embeddings;
removing a reference embedding from the stored reference embeddings; or
adjusting a Euclidean-space boundary associated with the stored reference embeddings.
4. The system of claim 1 , wherein the received text data is crawled from a web page by a text crawler configured to search for item text data associated with a particular item.
5. The system of claim 1 , wherein the received image data is crawled from a web page by an image crawler configured to search for item image data associated with a particular item.
6. The system of claim 1 , wherein the candidate item text is tokenized prior to the selection of the candidate text data.
7. The system of claim 1 , wherein the received text data is tokenized prior to applying the selected text correlation model to the received text data and the subset of the candidate item text.
8. The system of claim 1 , wherein at least one of the text correlation model or the image correlation model is selected based on a category of the reference item.
9. The system of claim 1 , wherein:
at least one of the received text data or the received image data is crawled from a web page by a crawler; and
at least one of the selected text correlation model or the selected image correlation model is trained using data sourced from a host of the web page.
10. The system of claim 1 , wherein at least one of the selected text correlation model or the selected image correlation model is a random forest model.
11. The system of claim 1 , wherein the selected text correlation model contains a text frequency parameter having a weight that is inversely related to a frequency of a character combination in a reference dataset.
12. The system of claim 1 , wherein the selected image correlation model is configured to segment the at least one image into segments and generate a unique identifier for one of the segments of the at least one image.
13. The system of claim 12 , wherein the selected image correlation model is configured to compare the generated unique identifier to an identifier generated for a segment of at least one of the candidate item images.
14. The system of claim 1 , wherein the selected image correlation model is configured to ignore an image property identified by a user input.
15. The system of claim 1 , wherein the calculated confidence score is a statistical combination of the determined first and second similarity scores.
16. The system of claim 1 , wherein the calculated confidence score is based on a previous match between the reference item and a candidate item.
17. A computer-implemented method for correlating item data, the method comprising:
receiving text data associated with a reference item;
receiving image data comprising at least one image associated with the reference item;
converting, using a computer-modeled embedding layer, the at least one image to an image embedding;
comparing the image embedding to reference embeddings, the stored reference embeddings being associated with candidate item images and candidate item text of candidate items;
selecting, based on the comparison, candidate text data, wherein the candidate text data is a subset of the candidate item text and includes a price;
selecting, based on the comparison, candidate image data, the candidate image data being a subset of the candidate item images;
selecting a text correlation model;
determining a first similarity score by applying the selected text correlation model to the received text data and the subset of the candidate item text;
selecting an image correlation model;
determining a second similarity score by applying the selected image correlation model to the received image data and the subset of the candidate item images;
calculating a confidence score based on the determined first and second similarity scores; and
performing a responsive action based on the calculated confidence score, wherein the responsive action comprises at least one of:
creating an association between the reference item and one of the candidate items;
monitoring a webpage associated with the reference item;
updating at least one of the reference embeddings;
modifying the selected text correlation model; or
modifying the selected image correlation model.
18. The computer-implemented method of claim 17 , wherein the received text data is crawled from a web page by a text crawler configured to search for item text data associated with a particular item.
19. The computer-implemented method of claim 17 , wherein the received image data is crawled from a web page by an image crawler configured to search for item image data associated with a particular item.
20. The computer-implemented method of claim 17 , wherein the selected image correlation model is configured to segment the at least one image into segments and generate a unique identifier for one of the segments of the at least one image.