Systems and computer-implemented methods to compare and identify duplicative presentations of comparable data files including imagery and data representations of rental property listings
View Patent ↗A datastore that includes property listings (e.g., for rental properties) may be accessed by a computing resource configured to execute a thumb printing algorithm to generate summary data values (e.g., thumbprint values, hash values) of digital data included one or more images of a property listing. The summary data values along with listing attributes (e.g., address, zip code, geolocation, brand, number of bedrooms/bathrooms/sleeps, amenities) of the property listing may be compared with summary data values and listing attributes of other property listings to determine if the summary data values of one or more images match and if one or more listing attributes match. A threshold number of image matches and listing attribute matches may indicate that the property listings are duplicate property listings, which may be merged, removed or replaced with links/pointers by the computer resource to increase data storage space and/or reduce data search latency times of the datastore.
1. A computer-implemented method for identifying duplicate property listings in a data store, comprising:
retrieving a first set of summary data values derived from a first subset of images and a second set of summary data values derived from a second subset of images, a summary data value being configured to uniquely distinguish an image associated with a rental property from other images associated with other rental properties;
comparing, using a computer resource, each summary data value in the first set of summary data values with each summary data value in the second set of summary data values;
incrementing an image count responsive to a match between the summary data values in the first and the second sets of summary data values;
retrieving first listing attributes associated with a first property listing data file associated with the first subset of images and second listing attributes associated with a second property listing data file associated with the second subset of images;
incrementing an attribute count responsive to a match between a first listing attribute and a second listing attribute;
correlating the image count and the attribute count to an image threshold count and an attribute threshold count to derive a correlation value indicating that one or more portions of the first property listing data file is similar to one or more portions of the second property listing data file; and
initiating generation of a control signal configured to cause the computer resource to perform a corrective action.
2. The method of claim 1 , wherein the corrective action further comprises
accessing, using the computer resource, a data warehouse where the first property listing data file and the second property listing data file are stored;
increasing, using the computer resource, data storage space in the data warehouse by consolidating data in the one or more portions of the second property listing data file into the one or more portions of the first property listing data file; and
deleting the second property listing data file from the data warehouse.
3. The method of claim 1 and further comprising:
computing, using the computer resource, when the image count meets or exceeds the image threshold count and the attribute count meets or exceeds the attribute threshold count, a weighted score that comprises a sum of the image count multiplied by a first weighting factor and the attribute count multiplied by a second weighting factor.
4. The method of claim 3 and further comprising:
determining, using the computer resource, based on the weighted score, that the first property listing data file and the second property listing data file are duplicate property listings;
accessing, using the computer resource, a data warehouse where the first property listing data file and the second property listing data file are stored; and
reducing, using the computer resource, data storage space in the data warehouse that is allocated to the first property listing data file, the second property listing data file or both.
5. The method of claim 1 , wherein the first listing attributes and the second listing attributes include data representing a zip code and the data representing the zip code is identical.
6. The method of claim 1 , wherein the computer resource comprises a plurality of computer server resources.
7. The method of claim 6 , wherein the plurality of computer server resources comprise virtual machines.
8. The method of claim 1 and further comprising:
accessing, using the computer resource, a data warehouse where the first property listing data file and the second property listing data file are stored; and
reducing, using the computer resource, a data search latency time for accessing data stored in the data warehouse by the computer resource by consolidating data in the second property listing data file into the first property listing data file and deleting the second property listing data file from the data warehouse.
9. The method of claim 1 and further comprising:
accessing, using the computer resource, a data warehouse where the first property listing data file and the second property listing data file are stored; and
reducing, using the computer resource, data storage space in the data warehouse that is allocated to the second property listing data file by replacing data in the second property listing data file with data representing a pointer to identical data in the first property listing data file.
10. A system for identifying duplicate property listings in a data store, comprising:
a computer resource configured to
retrieve a first set of summary data values derived from a first subset of images and a second set of summary data values derived from a second subset of images, a summary data value being configured to uniquely distinguish an image associated with a rental property from other images associated with other rental properties,
compare each summary data value in the first set of summary data values with each summary data value in the second set of summary data values,
increment an image count responsive to a match between the summary data values in the first and the second sets of summary data values,
retrieve first listing attributes associated with a first property listing data file associated with the first subset of images and second listing attributes associated with a second property listing data file associated with the second subset of images,
increment an attribute count responsive to a match between a first listing attribute and a second listing attribute,
correlate the image count and the attribute count to an image threshold count and an attribute threshold count to derive a correlation value indicating that one or more portions of the first property listing data file is similar to one or more portions of the second property listing data file, and
receive a control signal configured to cause the computer resource to perform a corrective action.
11. The system of claim 10 , wherein the computer resource is further configured to
access, after receiving the control signal, a data warehouse where the first property listing data file and the second property listing data file are stored,
increase data storage space in the data warehouse by consolidating data in the one or more portions of the second property listing data file into the one or more portions of the first property listing data file, and
delete the second property listing data file from the data warehouse.
12. The system of claim 10 , wherein the computer resource is further configured to
compute, when the image count meets or exceeds the image threshold count and the attribute count meets or exceeds the attribute threshold count, a weighted score that comprises a sum of the image count multiplied by a first weighting factor and the attribute count multiplied by a second weighting factor.
13. The system of claim 12 , wherein the computer resource is further configured to
determine, based on the weighted score, that the first property listing data file and the second property listing data file are duplicate property listings,
access a data warehouse where the first property listing data file and the second property listing data file are stored, and
reduce data storage space in the data warehouse that is allocated to the first property listing data file, the second property listing data file or both.
14. The system of claim 10 , wherein the first listing attributes and the second listing attributes include data representing a zip code and the data representing the zip code is identical.
15. The system of claim 10 , wherein the computer resource comprises a plurality of computer server resources.
16. The system of claim 15 , wherein the plurality of computer server resources comprise virtual machines.
17. The system of claim 10 , wherein the computer resource is further configured to
access a data warehouse where the first property listing data file and the second property listing data file are stored, and
reduce a data search latency time for accessing data stored in the data warehouse by consolidating data in the second property listing data file into the first property listing data file and deleting the second property listing data file from the data warehouse.
18. The system of claim 10 , wherein the computer resource is further configured to
access a data warehouse where the first property listing data file and the second property listing data file are stored, and
reduce data storage space in the data warehouse that is allocated to the second property listing data file by replacing data in the second property listing data file with data representing a pointer to identical data in the first property listing data file.
19. The system of claim 10 , wherein the first set of summary data values and the second set of summary data values comprise hash values.
20. A non-transitory computer-readable medium for identifying duplicate property listings in a data store, having instructions embodied thereon that when executed by a computer perform steps, comprising:
retrieving a first set of summary data values derived from a first subset of images and a second set of summary data values derived from a second subset of images, a summary data value being configured to uniquely distinguish an image associated with a rental property from other images associated with other rental properties;
comparing each summary data value in the first set of summary data values with each summary data value in the second set of summary data values;
incrementing an image count responsive to a match between the summary data values in the first and the second sets of summary data values;
retrieving first listing attributes associated with a first property listing data file associated with the first subset of images and second listing attributes associated with a second property listing data file associated with the second subset of images;
incrementing an attribute count responsive to a match between a first listing attribute and a second listing attribute;
correlating the image count and the attribute count to an image threshold count and an attribute threshold count to derive a correlation value indicating that one or more portions of the first property listing data file is similar to one or more portions of the second property listing data file; and
initiating generation of a control signal configured to cause a corrective action.