System and method for improved server performance for a deep feature based coarse-to-fine fast search
Disclosed are systems and methods for improving interactions with and between computers in a search system supported by or configured with search servers or platforms. The systems interact to identify and retrieve data across platforms, which data can be used to improve the quality of results data used in processing interactions between or among processors in such systems. The disclosed systems and methods provide a Deep Fast Search (DFS) that improves content search accuracy that executes independent of the search database, while achieving an increased content retrieval speed. The disclosed systems and methods employ two complementary deep feature searches: 1) a coarse deep feature search and a 2) fine deep feature search. Thus, the disclosed systems and methods employ a coarse-to-fine strategy that embodies the efficiency and cost effectiveness of the coarse deep feature search and the accuracy of the fine deep feature search.
1. A method comprising:
receiving, at a computing device over a network from a user, a search query comprising image data associated with a captured image;
identifying, via the computing device, features of the captured image from the image data, said features comprising information associated with a content category of the captured image, said features further indicating a pixel value of the captured image;
propagating, via the computing device by executing a forward-propagation algorithm, the features through a convolutional neural network (CNN), and using a hidden layer of the CNN to determine deep descriptor values of the features;
translating, via the computing device, based on said propagation, the deep descriptor values into a feature vector, the feature vector comprising a dimensional value proportional to the pixel value of the captured image;
transforming, via the computing device, the feature vector into a deep hash code (DHC) based on an applied Eigen-hash algorithm and based on hyper-plane values that indicate the content category and correspond to a number of layers in the CNN;
determining, via the computing device, based on the transformation of the feature vector, a DHC for the captured image;
generating, via the computing device, a search query comprising information related to the DHC of the captured image;
searching, via the computing device, an e-commerce (EC) database based on the generated search query, the EC database comprising DHCs of images, said search comprising analyzing positions of the captured image's DHC and the EC database's DHCs within a matrix defined by said hyperplane values, and based on said analysis, identifying a content category of the captured image; and
communicating, via the computing device over the network, a search result to said user comprising a set of EC images within said identified content category.
2. The method of claim 1 , further comprising:
determining a label for each EC image based on the extracted deep descriptors from the EC images, wherein each label comprises information indicating a particular content category.
3. The method of claim 2 , wherein each determined label for each EC image is stored in the EC database in association with a corresponding EC image.
4. The method of claim 1 , further comprising:
receiving, over the network from a device of the user, said DHC for the captured image, wherein said DHC for the captured image is determined by the user device and communicated to the computing device as at least part of the search query.
5. The method of claim 1 , further comprising:
determining a context of the captured image based on the extracted features;
communicating said context to an ad platform, over the network, to obtain an advertisement associated with said context; and
causing communication, over the network, of said identified advertisement to said user for display in association with the search result.
6. A non-transitory computer-readable storage medium tangibly encoded with computer-executable instructions, that when executed by a processor associated with a computing device, performs a method, comprising:
receiving, over a network from a user, a search query comprising image data associated with a captured image;
identifying features of the captured image from the image data, said features comprising information associated with a content category of the captured image, said features further indicating a pixel value of the captured image;
propagating, by executing a forward-propagation algorithm, the features through a convolutional neural network (CNN), and using a hidden layer of the CNN to determine deep descriptor values of the features;
translating, based on said propagation, the deep descriptor values into a feature vector, the feature vector comprising a dimensional value proportional to the pixel value of the captured image;
transforming the feature vector into a deep hash code (DHC) based on an applied Eigen-hash algorithm and based on hyper-plane values that indicate the content category and correspond to a number of layers in the CNN;
determining, based on the transformation of the feature vector, a DHC for the captured image;
generating a search query comprising information related to the DHC of the captured image;
searching an e-commerce (EC) database based on the generated search query, the EC database comprising DHCs of images, said search comprising analyzing positions of the captured image's DHC and the EC database's DHCs within a matrix defined by said hyperplane values, and based on said analysis, identifying a content category of the captured image; and
communicating, over the network, a search result to said user comprising a set of EC images within said identified content category.
7. The non-transitory computer-readable storage medium of claim 6 , further comprising:
determining a label for each EC image based on the extracted deep descriptors from the EC images, wherein each label comprises information indicating a particular content category, wherein each determined label for each EC image is stored in the EC database in association with a corresponding EC image.
8. The non-transitory computer-readable storage medium of claim 6 , further comprising:
receiving, over the network from a device of the user, said DHC for the captured image, wherein said DHC for the captured image is determined by the user device and communicated to the computing device as at least part of the search query.
9. A system, comprising:
a processor; and
a non-transitory computer-readable storage medium for tangibly storing thereon program logic for execution by the processor, the program logic comprising:
logic executed by the processor for receiving, over a network from a user, a search query comprising image data associated with a captured image;
logic executed by the processor for identifying features of the captured image from the image data, said features comprising information associated with a content category of the captured image, said features further indicating a pixel value of the captured image;
logic executed by the processor for propagating, by executing a forward-propagation algorithm, the features through a convolutional neural network (CNN), and using a hidden layer of the CNN to determine deep descriptor values of the features;
logic executed by the processor for translating based on said propagation, the deep descriptor values into a feature vector, the feature vector comprising a dimensional value proportional to the pixel value of the captured image;
logic executed by the processor for transforming the feature vector into a deep hash code (DHC) based on an applied Eigen-hash algorithm and based on hyper-plane values that indicate the content category and correspond to a number of layers in the CNN;
logic executed by the processor for determining, based on the transformation of the feature vector, a DHC for the captured image;
logic executed by the processor for generating a search query comprising information related to the DHC of the captured image;
logic executed by the processor for searching an e-commerce (EC) database based on the generated search query, the EC database comprising DHCs of images, said search comprising analyzing positions of the captured image's DHC and the EC database's DHCs within a matrix defined by said hyperplane values, and based on said analysis, identifying a content category of the captured image; and
logic executed by the processor for communicating, over the network, a search result to said user comprising a set of EC images within said identified content category.
10. The system of claim 9 , further comprising:
logic executed by the processor for determining a label for each EC image based on the extracted deep descriptors from the EC images, wherein each label comprises information indicating a particular content category, wherein each determined label for each EC image is stored in the EC database in association with a corresponding EC image.