Search system for providing search results using query understanding and semantic binary signatures
Technology for the improved processing of search queries is provided. In one embodiment, methods may return semantically relevant search results for a search query. During a pre-computing offline processing, an inventory semantic index may be generated and may include inventory binary hashing signatures that are associated with inventory listings, such as goods or services for sell, and the index may be partitioned by categories and shards. When a search query is received, relevant categories are determined using a relevant category recognition service, and a search query binary hashing signature maybe generated for the search query. The relevant categories are searched to determine hamming distances between the inventory binary hashing signatures and the search query binary hashing signature, where the hamming distance indicates semantic relevance.
1 . A system, the system comprising:
one or more processors; and
one or more computer storage media storing computer-useable instructions that, when used by the one or more processors, cause the one or more processors to execute operations comprising:
accessing a plurality of inventory listings, wherein each inventory listing is associated with a category from a plurality of categories;
employing a semantic-signature-generator to generate inventory binary hashing signatures based on an inventory identifier associated with each of the plurality of inventory listings, wherein the semantic-signature-generator is associated with a semantic-signature generator model that is trained to support generating binary hashing signatures; and
storing the inventory binary hashing signature for each inventory listing in an inventory semantic index, the inventory semantic index is partitioned by the plurality of categories comprising a first partitioned category having a first index identifier and a second partitioned category having a second index identifier,
wherein searching the first partitioned category and the second category comprises employing a parallel processing engine associated with identifying index identifiers, the parallel processing engine utilizes each index identifier to search the first partitioned category and the second partitioned category in parallel,
wherein the parallel processing engine is further associated with searching at least two relevant categories in parallel within the inventory semantic index that is partitioned by the plurality of categories by comparing search query binary hashing signatures to inventory binary hashing signatures for each inventory listing in the at least two relevant categories associated with a selected search query, wherein each of the plurality of categories is associated with an index identifier,
wherein index identifiers are used to initiate and terminate parallel searches, and
wherein the searching by the parallel processing engine continues until the parallel processing engine identifies a boundary defined by a different index identifier for each of the at least two relevant categories searched.
2 . The system of claim 1 , the operations further receiving a search query;
comprising:
generating a search query binary hashing signature of the search query;
identifying a relevant category for the search query;
accessing the inventory semantic index, wherein the inventory semantic index comprises a plurality of inventory listings associated with the relevant category, each inventory listing associated with an inventory binary hashing signature;
determining that the relevant category corresponds to the first partitioned category;
searching the plurality of inventory listings, within the first partitioned category of the inventory semantic index that corresponds to the relevant category, by computing a hamming distance between the search query binary hashing signature and each inventory binary hashing signature for the plurality of inventory listings in the first partitioned category;
based on computing the hamming distance, identifying an inventory listing based on the hamming distance representing a relative semantic similarity between the search query and the inventory listing; and
communicating the inventory listing as a search result for the search query.
3 . The system of claim 1 , wherein a first relevant category includes at least two index identifiers and a second relevant category includes one index identifier, and the parallel processing engine independently stops searching the first relevant category after processing the at least two index identifiers and stops searching the second relevant category after processing the one index identifier.
4 . The system of claim 1 , wherein the inventory semantic index is partitioned by leaf categories of the plurality of categories associated with the plurality of listings.
5 . The system of claim 2 , wherein generating the search query binary hashing signature is performed by the semantic-signature generator, the semantic-signature generator including the semantic-signature-generator model that generates binary hashing signatures from inputs, the semantic-signature-generator model is trained using stored behavioral data so that the binary hashing signatures generated are semantically related based on the stored behavioral data.
6 . The system of claim 2 , wherein identifying the relevant category includes determining a vector representation of the search query and comparing the vector representation of the search query to a vector representation of the relevant category, wherein the relevant category is identified based on a vector distance determined from the comparison.
7 . The system of claim 2 , wherein the inventory semantic index includes a plurality of shards that partition the relevant category, and wherein searching the relevant category of the inventory semantic index comprises employing the parallel processing engine to search inventory listings associated with each shard of the plurality of shards in parallel.
8 . The system of claim 2 , wherein computing the hamming distance comprises determining a number of differences between a set of binary digits representing the search query binary hashing signature and a set of binary digits representing the inventory binary hashing signature.
9 . One or more computer storage media having computer-executable instructions embodied thereon that, when executed, by one or more processors, cause the one or more processors to perform a method, the method comprising:
accessing a plurality of inventory listings, wherein each inventory listing is associated with a category from a plurality of categories;
employing a semantic-signature-generator to generate inventory binary hashing signatures based on an inventory identifier associated with each of the plurality of inventory listings, wherein the semantic-signature-generator is associated with a semantic-signature generator model that is trained to support generating binary hashing signatures; and
storing the inventory binary hashing signature for each inventory listing in an inventory semantic index, the inventory semantic index is partitioned by the plurality of categories comprising a first partitioned category having a first index identifier and a second partitioned category having a second index identifier,
wherein searching the first partitioned category and the second category comprises employing a parallel processing engine associated with identifying index identifiers, the parallel processing engine utilizes each index identifier to search the first partitioned category and the second partitioned category in parallel,
wherein the parallel processing engine is further associated with searching at least two relevant categories in parallel within the inventory semantic index that is partitioned by the plurality of categories by comparing search query binary hashing signatures to inventory binary hashing signatures for each inventory listing in the at least two relevant categories associated with a selected search query,
wherein each of the plurality of categories is associated with an index identifier,
wherein index identifiers are used to initiate and terminate parallel searches, and
wherein the searching by the parallel processing engine continues until the parallel processing engine identifies a boundary defined by a different index identifier for each of the at least two relevant categories searched.
10 . The media of claim 9 , the method further comprising:
receiving a search query;
generating a search query binary hashing signature of the search query;
identifying a relevant category for the search query;
accessing the inventory semantic index, wherein the inventory semantic index comprises a plurality of inventory listings associated with the relevant category, each inventory listing associated with an inventory binary hashing signature;
determining that the relevant category corresponds to the first partitioned category;
searching the plurality of inventory listings, within the first partitioned category of the inventory semantic index that corresponds to the relevant category, by computing a hamming distance between the search query binary hashing signature and each inventory binary hashing signature for the plurality of inventory listings in the first partitioned category;
based on computing the hamming distance, identifying an inventory listing based on the hamming distance representing a relative semantic similarity between the search query and the inventory listing; and
communicating the inventory listing as a search result for the search query.
11 . The media of claim 10 , wherein generating the search query binary hashing signature is performed by the semantic-signature generator, the semantic-signature generator including the semantic-signature-generator model that generates binary hashing signatures from inputs, the semantic-signature-generator model is trained using stored behavioral data so that the binary hashing signatures generated are semantically related based on the stored behavioral data.
12 . The media of claim 10 , wherein identifying the relevant category includes determining a vector representation of the search query and comparing the vector representation of the search query to a vector representation of the relevant category, wherein the relevant category is identified based on a vector distance determined from the comparison.
13 . The media of claim 10 , wherein a first relevant category includes at least two index identifiers and a second relevant category includes one index identifier, and the parallel processing engine independently stops searching the first relevant category after processing the at least two index identifiers and stops searching the second relevant category after processing the one index identifier.
14 . The media of claim 10 , wherein computing the hamming distance comprises determining a number of differences between a set of binary digits representing the search query binary hashing signature and a set of binary digits representing the inventory binary hashing signature.
15 . A computer-implemented method for processing search queries, the method comprising:
accessing a plurality of inventory listings, wherein each inventory listing is associated with a category from a plurality of categories;
employing a semantic-signature-generator to generate inventory binary hashing signatures based on an inventory identifier associated with each of the plurality of inventory listings, wherein the semantic-signature-generator is associated with a semantic-signature generator model that is trained to support generating binary hashing signatures; and
storing the inventory binary hashing signatures for each inventory listing in an inventory semantic index, the inventory semantic index is partitioned by the plurality of categories comprising a first partitioned category having a first index identifier and a second partitioned category having a second index identifier,
wherein searching the first partitioned category and the second category comprises employing a parallel processing engine associated with identifying index identifiers, the parallel processing engine utilizes each index identifier to search the first partitioned category and the second partitioned category in parallel,
wherein the parallel processing engine is further associated with searching at least two relevant categories in parallel within the inventory semantic index that is partitioned by the plurality of categories by comparing search query binary hashing signatures to inventory binary hashing signatures for each inventory listing in the at least two relevant categories associated with a selected search query,
wherein each of the plurality of categories is associated with an index identifier,
wherein index identifiers are used to initiate and terminate parallel searches, and
wherein the searching by the parallel processing engine continues until the parallel processing engine identifies a boundary defined by a different index identifier for each of the at least two relevant categories searched.
16 . The method of claim 15 , the method further comprising:
receiving a search query;
generating a search query binary hashing signature of the search query;
identifying a relevant category for the search query;
accessing the inventory semantic index, wherein the inventory semantic index comprises a plurality of inventory listings associated with the relevant category, each inventory listing associated with an inventory binary hashing signature;
determining that the relevant category corresponds to the first partitioned category;
searching the plurality of inventory listings, within the first partitioned category of the inventory semantic index that corresponds to the relevant category, by computing a hamming distance between the search query binary hashing signature and each inventory binary hashing signature for the plurality of inventory listings in the first partitioned category;
based on computing the hamming distance, identifying an inventory listing based on the hamming distance representing a relative semantic similarity between the search query and the inventory listing; and
communicating the inventory listing as a search result for the search query.
17 . The method of claim 16 , wherein generating the search query binary hashing signature is performed by the semantic-signature generator, the semantic-signature generator including the semantic-signature-generator model that generates binary hashing signatures from inputs, the semantic-signature-generator model is trained using stored behavioral data so that the binary hashing signatures generated are semantically related based on the stored behavioral data.
18 . The method of claim 16 , wherein identifying the relevant category includes determining a vector representation of the search query and comparing the vector representation of the search query to a vector representation of the relevant category, wherein the relevant category is identified based on a vector distance determined from the comparison.
19 . The method of claim 16 , wherein a first relevant category includes at least two index identifiers and a second relevant category includes one index identifier, and the parallel processing engine independently stops searching the first relevant category after processing the at least two index identifiers and stops searching the second relevant category after processing the one index identifier.
20 . The method of claim 16 , wherein computing the hamming distance comprises determining a number of differences between a set of binary digits representing the search query binary hashing signature and a set of binary digits representing the inventory binary hashing signature.