Multilingual mixed search method and system
Embodiments of the present application relate to a multilingual mixed search method, a system for multilingual mixed searching, and a computer program product for multilingual mixed searching. A multilingual mixed searching method is provided. The method includes receiving a query word entered by a user and identifying source language of the query word, translating the query word from the source language to query words of a plurality of target languages, the plurality of target languages being different than the source language, using the query word in a corresponding target language as a basis for searching an index of web page information corresponding to the corresponding target language and obtaining web page information in the corresponding target language, from the web page information obtained from the index corresponding to the corresponding target language, and translating the obtained web page information from the corresponding target language to the source language.
1. A method, comprising:
receiving a query word entered by a user and identifying source language of the query word, the source language being a natural language;
translating, using a processor, the query word from the source language to query words of a plurality of target languages, the plurality of target languages being different than the source language;
searching indices of web page information corresponding to the plurality of target languages based on the translated query words in the plurality of corresponding target languages to obtain web page information in the plurality of corresponding target languages, wherein the searching of the indices of web page information corresponding to the plurality of target languages comprises:
acquiring domain information for characteristic domains corresponding to web page information in at least one target language, the characteristic domains including product information title, product information ID, product information keywords, product information attributes, product information categories, general description of the product information, detailed description of the product information, or any combination thereof;
performing a normalization processing on the domain information based on semantic rules for the at least one target language and obtaining the smallest semantic unit in the at least one target language, the smallest semantic unit corresponds to a word or a phrase; wherein:
the normalization processing includes one or more of: encoding processing, upper to lower case conversion, garbage character processing, improper character processing special character processing, division into sentences, division into words, reduction to word root, or elimination of tones; and
establishing an index based on the characteristic domains and the smallest semantic unit in the corresponding characteristic domains; and
searching the indices of the web page information corresponding to the plurality of target languages based on the smallest semantic unit in the plurality of target languages to obtain web page information in a corresponding target language, wherein the obtaining of the web page information in the corresponding target language comprises:
calculating a first relevance weight of web page information in a first target language and a query word in the first target language;
calculating a second relevance weight of web page information in a second target language and a query word in the second target language; and
ranking web page information in the first and second target languages based on the first and second relevance weights to obtain the web page information in the corresponding target language;
from the web page information obtained from the indices corresponding to the plurality of target languages, translating, using the processor, the obtained web page information from the plurality of target languages to the source language; and
sending back at least one piece of the obtained translated web page information to the user.
2. The method as described in claim 1 , wherein the establishing of the index based on the characteristic domains and the smallest semantic unit in the corresponding characteristic domains comprises:
establishing a first reverse index based on the smallest semantic unit; and
ranking the characteristic domains to establish a second sequenced index based on the characteristic domains and the smallest semantic unit in the corresponding characteristic domains.
3. The method as described in claim 1 , the translating of the query word from the source language to the query words of the plurality of target languages comprises:
calculating a first translation weight for translation of the query word from the source language to the query words in the plurality of target languages.
4. The method as described in claim 1 , wherein the translating of the web page information from the plurality of target languages to the source language comprises:
acquiring second translation weights for translations of the web page information from the plurality of target languages into the source language.
5. The method as described in claim 4 , further comprising before the sending back of the at least one piece of the obtained translated web page information to the user:
for each piece of web page information, using a relevance weight of the web page information, a first translation weight, a second translation weight, or any combination thereof for normalizing and calculating of a total weight under a unitary standard; and
using the total weight to rank each piece of web page information.
6. The method as described in claim 1 , wherein the sending back of the at least one piece of the obtained translated web page information to the user comprises:
removing weights from web page information whose identifier characteristic domains are the same in the web page information.
7. A system, comprising:
at least one processor configured to:
receive a query word entered by a user and identifying source language of the query word, the source language being a natural language;
translate the query word from the source language to query words of a plurality of target languages, the plurality of target languages being different than the source language;
search indices of web page information corresponding to the plurality of target languages based on the translated query words in the plurality of corresponding target languages to obtain web page information in the plurality of corresponding target languages, wherein the searching of the indices of web page information corresponding to the plurality of target languages comprises to:
acquire domain information for characteristic domains corresponding to web page information in at least one target language, the characteristic domains including product information title, product information ID, product information keywords, product information attributes, product information categories, general description of the product information, detailed description of the product information, or any combination thereof;
perform a normalization processing on the domain information based on semantic rules for the at least one target language and obtaining the smallest semantic unit in the at least one target language, the smallest semantic unit corresponds to a word or a phrase; wherein:
the normalization processing includes one or more of: encoding processing, upper to lower case conversion, garbage character processing, improper character processing special character processing, division into sentences, division into words, reduction to word root, or elimination of tones; and
establish an index based on the characteristic domains and the smallest semantic unit in the corresponding characteristic domains; and
search the indices of the web page information corresponding to the plurality of target languages based on the smallest semantic unit in the plurality of target languages to obtain web page information in a corresponding target language, wherein the obtaining of the web page information in the corresponding target language comprises to:
calculate a first relevance weight of web page information in a first target language and a query word in the first target language;
calculate a second relevance weight of web page information in a second target language and a query word in the second target language; and
rank web page information in the first and second target languages based on the first and second relevance weights to obtain the web page information in the corresponding target language;
from the web page information obtained from the indices corresponding to the plurality of target languages, translate the obtained web page information from the plurality of target languages to the source language; and
send back at least one piece of the obtained translated web page information to the user; and
a memory coupled to the at least one processor and configured to provide the at least one processor with instructions.
8. A computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
receiving a query word entered by a user and identifying source language of the query word, the source language being a natural language;
translating the query word from the source language to query words of a plurality of target languages, the plurality of target languages being different than the source language;
searching indices of web page information corresponding to the plurality of target languages based on the translated query words in the plurality of corresponding target languages to obtain web page information in the plurality of corresponding target languages, wherein the searching of the indices of web page information corresponding to the plurality of target languages comprises:
acquiring domain information for characteristic domains corresponding to web page information in at least one target language, the characteristic domains including product information title, product information ID, product information keywords, product information attributes, product information categories, general description of the product information, detailed description of the product information, or any combination thereof;
performing a normalization processing on the domain information based on semantic rules for the at least one target language and obtaining the smallest semantic unit in the at least one target language, the smallest semantic unit corresponds to a word or a phrase; wherein:
the normalization processing includes one or more of: encoding processing, upper to lower case conversion, garbage character processing, improper character processing special character processing, division into sentences, division into words, reduction to word root, or elimination of tones; and
establishing an index based on the characteristic domains and the smallest semantic unit in the corresponding characteristic domains; and
searching the indices of the web page information corresponding to the plurality of target languages based on the smallest semantic unit in the plurality of target languages to obtain web page information in a corresponding target language, wherein the obtaining of the web page information in the corresponding target language comprises:
calculating a first relevance weight of web page information in a first target language and a query word in the first target language;
calculating a second relevance weight of web page information in a second target language and a query word in the second target language; and
ranking web page information in the first and second target languages based on the first and second relevance weights to obtain the web page information in the corresponding target language;
from the web page information obtained from the indices corresponding to the plurality of target languages, translating the obtained web page information from the plurality of target languages to the source language; and
sending back at least one piece of the obtained translated web page information to the user.
9. A method, comprising:
receiving a query word entered by a user and identifying source language of the query word;
translating, using a processor, the query word from the source language to query words of a plurality of target languages, the plurality of target languages being different than the source language;
searching indices of web page information corresponding to the plurality of target languages based on the translated query words in the plurality of corresponding target languages to obtain web page information in the plurality of corresponding target languages, wherein the searching of the indices of web page information corresponding to the plurality of target languages comprises:
performing a normalization processing on the query words of the plurality of target languages based on semantic rules for the plurality of target languages to obtain the smallest semantic unit in the plurality of target languages; and
searching the indices of the web page information corresponding to the plurality of target languages based on the smallest semantic unit in the plurality of target languages to obtain web page information in a corresponding target language, wherein the obtaining of the web page information in the corresponding target language comprises:
calculating a first relevance weight of web page information in a first target language and a query word in the first target language, wherein the calculating of the first relevance weight of the web page information and the query word in the first target language comprises:
calculating a ratio f1 of a number of same words in the query word and in title characteristic domain of the web page information to a number of words in the query word;
calculating a ratio f2 of a number of same words in the query words and in the title characteristic domain of the web page information to a number of words in the title characteristic domain;
calculating a ratio f3 of a number of smallest semantic unit corresponding to the query word to a number of smallest semantic unit of the title characteristic domain;
calculating a ratio f4 of a number of query word to a number of keywords of keyword characteristic domain of the web page information;
calculating a match degree f5 between a smallest semantic unit series of the query word and the smallest semantic unit series of the title characteristic domain of the web page information;
calculating a match degree f6 between the smallest semantic unit series of the query words and keyword series of the keyword characteristic domain of the web page information; and
obtaining the first relevance weight through a linear regression fitting based on the f1, f2, f3, f4, f5 and f6;
calculating a second relevance weight of web page information in a second target language and a query word in the second target language; and
ranking web page information in the first and second target languages based on the first and second relevance weights to obtain the web page information in the corresponding target language;
from the web page information obtained from the indices corresponding to the plurality of target languages, translating, using the processor, the obtained web page information from the plurality of target languages to the source language; and
sending back at least one piece of the obtained translated web page information to the user.
10. A system, comprising:
a processor; and
a memory coupled with the processor, wherein the memory is configured to provide the processor with instructions which when executed cause the processor to:
receive a query word entered by a user and identifying source language of the query word;
translate the query word from the source language to query words of a plurality of target languages, the plurality of target languages being different than the source language;
search indices of web page information corresponding to the plurality of target languages based on the translated query words in the plurality of corresponding target languages to obtain web page information in the plurality of corresponding target languages, wherein the searching of the indices of web page information corresponding to the plurality of target languages comprises:
perform a normalization processing on the query words of the plurality of target languages based on semantic rules for the plurality of target languages to obtain the smallest semantic unit in the plurality of target languages; and
search the indices of the web page information corresponding to the plurality of target languages based on the smallest semantic unit in the plurality of target languages to obtain web page information in a corresponding target language, wherein the obtaining of the web page information in the corresponding target language comprises:
calculate a first relevance weight of web page information in a first target language and a query word in the first target language, wherein the calculating of the first relevance weight of the web page information and the query word in the first target language comprises:
calculate a ratio f1 of a number of same words in the query word and in title characteristic domain of the web page information to a number of words in the query word;
calculate a ratio f2 of a number of same words in the query words and in the title characteristic domain of the web page information to a number of words in the title characteristic domain;
calculate a ratio f3 of a number of smallest semantic unit corresponding to the query word to a number of smallest semantic unit of the title characteristic domain;
calculate a ratio f4 of a number of query word to a number of keywords of keyword characteristic domain of the web page information;
calculate a match degree f5 between a smallest semantic unit series of the query word and the smallest semantic unit series of the title characteristic domain of the web page information;
calculate a match degree f6 between the smallest semantic unit series of the query words and keyword series of the keyword characteristic domain of the web page information; and
obtain the first relevance weight through a linear regression fitting based on the f1, f2, f3, f4, f5 and f6;
calculate a second relevance weight of web page information in a second target language and a query word in the second target language; and
rank web page information in the first and second target languages based on the first and second relevance weights to obtain the web page information in the corresponding target language;
from the web page information obtained from the indices corresponding to the plurality of target languages, translate the obtained web page information from the plurality of target languages to the source language; and
send back at least one piece of the obtained translated web page information to the user.
11. A computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
receiving a query word entered by a user and identifying source language of the query word;
translating the query word from the source language to query words of a plurality of target languages, the plurality of target languages being different than the source language;
searching indices of web page information corresponding to the plurality of target languages based on the translated query words in the plurality of corresponding target languages to obtain web page information in the plurality of corresponding target languages, wherein the searching of the indices of web page information corresponding to the plurality of target languages comprises:
performing a normalization processing on the query words of the plurality of target languages based on semantic rules for the plurality of target languages to obtain the smallest semantic unit in the plurality of target languages; and
searching the indices of the web page information corresponding to the plurality of target languages based on the smallest semantic unit in the plurality of target languages to obtain web page information in a corresponding target language, wherein the obtaining of the web page information in the corresponding target language comprises:
calculating a first relevance weight of web page information in a first target language and a query word in the first target language, wherein the calculating of the first relevance weight of the web page information and the query word in the first target language comprises:
calculating a ratio f1 of a number of same words in the query word and in title characteristic domain of the web page information to a number of words in the query word;
calculating a ratio f2 of a number of same words in the query words and in the title characteristic domain of the web page information to a number of words in the title characteristic domain;
calculating a ratio f3 of a number of smallest semantic unit corresponding to the query word to a number of smallest semantic unit of the title characteristic domain;
calculating a ratio f4 of a number of query word to a number of keywords of keyword characteristic domain of the web page information;
calculating a match degree f5 between a smallest semantic unit series of the query word and the smallest semantic unit series of the title characteristic domain of the web page information;
calculating a match degree f6 between the smallest semantic unit series of the query words and keyword series of the keyword characteristic domain of the web page information; and
obtaining the first relevance weight through a linear regression fitting based on the f1, f2, f3, f4, f5 and f6;
calculating a second relevance weight of web page information in a second target language and a query word in the second target language; and
ranking web page information in the first and second target languages based on the first and second relevance weights to obtain the web page information in the corresponding target language;
from the web page information obtained from the indices corresponding to the plurality of target languages, translating the obtained web page information from the plurality of target languages to the source language; and
sending back at least one piece of the obtained translated web page information to the user.