IP Library Granted Patent US 11,176,330
Granted Patent B2
US 11,176,330 · App. 16/814,842 · Granted Nov 16, 2021

Generating recommendation information

Inventors: Xiexiong Lin (Hangzhou, CN); Taifeng Wang (Hangzhou, CN); Jing Huang (Hangzhou, CN); Mengshu Sun (Hangzhou, CN)
Assignee: Advanced New Technologies Co., Ltd.
G06F40/30G06F40/279G06N3/0454G06N3/088G06Q30/0276
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,176,330
App. No.
16/814,842
Granted
Nov 16, 2021
Kind
B2
Abstract

Implementations of this disclosure provide methods and apparatuses for generating recommendation information. An example method includes matching text content from a text content library based on a plurality of predetermined scenario-related words; extracting keywords from the related text content, to generate a plurality of training samples; and for each training sample, providing a source sequence of a sequence pair corresponding to the training sample as an input to a recommendation information generation model, obtaining, from the recommendation information generation model, a predicted word, and adjusting a model parameter of the recommendation information generation model based on a comparison between the predicted word and a corresponding word in a target sequence of the sequence pair corresponding to the training sample, to train the recommendation information generation model.

Claims (101)

1. A computer-implemented method comprising:

selecting a plurality of scenario-related words for a scenario;

matching the plurality of scenario-related words with text content from a text content library identify related text content that are related to the scenario;

generating a plurality of training samples based on the related text content, wherein generating the plurality of training samples comprises, for each training sample of the plurality of training samples

obtaining source keywords, by extracting keywords from the related text content,

after obtaining the source keywords, obtaining a target sequence, the target sequence being a sequence of words obtained from the related text content, and

constructing the training sample, wherein the training sample comprises a sequence pair comprising the source keywords and the target sequence; and

for each training sample of the plurality of training samples,

providing the source keywords of the sequence pair corresponding to the training sample as an input to a recommendation information generation model, wherein the recommendation information generation model comprises an encoder-decoder network,

obtaining, from the recommendation information generation model, a predicted word of a word sequence, wherein the recommendation information generation model adjusts a probability distribution of words in a word list using pointer distribution to increase a probability that keywords in the source keywords that are the input to the recommendation information generation model will appear in a prediction result, and

adjusting a model parameter of the recommendation information generation model based on a comparison between the predicted word of the recommendation information generation model and a corresponding word in the target sequence of the sequence pair corresponding to the training sample to train the recommendation information generation model.

2. The computer-implemented method of claim 1 , wherein matching the plurality of scenario-related words with text content from the text content library comprises:

determining a quantity of scenario-related words in the text content from the text content library; and

in response to the quantity of scenario-related words in the text content being greater than a predetermined quantity of scenario-related words, identifying the text content as the related text content.

3. The computer-implemented method of claim 1 , wherein matching the plurality of scenario-related words with text content from the text content library comprises:

extracting keywords from the text content in the text content library;

determining a word matching degree between the keywords extracted from the text content and the plurality of scenario-related words; and

in response to the word matching degree being greater than a matching degree threshold, identifying the text content as the related text content.

4. The computer-implemented method of claim 1 , further comprising:

after identifying the related text content, inputting the related text content to a pre-trained classification model, to determine at least one text content category of the related text content; and

screening out text content for which the at least one text content category does not include a predetermined category.

5. The computer-implemented method of claim 1 , wherein the encoder-decoder network comprises an encoder neural network and a decoder neural network, and adjusting the model parameter of the recommendation information generation model comprises:

transmitting, to the decoder neural network, (i) the source keywords that have been provided as input to the recommendation information generation model and (ii) a semantic vector obtained by the encoder neural network based on the source keywords, and

predicting the word sequence, by the decoder neural network, the predicting comprising

(i) predicting an initial word of the word sequence based on the semantic vector, and

(ii) after predicting the initial word of the word sequence, sequentially predicting subsequent words of the word sequence, based on the semantic vector and a sequence of previously predicted words of the word sequence.

6. The computer-implemented method of claim 1 , wherein a pointer distribution weight is positively correlated with

weighted values of a state of a neuron at a current moment,

an output of the neuron at the current moment, and

an input word at the current moment, and weights applied to each word in a sequence are model parameters; and

using pointer distribution to adjust the probability distribution of words in the word list comprises:

using the pointer distribution weight as a weight coefficient of a word distribution probability of one of the words in the word list, and using a difference between 1 and the pointer distribution weight as the weight coefficient of the word distribution probability of the input word received by an encoder neural network of the recommendation information generation model at the current moment.

7. The computer-implemented method of claim 6 , wherein the weight coefficient in the pointer distribution weight is adjusted based on a loss function to which a penalty of similarity between a current attention vector and a sum of historical attention vectors is added, the loss function being positively correlated with a similarity between the current attention vector and the sum of historical attention vectors.

8. A non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform operations comprising:

selecting a plurality of scenario-related words for a scenario;

matching the plurality of scenario-related words with text content from a text content library to identify related text content that are related to the scenario;

generating a plurality of training samples based on the related text content, wherein generating the plurality of training samples comprises, for each training sample of the plurality of training samples

obtaining source keywords, by extracting keywords from the related text content,

after obtaining the source keywords, obtaining a target sequence, the target sequence being a sequence of words obtained from the related text content, and

constructing the training sample, wherein the training sample comprises a sequence pair comprising the source keywords and the target sequence; and

for each training sample of the plurality of training samples,

providing the source keywords of the sequence pair corresponding to the training sample as an input to a recommendation information generation model, wherein the recommendation information generation model comprises an encoder-decoder network,

obtaining, from the recommendation information generation model, a predicted word of a word sequence, wherein the recommendation information generation model adjusts a probability distribution of words in a word list using pointer distribution to increase a probability that keywords in the source keywords that are the input to the recommendation information generation model will appear in a prediction result, and

adjusting a model parameter of the recommendation information generation model based on a comparison between the predicted word of the recommendation information generation model and a corresponding word in the target sequence of the sequence pair corresponding to the training sample to train the recommendation information generation model.

9. The computer-readable medium of claim 8 , wherein matching the plurality of scenario-related words with text content from the text content library comprises:

determining a quantity of scenario-related words in the text content from the text content library; and

in response to the quantity of scenario-related words in the text content being greater than a predetermined quantity of scenario-related words, identifying the text content as the related text content.

10. The computer-readable medium of claim 8 , wherein matching the plurality of scenario-related words with text content from the text content library comprises:

extracting keywords from the text content in the text content library;

determining a word matching degree between the keywords extracted from the text content and the plurality of scenario-related words; and

in response to the word matching degree being greater than a matching degree threshold, identifying the text content as the related text content.

11. The computer-readable medium of claim 8 , the operations further comprising:

after identifying the related text content, inputting the related text content to a pre-trained classification model, to determine at least one text content category of the related text content; and

screening out text content for which the at least one text content category does not include a predetermined category.

12. The computer-readable medium of claim 8 , wherein the encoder-decoder network comprises an encoder neural network and a decoder neural network, and adjusting the model parameter of the recommendation information generation model comprises:

transmitting, to the decoder neural network, (i) the source keywords that have been provided as input to the recommendation information generation model and (ii) a semantic vector obtained by the encoder neural network based on the source keywords, and

predicting the word sequence, by the decoder neural network, the predicting comprising

(i) predicting an initial word of the word sequence based on the semantic vector, and

(ii) after predicting the initial word of the word sequence, sequentially predicting subsequent words of the word sequence, based on the semantic vector and a sequence of previously predicted words of the word sequence.

13. The computer-readable medium of claim 8 , wherein a pointer distribution weight is positively correlated with

weighted values of a state of a neuron at a current moment,

an output of the neuron at the current moment, and

an input word at the current moment, and weights applied to each word in a sequence are model parameters; and

using pointer distribution to adjust the probability distribution of words in the word list comprises:

using the pointer distribution weight as a weight coefficient of a word distribution probability of one of the words in the word list, and using a difference between 1 and the pointer distribution weight as the weight coefficient of the word distribution probability of the input word received by an encoder neural network of the recommendation information generation model at the current moment.

14. The computer-readable medium of claim 13 , wherein the weight coefficient in the pointer distribution weight is adjusted based on a loss function to which a penalty of similarity between a current attention vector and a sum of historical attention vectors is added, the loss function being positively correlated with a similarity between the current attention vector and the sum of historical attention vectors.

15. A computer-implemented system, comprising:

one or more computers; and

one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing one or more instructions that, when executed by the one or more computers, perform one or more operations comprising:

selecting a plurality of scenario-related words for a scenario;

matching the plurality of scenario-related words with text content from a text content library identify related text content that are related to the scenario;

generating a plurality of training samples based on the related text content, wherein generating the plurality of training samples comprises, for each training sample of the plurality of training samples

obtaining source keywords, by extracting keywords from the related text content,

after obtaining the source keywords, obtaining a target sequence, the target sequence being a sequence of words obtained from the related text content, and

constructing the training sample, wherein the training sample comprises a sequence pair comprising the source keywords and the target sequence; and

for each training sample of the plurality of training samples,

providing the source keywords of the sequence pair corresponding to the training sample as an input to a recommendation information generation model, wherein the recommendation information generation model comprises an encoder-decoder network,

obtaining, from the recommendation information generation model, a predicted word of a word sequence, wherein the recommendation information generation model adjusts a probability distribution of words in a word list using pointer distribution to increase a probability that keywords in the source keywords that are the input to the recommendation information generation model will appear in a prediction result, and

adjusting a model parameter of the recommendation information generation model based on a comparison between the predicted word of the recommendation information generation model and a corresponding word in the target sequence of the sequence pair corresponding to the training sample to train the recommendation information generation model.

16. The computer-implemented system of claim 15 , wherein matching the plurality of scenario-related words with text content from the text content library comprises:

determining a quantity of scenario-related words in the text content from the text content library; and

in response to the quantity of scenario-related words in the text content being greater than a predetermined quantity of scenario-related words, identifying the text content as the related text content.

17. The computer-implemented system of claim 15 , wherein matching the plurality of scenario-related words with text content from the text content library comprises:

extracting keywords from the text content in the text content library;

determining a word matching degree between the keywords extracted from the text content and the plurality of scenario-related words; and

in response to the word matching degree being greater than a matching degree threshold, identifying the text content as the related text content.

18. The computer-implemented system of claim 15 , the operations further comprising:

after identifying the related text content, inputting the related text content to a pre-trained classification model, to determine at least one text content category of the related text content; and

screening out text content for which the at least one text content category does not include a predetermined category.

19. The computer-implemented system of claim 15 , wherein the encoder-decoder network comprises an encoder neural network and a decoder neural network, and adjusting the model parameter of the recommendation information generation model comprises:

transmitting, to the decoder neural network, (i) the source keywords that have been provided as input to the recommendation information generation model and (ii) a semantic vector obtained by the encoder neural network based on the source keywords, and

predicting the word sequence, by the decoder neural network, the predicting comprising

(i) predicting an initial word of the word sequence based on the semantic vector, and

(ii) after predicting the initial word of the word sequence, sequentially predicting subsequent words of the word sequence, based on the semantic vector and a sequence of previously predicted words of the word sequence.

20. The computer-implemented system of claim 15 , wherein a pointer distribution weight is positively correlated with

weighted values of a state of a neuron at a current moment,

an output of the neuron at the current moment, and

an input word at the current moment, and weights applied to each word in a sequence are model parameters; and

using pointer distribution to adjust the probability distribution of words in the word list comprises:

using the pointer distribution weight as a weight coefficient of a word distribution probability of one of the words in the word list, and using a difference between 1 and the pointer distribution weight as the weight coefficient of the word distribution probability of the input word received by an encoder neural network of the recommendation information generation model at the current moment.

21. The computer-implemented system of claim 20 , wherein the weight coefficient in the pointer distribution weight is adjusted based on a loss function to which a penalty of similarity between a current attention vector and a sum of historical attention vectors is added, the loss function being positively correlated with a similarity between the current attention vector and the sum of historical attention vectors.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 10, 2020
From: ADVANTAGEOUS NEW TECHNOLOGIES CO., LTD.
To: ADVANCED NEW TECHNOLOGIES CO., LTD.
Reel/Frame 053754/0625 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 31, 2020
From: ALIBABA GROUP HOLDING LIMITED
To: ADVANTAGEOUS NEW TECHNOLOGIES CO., LTD.
Reel/Frame 053743/0464 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 30, 2020
From: LIN, XIEXIONG; WANG, TAIFENG; HUANG, JING; SUN, MENGSHU
To: ALIBABA GROUP HOLDING LIMITED
Reel/Frame 052263/0816 →
Priority Claims (1)
CN 201910662196.3 · Jul 22, 2019 · national
Continuity (2)
Continuation PCTCN2020071682 · Jan 13, 2020
Related Publication 20210027018A1 · Jan 28, 2021