Sequence recommendation method based on extracting and modeling of complex multi-mode user interests
A sequence recommendation method based on extraction and modeling of complex multi-mode user interests is provided, including: obtaining long-term and short-term embedding sequences; obtaining updated long-term and short-term embedding sequences through the long-term and short-term embedding sequences; with embedding vectors of last items in the updated long-term and short-term embedding sequences as long-term and short-term dynamic interests of a user, obtaining long-term and short-term static interests of the user through weighted calculation; concatenating the long-term and short-term dynamic interests and the long-term and short-term static interests, and performing nonlinear change to obtain long-term and short-term evolutionary interests of the user; obtaining a dynamic interest, a static interest and an evolutionary interest of the user through element-wise summation; performing weighted summation to obtain a fused user interest; calculating a product of the fused interest with embedding of each item as a recommendation score of each item.
1 . A sequence recommendation method based on extracting and modeling of complex multi-mode user interests, comprising:
Step 1, acquiring a historical item interaction sequence of a user, and obtaining a long-term embedding sequence
F
l
=
(
f
l
1
,
f
l
2
,
…
,
f
l
m
)
and a short-term embedding sequence
F
s
=
(
f
s
1
,
f
s
2
,
…
,
f
s
n
)
through a self-learning item embedding matrix, where m>n;
Step 2, inputting the long-term embedding sequence and the short-term embedding sequence into two independent multi-head self-attention modules, respectively, to obtain an updated long-term embedding sequence and an updated short-term embedding sequence;
Step 3, with an embedding vector of a last item in the updated long-term embedding sequence as a long-term dynamic interest of the user, calculating attention weights of the updated long-term embedding sequence to the embedding vector of the last item, and performing weighted summation to obtain a long-term static interest of the user, and similarly, obtaining a short-term dynamic interest and a short-term static interest based on the updated short-term embedding sequence, wherein:
the long-term dynamic interest
p
l
d
of the user is the embedding vector
e
^
l
m
of the last item in the updated long-term embedding sequence
E
^
l
=
(
e
^
l
1
,
e
^
l
2
,
…
,
e
^
l
m
)
,
that is, the long-term dynamic interest is
p
l
d
=
e
^
l
m
,
and the long-term static interest is defined as:
p
l
x
=
∑
i
=
1
m
α
i
e
^
l
i
α
i
=
exp
(
e
^
ι
m
T
Re
LU
(
W
l
e
^
l
i
)
)
∑
j
∈
{
1
,
2
,
…
,
m
}
exp
(
e
ˆ
l
m
T
Re
LU
(
W
l
e
^
l
j
)
)
where W l ∈ (o×d′)×(o×d′) is a parameter to be trained, ReLU is an activation function, and
e
^
l
m
the embedding vector of the last item;
with an embedding vector
e
^
s
n
of a last item in the updated short-term embedding sequence
E
^
s
=
(
e
^
s
1
,
e
^
s
2
,
…
,
e
^
s
n
)
as a short-term dynamic interest
p
s
d
of the user, calculating attention weights of the updated short-term embedding sequence Ê s to the embedding vector
e
^
s
n
of the last item, and performing weighted summation to obtain a short-term static interest
p
s
x
of the user, specifically:
the short-term dynamic interest
p
s
d
and the short-term static interest
p
s
x
are obtained based on the short-term embedding sequence Ê s :
p
s
d
=
e
^
s
n
p
s
x
=
∑
i
=
1
n
β
i
e
^
s
i
β
i
=
exp
(
e
^
s
n
T
Re
LU
(
W
s
e
^
s
i
)
)
∑
j
∈
{
1
,
2
,
…
,
n
}
exp
(
e
ˆ
s
n
T
Re
LU
(
W
s
e
ˆ
s
j
)
)
where W s ∈ (o×d′)×(o×d′) is a parameter to be trained;
Step 4, concatenating the long-term dynamic interest
p
l
d
and the long-term static interest
p
l
x
,
and performing nonlinear change to obtain a long-term evolutionary interest
p
l
y
of the user;
and similarly, obtaining a short-term evolutionary interest
p
s
y
of the user based on the short-term dynamic interest
p
s
d
and the short-term static interest
p
s
x
;
Step 5, obtaining a dynamic interest, a static interest and an evolutionary interest of the user through element-wise summation, wherein:
the dynamic interest of the user is obtained through element-wise summation of the long-term dynamic interest and the short-term dynamic interest;
the static interest of the user is obtained through element-wise summation of the long-term static interest and the short-term static interest;
the evolutionary interest of the user is obtained through element-wise summation of the long-term evolutionary interest and the short-term evolutionary interest;
Step 6, calculating attention weights of the dynamic interest, the static interest and the evolutionary interest to the embedding vector of the last item, and performing weighted summation to obtain a fused user interest;
Step 7, calculating a product of the fused user interest with embedding of each item as a recommendation score of each item, and recommending top items with highest scores for the user.
2 . The method according to claim 1 , wherein in Step 1, the historical item interaction sequence of the user is expressed as H=(h 1 , h 2 , . . . , h t ), where h i is an item corresponding to an i-th interaction behavior; a long-term sequence is a latest
H
l
=
(
h
l
1
,
h
l
2
,
⋯
,
h
l
m
)
with a length of m, and a short-term sequence is a latest sequence
H
s
=
(
h
s
1
,
h
s
2
,
⋯
,
h
s
n
)
with a length of n; based on a self-learning item embedding matrix F∈ k×d , items involved in the sequences are embedded to obtain a long-term embedding sequence
F
l
=
(
f
l
1
,
f
l
2
,
⋯
,
f
l
m
)
and a short-term embedding sequence
F
s
=
(
f
s
1
,
f
s
2
,
⋯
,
f
s
n
)
,
where k indicates a number of kinds of all items in all sequences, where
F
l
∈
ℝ
m
×
d
,
F
s
∈
ℝ
n
×
d
,
f
l
i
∈
ℝ
d
,
f
s
i
∈
ℝ
d
,
and d indicates a vector embedding dimension.
3 . The method according to claim 2 , wherein in Step 2, the updated long-term embedding sequence Ê l is defined as:
E
^
l
=
(
g
^
l
1
,
g
^
l
2
,
⋯
,
g
^
l
m
)
=
(
e
^
l
1
,
e
^
l
2
,
⋯
,
e
^
l
m
)
where
e
^
l
i
is an embedding representation of a corresponding item i after being updated by an attention mechanism, and
g
^
l
j
∈
ℝ
m
×
d
′
is an embedding sequence obtained by a j-th attention head, which is defined as:
g
ˆ
l
j
=
softmax
(
A
j
B
j
T
d
/
o
)
C
i
where
A
j
=
G
l
W
A
j
,
B
j
=
G
l
W
B
j
,
C
j
=
G
l
W
C
j
,
G
l
=
(
g
l
1
,
g
l
2
,
⋯
,
g
l
m
)
is a long-term embedding sequence after being updated in position,
g
l
i
=
f
l
i
+
pos
l
i
is an embedding representation of the item i after being updated in position,
f
l
i
is an initial embedding representation of the item i, and
pos
l
i
∈
ℝ
d
is a self-learning position vector of the item i,
where
W
A
j
∈
ℝ
d
×
d
′
,
W
B
j
∈
ℝ
d
×
d
′
and
W
C
j
∈
ℝ
d
×
d
′
are three parameter matrices.
4 . The method according to claim 3 , wherein in Step 4, the long-term evolutionary interest of the user is defined as:
p
l
y
=
R
e
L
U
(
W
l
y
(
p
l
d
p
l
x
)
)
where
p
l
d
is the long-term dynamic interest,
p
l
x
is the long-term static interest, (⋅|⋅) is a concatenation operation, and
W
l
y
∈
ℝ
(
o
×
d
′
)
×
(
2
×
o
×
d
′
)
is a parameter to be trained;
the short-term evolutionary interest of the user is defined as:
p
s
y
=
R
e
L
U
(
W
s
x
(
p
s
d
p
s
x
)
)
.
5 . The method according to claim 4 , wherein in Step 5, the dynamic interest p d of the user is defined as:
p
d
=
p
l
d
⊕
p
s
d
,
where ⊕ indicates element-wise addition,
p
l
d
is the long-term dynamic interest, and
p
s
d
is the short-term dynamic interest;
the static interest p x of the user is defined as:
p
x
=
p
l
x
⊕
p
s
x
,
where ⊕ indicates element-wise addition,
p
l
x
is the long-term static interest, and
p
s
x
is the short-term static interest;
the evolutionary interest p y of the user is defined as:
p
y
=
p
l
y
⊕
p
s
y
,
where ⊕ indicates element-wise addition,
p
l
y
is the long-term evolutionary interest, and
p
s
y
is the short-term evolutionary interest.
6 . The method according to claim 5 , wherein in Step 6, the fused user interest is defined as:
p
=
W
r
(
α
p
p
d
+
α
x
p
x
+
α
y
p
y
)
α
p
=
exp
(
e
^
l
m
Re
LU
(
W
f
p
d
)
)
exp
(
e
^
l
m
Re
LU
(
W
f
p
d
)
)
+
exp
(
e
^
l
m
Re
LU
(
W
f
p
x
)
)
+
exp
(
e
^
l
m
Re
LU
(
W
f
p
y
)
)
α
x
=
exp
(
e
^
l
m
Re
LU
(
W
f
p
x
)
)
exp
(
e
^
l
m
Re
LU
(
W
f
p
d
)
)
+
exp
(
e
^
l
m
Re
LU
(
W
f
p
x
)
)
+
exp
(
e
^
l
m
Re
LU
(
W
f
p
y
)
)
α
y
=
exp
(
e
^
l
m
Re
LU
(
W
f
p
y
)
)
exp
(
e
^
l
m
Re
LU
(
W
f
p
d
)
)
+
exp
(
e
^
l
m
Re
LU
(
W
f
p
x
)
)
+
exp
(
e
^
l
m
Re
LU
(
W
f
p
y
)
)
where W r ∈ d×(o×d′) is a parameter to be trained, p d is the dynamic interest, p x is the static interest, p y is the evolutionary interest, and
e
^
l
m
is the embedding vector of the last item.