Methods and systems for detecting topic transitions in a multimedia content
A method for detecting one or more topic transitions in a multimedia content includes identifying, one or more frames from a plurality of frames of the multimedia content based on a comparison between one or more content items in a first frame of the plurality of frames, and the one or more content items in a first set of frames of the plurality of frames. The method further includes determining at least a first score, and a second score for each of the one or more frames. Additionally, the method includes determining a likelihood for each of the one or more frames based at least on the first score, and the second score, wherein the likelihood is indicative of a topic transition among the one or more frames.
1. A method for detecting one or more topic transition points in multimedia content, the method comprising:
identifying, by one or more processors, one or more frames from a plurality of frames in the multimedia content, wherein the one or more frames are identified based on a comparison between one or more content items in a first frame of the plurality of frames and the one or more content items in a first set of frames of the plurality of frames, an occurrence of the first set of frames is temporally prior to the first frame;
determining, by the one or more processors, at least a first score and a second score for each of the one or more frames based on a set of dissimilar content items and a set of similar content items among the one or more frames; and
determining, by the one or more processors, a likelihood for each of the one or more frames, wherein the likelihood is indicative of a topic transition for each of the plurality of the one or more frames based at least on the first score and the second score, wherein
determining of the likelihood for each of the one or more frames comprises identifying the one or more topic transition points in the multimedia content using the determined first score and the determined second score for each of the plurality of frames, wherein
the identifying of the one or more topic transition points further comprises
determining the likelihood of the topic transition among the plurality of frames by using
P
(
S
t
=
1
❘
f
S
t
)
=
P
(
f
S
t
❘
S
t
=
1
)
×
P
(
S
t
=
1
)
P
(
f
S
t
)
where P(S t =1|f S t ) represents a conditional likelihood of frame S t to be indicative of the topic transition, when two-dimensional vector f S t ={f 1 S t ,f 2 S t },
P(f S t |S t =1) represents a conditional likelihood of occurrence of f S t ={f 1 S t ,f 2 S t }, when the frame S t is indicative of the topic transition,
P(S t =1) represents a likelihood that the frame S t is indicative of the topic transition, and
P(f S t ) represents a likelihood of occurrence of f S t ={f 1 S t ,f 2 S t } in the frame S t .
2. A process for identifying one or more topic transition points in multimedia content, comprising:
extracting one or more content items from a plurality of frames in the multimedia content, wherein the one of more content items comprise one or more keywords displayed during playback of the multimedia content and are obtained through optical character recognition;
calculating a first score and a second score for each of the plurality of frames, the first score is based on a set of dissimilar content items, and the second score is based a set of similar content items, among the plurality of frames; and
identifying the one or more topic transition points in the multimedia content using the calculated first score and the calculated second score for each of the plurality of frames, wherein
the identifying of the one or more topic transition points further comprises
determining a likelihood of a topic transition among the plurality of frames by using
P
(
S
t
=
1
❘
f
S
t
)
=
P
(
f
S
t
❘
S
t
=
1
)
×
P
(
S
t
=
1
)
P
(
f
S
t
)
where P(S t =1|f S t ) represents a conditional likelihood of frame S t to be indicative of the topic transition, when two-dimensional vector f S t ={f 1 S t ,f 2 S t },
P(f S t |S t =1) represents a conditional likelihood of occurrence of f S t ={f 1 S t ,f 2 S t }, when the frame S t is indicative of the topic transition,
P(S t =1) represents a likelihood that the frame S t is indicative of the topic transition, and
P(f S t ) represents a likelihood of occurrence of f S t ={f 1 S t ,f 2 S t } in the frame S t .
3. The process of claim 2 , further comprising:
calculating a saliency score for the one or more content items in the plurality of frames, wherein
the saliency score represents importance of the one or more content items.
4. The process of claim 2 , wherein the calculating of the first score comprises:
determining the first score for the frame S t based on a vector, wherein
the first score is defined by f 1 S t =zF S t , where z is a weight vector, and
the vector is defined by F S t ={V(v 1 ,S t ),V(v 2 ,S t ),V(v 3 ,S t ), . . . }, where F S t is the vector of a set of saliency scores of the set of dissimilar content items in the frame S t , and v 1 , v 2 , v 3 , . . . corresponds to dissimilar content items in the set of dissimilar content items in the frame S t .
5. The process of claim 2 , wherein the calculating of the second score comprises:
determining the second score for the frame S t based on a vector, wherein
the second score is defined by
f
2
S
t
=
Σ
w
∈
c
t
max
V
(
w
,
j
)
❘
j
∈
N
t
V
(
w
,
S
t
)
,
where
maxV(w,j)|j∈N t is a maximum saliency score of similar content item w in the set of similar content items c t in a set of frames N t , and
V(w,S t ) is a saliency score of the similar content item w in the set of similar content items c t in the frame S t .
6. The process of claim 2 , further comprising:
identifying the set of dissimilar content items and the set of similar content items among the plurality of frames by comparing the one or more content items in a frame and one or more content items in a set of frames.
7. The process of claim 2 , wherein the calculating of the first score and the second score comprises utilizing a saliency score associated with each content item in the set of dissimilar content items and the set of similar content items.
8. The process of claim 2 , wherein the identifying of the one or more topic transition points comprises assigning a topic transition score to each frame based on the determined likelihood.
9. The process of claim 8 , wherein the identifying of the one or more topic transition points comprises comparing the topic transition score for each frame to a predetermined topic transition threshold.
10. The process of claim 2 , wherein the identifying of the one or more topic transition points comprises
determining a likelihood of a topic transition by utilizing a normal distribution over the first score and the second score.
11. The process of claim 10 , wherein the normal distribution is defined by
P ( S t =1| f t )= N ( f 1 S t |μ 1 ,σ 1 )× N ( f 2 S t |μ 2 ,σ 2 )
where N(f 1 S t |μ 1 ,σ 1 ) is the normal distribution over f 1 S t ,μ 1 as a means of f 1 S t and σ 1 as a standard deviation of f 1 S t , and
N(f 2 S t |μ 2 ,σ 2 ) is the normal distribution over f 2 S t ,μ 2 as a means of f 2 S t and σ 2 as a standard deviation of f 2 S t .
12. The process of claim 11 , wherein μ 1 is set as max f 1 S t and μ 2 is set as max f 2 S t .