IP Library Granted Patent US 11,625,433
Granted Patent B2
US 11,625,433 · App. 17/182,960 · Granted Apr 11, 2023

Method and apparatus for searching video segment, device, and medium

Inventors: Xiang Long (Beijing, CN); Ping Wang (Beijing, CN); Fu Li (Beijing, CN); Dongliang He (Beijing, CN); Hao Sun (Beijing, CN); Shilei Wen (Beijing, CN)
Assignee: Beijing Baidu Netcom Science and Technology Co., Ltd.
G06F16/7847G06F18/22G06V20/46G06V20/48
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,625,433
App. No.
17/182,960
Granted
Apr 11, 2023
Kind
B2
Abstract

Embodiments of the present disclosure disclose a method and apparatus for searching a video segment, a device and a medium, and relate to the field of video data search. The method includes: sampling video frames from a target video and videos to be searched in a video library, and extracting features from the sampled frames; matching the target video and the videos to be searched according to the extracted features to determine a candidate video to be searched that matches the target video; determining at least one candidate video segment from the determined candidate video, and calculating a degree of matching between the target video and each candidate video segment based on the extracted features of each sampled frame; and determining a video segment matching the target video in the videos to be searched according to the calculated degree of matching between the target video and each candidate video segment.

Claims (60)

1. A method for searching a video segment, comprising:

sampling video frames from a target video and videos to be searched in a video library, and extracting features from the sampled video frames;

matching the target video and the videos to be searched according to the extracted features to determine a candidate video to be searched that matches the target video;

determining at least one candidate video segment from the determined candidate video to be searched, wherein a first ratio of a length of a first candidate video segment alone to a length of the target video is within a preset interval, and a second ratio of a length of a second candidate video segment alone to the length of the target video is within the preset interval, and calculating a degree of matching between the target video and each candidate video segment based on the extracted features of each sampled frame; and

determining a video segment matching the target video in the videos to be searched according to the degree of matching between the target video and each candidate video segment.

2. The method according to claim 1 , wherein the matching comprises:

determining, according to the extracted features of the sampled video frames, matching results between the sampled video frames in the target video and the sampled video frames in the videos to be searched; and

counting the matching results, and determining, according to the counted matching results, the candidate video to be searched that matches the target video.

3. The method according to claim 1 , wherein the determining at least one candidate video segment from the candidate video to be searched comprises:

combining every two sampled video frames of the candidate video to be searched, to obtain at least one sampled frame pair, and using the two sampled video frames in the sampled frame pair as a start frame and an end frame of a candidate video segment to obtain the at least one candidate video segment.

4. The method according to claim 1 , wherein the calculating comprises:

calculating a similarity between each sampled frame in the target video and each sampled frame in the candidate video segment based on the extracted features of each sampled frame; and

determining the degree of matching between the target video and the candidate video segment based on the similarity between each sampled frame in the target video and each sampled frame in the candidate video segment.

5. The method according to claim 4 , wherein the determining the degree of matching between the target video and the candidate video segment comprises:

determining a similarity between the target video and the candidate video segment based on the similarity between each sampled frame in the target video and each sampled frame in the candidate video segment; and

superimposing the similarity between the target video and the candidate video segment with a preset value of a reward function to obtain the degree of matching between the target video and the candidate video segment,

wherein the preset value of the reward function is positively correlated with a length of the candidate video segment.

6. The method according to claim 1 , wherein the determining a video segment matching the target video in the videos to be searched comprises:

determining, from the candidate video segment, a target candidate video segment whose degree of matching with the target video satisfies a preset condition; and

searching previous frames based on a position of a sampled frame serving as a start frame of the target candidate video segment in a corresponding video to be searched, searching subsequent frames based on a position of the sampled frame serving as an end frame of the target candidate video segment in the corresponding video to be searched, and determining a start frame and an end frame of the video segment matching the target video in the corresponding video to be searched.

7. An electronic device, comprising:

one or more processors; and

a storage apparatus, storing one or more programs,

wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform an operation for searching a video segment, comprising:

sampling video frames from a target video and videos to be searched in a video library, and extracting features from the sampled video frames;

matching the target video and the videos to be searched according to the extracted features to determine a candidate video to be searched that matches the target video;

determining at least one candidate video segment from the determined candidate video to be searched, wherein a first ratio of a length of a first candidate video segment alone to a length of the target video is within a preset interval, and a second ratio of a length of a second candidate video segment alone to the length of the target video is within the preset interval, and calculating a degree of matching between the target video and each candidate video segment based on the extracted features of each sampled frame; and

determining a video segment matching the target video in the videos to be searched according to the degree of matching between the target video and each candidate video segment.

8. The electronic device according to claim 7 , wherein the matching comprises:

determining, according to the extracted features of the sampled video frames, matching results between the sampled video frames in the target video and the sampled video frames in the videos to be searched; and

counting the matching results, and determining, according to the counted matching results, the candidate video to be searched that matches the target video.

9. The electronic device according to claim 7 , wherein the determining at least one candidate video segment from the candidate video to be searched comprises:

combining every two sampled video frames of the candidate video to be searched, to obtain at least one sampled frame pair, and using the two sampled video frames in the sampled frame pair as a start frame and an end frame of a candidate video segment to obtain the at least one candidate video segment.

10. The electronic device according to claim 7 , wherein the calculating comprises:

calculating a similarity between each sampled frame in the target video and each sampled frame in the candidate video segment based on the extracted features of each sampled frame; and

determining the degree of matching between the target video and the candidate video segment based on the similarity between each sampled frame in the target video and each sampled frame in the candidate video segment.

11. The electronic device according to claim 10 , wherein the determining the degree of matching between the target video and the candidate video segment comprises:

determining a similarity between the target video and the candidate video segment based on the similarity between each sampled frame in the target video and each sampled frame in the candidate video segment; and

superimposing the similarity between the target video and the candidate video segment with a preset value of a reward function to obtain the degree of matching between the target video and the candidate video segment,

wherein the preset value of the reward function is positively correlated with a length of the candidate video segment.

12. The electronic device according to claim 7 , wherein the determining a video segment matching the target video in the videos to be searched comprises:

determining, from the candidate video segment, a target candidate video segment whose degree of matching with the target video satisfies a preset condition; and

searching previous frames based on a position of a sampled frame serving as a start frame of the target candidate video segment in a corresponding video to be searched, searching subsequent frames based on a position of the sampled frame serving as an end frame of the target candidate video segment in the corresponding video to be searched, and determining a start frame and an end frame of the video segment matching the target video in the corresponding video to be searched.

13. A non-transitory computer-readable storage medium, storing a computer program thereon, wherein the computer program, when executed by a processor, causes the processor to perform an operation for searching a video segment, comprising:

sampling video frames from a target video and videos to be searched in a video library, and extracting features from the sampled video frames;

matching the target video and the videos to be searched according to the extracted features to determine a candidate video to be searched that matches the target video;

determining at least one candidate video segment from the determined candidate video to be searched, wherein a first ratio of a length of a first candidate video segment alone to a length of the target video is within a preset interval, and a second ratio of a length of a second candidate video segment alone to the length of the target video is within the preset interval, and calculating a degree of matching between the target video and each candidate video segment based on the extracted features of each sampled frame; and

determining a video segment matching the target video in the videos to be searched according to the degree of matching between the target video and each candidate video segment.

14. The non-transitory computer-readable storage medium according to claim 13 , wherein the matching comprises:

determining, according to the extracted features of the sampled video frames, matching results between the sampled video frames in the target video and the sampled video frames in the videos to be searched; and

counting the matching results, and determining, according to the counted matching results, the candidate video to be searched that matches the target video.

15. The non-transitory computer-readable storage medium according to claim 13 , wherein the determining at least one candidate video segment from the candidate video to be searched comprises:

combining every two sampled video frames of the candidate video to be searched, to obtain at least one sampled frame pair, and using the two sampled video frames in the sampled frame pair as a start frame and an end frame of a candidate video segment to obtain the at least one candidate video segment.

16. The non-transitory computer-readable storage medium according to claim 13 , wherein the calculating comprises:

calculating a similarity between each sampled frame in the target video and each sampled frame in the candidate video segment based on the extracted features of each sampled frame; and

determining the degree of matching between the target video and the candidate video segment based on the similarity between each sampled frame in the target video and each sampled frame in the candidate video segment.

17. The non-transitory computer-readable storage medium according to claim 16 , wherein the determining the degree of matching between the target video and the candidate video segment comprises:

determining a similarity between the target video and the candidate video segment based on the similarity between each sampled frame in the target video and each sampled frame in the candidate video segment; and

superimposing the similarity between the target video and the candidate video segment with a preset value of a reward function to obtain the degree of matching between the target video and the candidate video segment,

wherein the preset value of the reward function is positively correlated with a length of the candidate video segment.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 23, 2021
From: LONG, XIANG; WANG, PING; LI, FU; HE, DONGLIANG; SUN, HAO; WEN, SHILEI
To: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY CO., LTD.
Reel/Frame 055375/0936 →
Priority Claims (1)
CN 202010274782.3 · Apr 9, 2020 · national
Continuity (1)
Related Publication 20210319062A1 · Oct 14, 2021