IP Library › Granted Patent US 10,642,892
Granted Patent B2
US 10,642,892 · App. 15/712,316 · Granted May 5, 2020

Video search method and apparatus

Inventors: Ying Xiao (Shenzhen, CN); Zhenyu Yang (Shenzhen, CN)
Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
G06F16/7328G06F16/739G06F16/743G06F16/745G06F16/783G06K9/00718G06K9/00765G06F16/71G06F16/7837
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,642,892
App. No.
15/712,316
Granted
May 5, 2020
Kind
B2
Abstract

Aspects of the disclosure provide a method for video search. The method includes obtaining a video to be labeled, predicting feature tags respectively for video frames in the video using a classification model, combining at least a first video frame and a second video frame adjacent in time into a video segment when the first video frame and the second video frame have a same predicted feature tag, and labeling the video segment with the same predicted feature tag as a segment feature tag. In an embodiment, the method includes determining a search tag in response to a search request, searching one or more candidate videos that have been labeled with feature tags according to the search tag, and presenting an output video when the output video includes a matching feature tag to the search tag.

Claims (83)

1. A method for video search, comprising:

obtaining a video to be labeled;

selecting feature tags respectively for video frames in the video using a classification model;

combining at least a first video frame and an adjacent second video frame of the video frames into a video segment when the respective feature tag selected using the classification model for the first video frame and the respective feature tag selected using the classification model for the second video frame are matching; and

labeling, by circuitry of an information processing apparatus, the video segment with a segment feature tag corresponding to the matching feature tags.

2. The method according to claim 1 , further comprising:

determining a search tag in response to a search request;

searching one or more candidate videos that have been labeled with feature tags according to the search tag; and

presenting an output video when the output video includes a matching feature tag to the search tag.

3. The method according to claim 1 , wherein the selecting the feature tags respectively for the video frames in the video using the classification model comprises:

extracting a first key video frame in the video to be labeled;

obtaining first image features of the first key video frame; and

selecting a feature tag for the first key video frame according to the first image features using the classification model.

4. The method according to claim 1 , further comprising:

collecting one or more source videos that have been pre-labeled with scenario tags;

extracting second key video frames in the one or more source videos;

determining second feature tags respectively for the second key video frames according to the scenario tags;

obtaining second image features respectively for the second key video frames; and

training the classification model based on the second image features and the second feature tags.

5. The method according to claim 2 , wherein the determining the search tag in response to the search request comprises:

performing a semantic recognition on the search request based on a neural network model; and

determining the search tag in response to the search request according to a semantic recognition result and available feature tags for classification.

6. The method according to claim 5 , further comprising:

collecting search content with corresponding search tags; and

training the neural network model based on the search content and the corresponding search tags.

7. The method according to claim 2 , wherein the determining the search tag in response to the search request further comprises:

receiving the search request in a search box for a video during a playback to search the video during the playback.

8. The method according to claim 7 , wherein the presenting the output video when the output video includes the matching feature tag to the search tag further comprises:

determining, on a progress bar, a specific position corresponding to a specific video segment with the matching feature tag; and

prompting the specific position for a selection of the specific video segment to play.

9. The method according to claim 2 , wherein the determining the search tag in response to the search request further comprises:

receiving the search request in a search box for a plurality of videos to search the plurality of videos.

10. The method according to claim 9 , wherein the presenting the output video when the output video includes the matching feature tag to the search tag further comprises:

presenting a list of output videos respectively in a form of video information and attribute information.

11. An information processing apparatus, comprising:

interface circuitry configured to receive a video to be labeled; and

processing circuitry configured to:

select feature tags respectively for video frames in the video using a classification model;

combine at least a first video frame and an adjacent second video frame of the video frames into a video segment when the respective feature tag selected using the classification model for the first video frame and the respective feature tag selected using the classification model for the second video frame are matching; and

label the video segment with a segment feature tag corresponding to the matching feature tags.

12. The information processing apparatus according to claim 11 , wherein the processing circuitry is configured to:

determine a search tag in response to a search request;

search one or more candidate videos that have been labeled with feature tags according to the search tag; and

present an output video when the output video includes a matching feature tag to the search tag.

13. The information processing apparatus according to claim 11 , wherein

the processing circuitry is configured to:

extract a first key video frame in the video to be labeled;

obtain first image features of the first key video frame; and

select a feature tag for the first key video frame according to the first image features using the classification model.

14. The information processing apparatus according to claim 11 , wherein the processing circuitry is configured to:

receive one or more source videos that have been pre-labeled with scenario tags;

extract second key video frames in the one or more source videos;

determine second feature tags respectively for the second key video frames according to the scenario tags;

obtain second image features respectively for the second key video frames; and

train the classification model based on the second image features and the second feature tags.

15. The information processing apparatus according to claim 12 , wherein the processing circuitry is configured to:

perform a semantic recognition on the search request based on a neural network model, and

determine the search tag in response to the search request according to a semantic recognition result and available feature tags for classification.

16. The information processing apparatus according to claim 15 , wherein the processing circuitry is configured to:

receive collected search content with corresponding search tags, and

train the neural network model based on the search content and the corresponding search tags.

17. The information processing apparatus according to claim 12 , wherein the interface circuitry is configured to

receive the search request that is entered in a search box associated with a video in play to request a search of the video in play.

18. The information processing apparatus according to claim 17 , wherein the processing circuitry is configured to

determine, on a progress bar, a specific position corresponding to a specific video segment with the matching feature tag; and

prompt, via the interface circuitry, the specific position for a selection of the specific video segment to play.

19. The information processing apparatus according to claim 12 , wherein the interface circuitry is configured to

receive the search request that is entered in a search box associated with a plurality of videos to search the plurality of videos.

20. The information processing apparatus according to claim 19 , wherein the processing circuitry is configured to

present, via the interface circuitry, a list of output videos respectively in a form of video information and attribute information.

21. A method for video search, comprising:

determining a search tag in response to a search request;

searching one or more candidate videos that have been labeled with feature tags according to the search tag; and

presenting an output video when the output video includes a matching feature tag to the search tag, wherein

the presenting the output video includes indicating, on a progress bar of the output video, one or more ranges respectively corresponding to one or more video segments having the matching feature tag, and

receiving a selection of one of the ranges for playing a corresponding one of the video segments.

22. The method according to claim 21 , wherein the determining the search tag in response to the search request further comprises:

in response to a determination that an indicator has been moved into a search icon area during playback of a video, displaying a search box;

receiving the search tag as input into the search box; and

searching the video as a candidate video according to the search tag.

23. The method according to claim 21 , wherein

the indicating the one or more ranges on the progress bar includes displaying one or more video frames respectively associated with each of the one or more ranges, and

the receiving the selection of the one of the ranges includes receiving a selection of a video frame associated with the one of the ranges.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 22, 2017
From: XIAO, YING; YANG, ZHENYU
To: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
Reel/Frame 043661/0090 →
Priority Claims (1)
CN 2015 1 1017439 · Dec 30, 2015 · national
Continuity (2)
Continuation PCTCN2016112390 · Dec 27, 2016
Related Publication 20180025079A1 · Jan 25, 2018
Cited By (3)
US 12,225,270 US 12,277,171 US 12,356,048