IP Library › Granted Patent US 11,748,986
Granted Patent B2
US 11,748,986 · App. 17/208,684 · Granted Sep 5, 2023

Method and apparatus for recognizing key identifier in video, device and storage medium

Inventors: Ruifeng Li (Beijing, CN); Lei Yuan (Beijing, CN); Xing Dai (Beijing, CN); Qinglin Qin (Beijing, CN); Zhuo Sun (Beijing, CN); Zhiqiang Liu (Beijing, CN)
Assignee: Beijing Baidu Netcom Science and Technology Co., Ltd.
G06V20/46G06F18/214G06V20/635
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,748,986
App. No.
17/208,684
Granted
Sep 5, 2023
Kind
B2
Abstract

A method and an apparatus for recognizing a key identifier in a video, a device and a storage medium are disclosed. The method includes: extracting a plurality of key frames from the video; generating a mask of the key identifier by using a difference between the plurality of key frames; determining, in video frames of the video, a key identifier area image by using the mask; and recognizing the key identifier area image to obtain a key identifier category included in the video.

Claims (55)

1. A method for recognizing a key identifier in a video, comprising:

extracting a plurality of key frames from the video;

generating a mask of the key identifier by using a difference between the plurality of key frames;

determining, in video frames of the video, a key identifier area image by using the mask; and

recognizing the key identifier area image to obtain a key identifier category comprised in the video,

wherein the extracting the plurality of key frames from the video, comprises:

extracting a key frame of a scene change in the video according to a difference between adjacent frames in the video, and

wherein the extracting the key frame of the scene change in the video according to the difference between adjacent frames in the video, comprises:

acquiring three consecutive frames including a current frame from the video;

calculating mean absolute differences between pixels of every two adjacent frames in the three consecutive frames to obtain a first mean absolute difference and a second mean absolute difference;

calculating a first difference value between the first mean absolute difference and the second mean absolute difference; and

extracting the current frame among the three consecutive frames as the key frame, in a case that a minimum value of the first mean absolute difference, the second mean absolute difference, and the first difference value is greater than a first threshold.

2. The method of claim 1 , further comprising:

performing a black border detection on the key frame, and removing a black border area of the key frame.

3. The method of claim 1 , wherein the mask comprises a non-dynamic area in the key frame, and the generating the mask of the key identifier by using the difference between the plurality of key frames, comprises:

converting the plurality of key frames into grayscale images, respectively;

calculating a difference value between the grayscale image of a key frame to be processed and the grayscale image of a previous key frame, to obtain a second difference value corresponding to each pixel position of the key frame to be processed; and

traversing the key frames to be processed, and determining a pixel position with a pixel value in the grayscale image of the key frame to be processed greater than a minimum pixel value, and/or with the second difference value less than a maximum change value as a pixel position belonging to the mask.

4. The method of claim 3 , wherein the generating the mask of the key identifier by using the difference between the plurality of key frames, further comprises:

obtaining an intersection of the pixel position belonging to the mask in the key frame to be processed and the pixel position belonging to the mask in the previous key frame.

5. The method of claim 1 , further comprising:

performing corrosion, expansion and connection processing on the mask.

6. The method of claim 1 , wherein the determining, in video frames of the video, the key identifier area image by using the mask, comprises:

using the mask and the video frames of the video to perform alpha calculation, extracting at least one key identifier area image, and filling other parts of the video frames with a first pixel value.

7. The method of claim 1 , further comprising:

dividing the filled video frames into a plurality of areas;

extracting area images where four corners of the divided video frames are located; and

determining a key identifier area image for recognition according to the area images where the four corners are located and the extracted key identifier area image.

8. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method of claim 1 .

9. An apparatus for recognizing a key identifier in a video, comprising:

a processor and a memory for storing one or more computer programs executable by the processor,

wherein when executing at least one of the computer programs, the processor is configured to perform operations comprising:

extracting a plurality of key frames from the video;

generating a mask of the key identifier by using a difference between the plurality of key frames;

determining, in video frames of the video, a key identifier area image by using the mask; and

recognizing the key identifier area image to obtain a key identifier category comprised in the video,

wherein when executing at least one of the computer programs, the processor is further configured to perform operations comprising extracting a key frame of a scene change in the video according to a difference between adjacent frames in the video, and

wherein when executing at least one of the computer programs, the processor is further configured to perform operations comprising:

acquiring three consecutive frames including a current frame from the video;

calculating mean absolute differences between pixels of every two adjacent frames in the three consecutive frames to obtain a first mean absolute difference and a second mean absolute difference;

calculating a first difference value between the first mean absolute difference and the second mean absolute difference; and

extracting the current frame among the three consecutive frames as the key frame, in a case that a minimum value of the first mean absolute difference, the second mean absolute difference, and the first difference value is greater than a first threshold.

10. The apparatus of claim 9 , wherein when executing at least one of the computer programs, the processor is further configured to perform operations comprising:

performing a black border detection on the key frame, and removing a black border area of the key frame.

11. The apparatus of claim 9 , wherein the mask comprises a non-dynamic area in the key frame, and wherein when executing at least one of the computer programs, the processor is further configured to perform operations comprising:

converting the plurality of key frames into grayscale images, respectively;

calculating a difference value between the grayscale image of a key frame to be processed and the grayscale image of a previous key frame, to obtain a second difference value corresponding to each pixel position of the key frame to be processed; and

traversing the key frames to be processed, and determining a pixel position with a pixel value in the grayscale image of the key frame to be processed greater than a minimum pixel value, and/or with the second difference value less than a maximum change value as a pixel position belonging to the mask.

12. The apparatus of claim 11 , wherein when executing at least one of the computer programs, the processor is further configured to perform operations comprising:

obtaining an intersection of the pixel position belonging to the mask in the key frame to be processed and the pixel position belonging to the mask in the previous key frame.

13. The apparatus of claim 9 , wherein when executing at least one of the computer programs, the processor is further configured to perform operations comprising:

performing corrosion, expansion and connection processing on the mask.

14. The apparatus of claim 9 , wherein when executing at least one of the computer programs, the processor is further configured to perform operations comprising: using the mask and the video frames of the video to perform alpha calculation, extracting at least one key identifier area image, and filling other parts of the video frames with a first pixel value.

15. The apparatus of claim 9 , wherein when executing at least one of the computer programs, the processor is further configured to perform operations comprising:

dividing the filled video frames into a plurality of areas; extracting area images where four corners of the divided video frames are located; and determining a key identifier area image for recognition according to the area images where the four corners are located and the extracted key identifier area image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 22, 2021
From: LI, RUIFENG; YUAN, LEI; DAI, XING; QIN, QINGLIN; SUN, ZHUO; LIU, ZHIQIANG
To: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY CO., LTD.
Reel/Frame 055675/0721 →
Priority Claims (1)
CN 202010599295.4 · Jun 28, 2020 · national
Continuity (1)
Related Publication 20210209375A1 · Jul 8, 2021