IP Library › Granted Patent US 11,310,443
Granted Patent B2
US 11,310,443 · App. 17/244,703 · Granted Apr 19, 2022

Video processing method, apparatus and storage medium

Inventors: Binglin Chang (Beijing, CN); Kailun Ma (Beijing, CN); Zhenghui Shi (Beijing, CN); Qingmin Wang (Beijing, CN)
Assignee: Beijing Xiaomi Pinecone Electronics Co., Ltd.
H04N5/265G06T7/30G06T7/70G11B27/036H04N5/272H04N5/91G06T2207/10016G06T2207/20221
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,310,443
App. No.
17/244,703
Granted
Apr 19, 2022
Kind
B2
Abstract

The present disclosure relates to a video processing method and apparatus, and a storage medium. The method is applied to a terminal and includes: a background frame for a time static special effect is determined from video frames in a video to be processed; for each of the video frames in the video, an image area, where a target object is located is acquired from the respective video frame and the image area is fused into the background frame to generate a special effect frame with the time static special effect.

Claims (58)

1. A video processing method, implemented by a terminal, the method comprising:

selecting, from video frames in a video to be processed, a background frame for a static special effect; and

for each of the video frames in the video, acquiring an image area, where a target object is located, from the respective video frame, and fusing the image area into the background frame, to generate a special effect frame with the static special effect,

wherein selecting, from the video frames in the video to be processed, the background frame for the static special effect comprises:

determining a first frame after the video to be processed enters a time static mode in which the static special effect is formed, as the background frame for the static special effect.

2. The method of claim 1 , wherein fusing the image area into the background frame, to generate the special effect frame with the static special effect comprises:

for each of video frames starting from a start frame in the video, fusing the image area, which is acquired from the respective video frame, into the background frame, to generate the special effect frame with the static special effect; or,

for each of video frames between a start frame and an end frame in the video, fusing the image area, which is acquired from the respective video frame, into the background frame, to generate the special effect frame with the static special effect.

3. The method of claim 2 , further comprising:

displaying the video frames in the video when being recorded;

in response to a number of video frames in a predetermined storage space not exceeding a storage amount threshold, buffering the video frames starting from the start frame; and

fusing the image area, which is acquired from each of the buffered video frames, into the background frame to generate and display the special effect frame.

4. The method of claim 2 , further comprising:

detecting a posture of the target object in each of the video frames;

determining a video frame, in which the posture of the target object is detected as a first predetermined posture, to be the start frame; and

determining a video frame, in which the posture of the target object is detected as a second predetermined posture, to be the end frame.

5. The method of claim 1 , wherein fusing the image area into the background frame to generate the special effect frame with the static special effect comprises:

overlaying an area at a target position in the background frame with the image area; and

feathering edges of the image area with which the background frame was overlaid to form the special effect frame.

6. The method of claim 5 , wherein overlaying the area at the target position in the background frame with the image area comprises:

determining an alignment parameter according to the position of the target object in each of the video frames; and

overlaying, according to the alignment parameter, the area at the target position in the background frame with the image area of the target object.

7. The method of claim 6 , wherein determining the alignment parameter according to the position of the target object in each of the video frames comprises:

selecting a video frame from the video frames in the video as a reference frame; and

determining the alignment parameter according to a position offset of the target object in the video frame relative to the target object in the reference frame.

8. A terminal, comprising:

a processor; and

a memory for storing executable instructions executable by the processor,

wherein the processor is arranged to:

select, from video frames in a video to be processed, a background frame for a static special effect; and

for each of the video frames in the video, acquire, through a multimedia component, an image area, where a target object is located, from the respective video frame and fuse the image area into the background frame, to generate a special effect frame with the static special effect,

wherein the processor is further arranged to:

determine a first frame after the video to be processed enters a time static mode in which the static special effect is formed, as the background frame for the static special effect.

9. The terminal of claim 8 , wherein the processor is further arranged to:

for each of video frames starting from a start frame in the video, fuse the image area, which is acquired from the respective video frame, into the background frame, to generate the special effect frame with the static special effect; or,

for each of video frames between a start frame and an end frame in the video, fuse the image area, which is acquired from the respective video frame, into the background frame, to generate the special effect frame with the static special effect.

10. The terminal of claim 9 , wherein the processor is further arranged to:

display, through the multimedia component, the video frames in the video when being recorded;

in response to a number of video frames in a predetermined storage space not exceeding a storage amount threshold, buffer the video frames starting from the start frame; and

fuse the image area, which is acquired from each of the buffered video frames, into the background frame to generate and display the special effect frame.

11. The terminal of claim 9 , wherein the processor is further arranged to:

detect a posture of the target object in each of the video frames;

determine a video frame, in which the posture of the target object is detected as a first predetermined posture, to be the start frame; and

determine a video frame, in which the posture of the target object is detected as a second predetermined posture, to be the end frame.

12. The terminal of claim 8 , wherein the processor is further arranged to:

overlay an area at a target position in the background frame with the image area; and

feather edges of the image area with which the background frame was overlaid to form the special effect frame.

13. The terminal of claim 12 , wherein the processor is further arranged to:

determine an alignment parameter according to the position of the target object in each of the video frames; and

overlay, according to the alignment parameter, the area at the target position in the background frame with the image area of the target object.

14. The terminal of claim 13 , wherein the processor is further arranged to:

select a video frame from the video frames in the video as a reference frame; and

determine the alignment parameter according to a position offset of the target object in the video frame relative to the target object in the reference frame.

15. A non-transitory computer readable storage medium having computer executable instructions stored thereon, the computer executable instructions, when being executed by a processor, cause the processor to implement:

selecting, from video frames in a video to be processed, a background frame for a static special effect; and

for each of the video frames in the video, acquiring an image area, where a target object is located, from the respective video frame, and fusing the image area into the background frame, to generate a special effect frame with the static special effect,

wherein selecting, from the video frames in the video to be processed, the background frame for the static special effect comprises:

determining a first frame after the video to be processed enters a time static mode in which the static special effect is formed, as the background frame for the static special effect.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 1, 2021
From: CHANG, BINGLIN; MA, KAILUN; SHI, ZHENGHUI; WANG, QINGMIN
To: BEIJING XIAOMI PINECONE ELECTRONICS CO., LTD.
Reel/Frame 056107/0942 →
Priority Claims (1)
CN 202010736003.7 · Jul 28, 2020 · national
Continuity (1)
Related Publication 20220038641A1 · Feb 3, 2022