IP Library Granted Patent US 11,558,549
Granted Patent B2
US 11,558,549 · App. 17/159,984 · Granted Jan 17, 2023

Methods and devices for capturing high-speed and high-definition videos

Inventors: Juwei Lu (North York, CA); Wei Li (North York, CA); Jin Tang (Markham, CA)
Assignee: Huawei Technologies Co., Ltd.
H04N5/23232G06T7/207H04N5/23222H04N5/783
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,558,549
App. No.
17/159,984
Granted
Jan 17, 2023
Kind
B2
Abstract

Methods and devices for generating a slow motion video segment are described. A first set of video frames captures a video view at a first resolution and at a first frame rate. A second set of video frames captures the video view at a second lower resolution, and at a second frame rate that is greater for at least a portion of the second set. At least two high resolution frames are identified in the first set for generating the slow motion video segment. One or more low resolution frames are identified in the second set corresponding to an inter-frame time period between the identified high resolution frames. The slow motion video segment is generated by generating at least one high resolution frame corresponding to the inter-frame time period using interpolation based on the identified high resolution frames and the identified low resolution frames.

Claims (79)

1. A device for generating a slow motion video segment, the device comprising:

a memory;

a processor coupled to the memory and configured to execute computer-readable instructions to cause the device to:

receive a first video capturing a video view at a first resolution and at a first frame rate;

receive a second video capturing the video view at a second resolution, lower than the first resolution, and at a second frame rate, the second frame rate being a variable frame rate;

estimate a speed of motion of an object in the video view;

in response to the estimated speed being at or above a defined speed threshold, control the second frame rate to be at a high frame rate that is greater than the first frame rate;

vary the second frame rate to be proportional to the estimated speed;

store, in the memory, the received first video as a first set of video frames;

store, in the memory, the received second video as a second set of video frames; and

in response to instructions to generate the slow motion video segment:

identify at least two high resolution frames in the first set for generating the slow motion video segment;

identify whether there is at least one low resolution frame in the second set corresponding to an inter-frame time period between the identified at least two high resolution frames; and

when there is at least one low resolution frame corresponding to the inter-frame time period,

generate the slow motion video segment by generating at least one high resolution frame corresponding to the inter-frame time period using interpolation based on the identified at least two high resolution frames and the at least one low resolution frame; and

an output device coupled to the processor for displaying the slow motion video segment using the at least one generated high resolution frame.

2. The device of claim 1 , wherein the processor is further configured to execute instructions to cause the device to generate the at least one high resolution frame using a linear interpolation.

3. The device of claim 1 , wherein the instructions to generate the slow motion video segment include an indication of a time period for generating the slow motion video segment, and wherein the processor is further configured to execute instructions to cause the device to identify the at least two high resolution frames by identifying at least two frames in the first set spanning the indicated time period.

4. The device of claim 1 further comprising:

a first camera for capturing the first video; and

a second camera for capturing the second video;

wherein the processor is coupled to the first and second cameras to control operation of the first and second cameras; and

wherein the first and second cameras are provided on a same side of the device.

5. The device of claim 1 , wherein the second frame rate is higher than the first frame rate for a portion of the second set.

6. The device of claim 1 , wherein the second frame rate is substantially constant for all of the second set of video frames.

7. The device of claim 1 , wherein the first and second sets of video frames are stored in the memory in association with each other.

8. The device of claim 7 , wherein each frame in the second set of video frames includes a relative timestamp providing temporal relation to at least one frame in the first set of video frames.

9. The device of claim 1 , wherein the memory further stores a third set of video frames, the third set capturing the video view at a third resolution and at a third frame rate, the third resolution being lower than the second resolution, and the third frame rate being greater than the second frame rate for at least a portion of the third set, and wherein the processor is further configured to execute instructions to cause the device to:

generate the slow motion video segment recursively by:

generating a set of intermediate slow motion video frames, having an intermediate resolution, using the second and third sets of video frames; and

generating the slow motion video segment using the set of intermediate slow motion video frames and the first set of video frames.

10. The device of claim 1 , wherein to estimate the speed of motion of the object in the video view, the computer-readable instructions further cause the device to:

control the second frame rate to be a moderate frame rate that is lower than the high frame rate;

store a moving window of the second set of video frames in a video buffer as buffered frames; and

process the buffered frames to detect a magnitude of high-speed motion in the video view.

11. The device of claim 10 , wherein processing the buffered frames includes analyzing an entire area of each buffered frame.

12. The device of claim 10 , wherein processing the buffered frames includes analyzing only a portion of each buffered frame.

13. A method for generating a slow motion video segment, the method comprising:

receiving a first video capturing a video view at a first resolution and at a first frame rate;

receiving a second video capturing the video view at a second resolution, lower than the first resolution, and at a second frame rate, the second frame rate being a variable frame rate;

estimating a speed of motion of an object in the video view;

in response to the estimated speed being at or above a defined speed threshold, controlling the second frame rate to be at a high frame rate that is greater than the first frame rate;

varying the second frame rate to be proportional to the estimated speed;

storing the received first video as a first set of video frames;

storing the received second video as a second set of video frames; and

in response to instructions to generate the slow motion video segment:

identifying at least two high resolution frames in the first set for generating the slow motion video segment;

identifying whether there is at least one low resolution frame in the second set corresponding to an inter-frame time period between the identified at least two high resolution frames;

when there is at least one low resolution frames corresponding to the inter-frame time period,

generating the slow motion video segment by generating at least one high resolution frame corresponding to the inter-frame time period using interpolation based on the identified at least two high resolution frames and the at least one low resolution frame; and

providing the at least one generated high resolution frame to an output device to display the slow motion video segment.

14. The method of claim 13 , wherein the interpolation is a linear interpolation.

15. The method of claim 13 , wherein the instructions to generate the slow motion video segment include an indication of a time period for generating the slow motion video segment, and wherein identifying the at least two high resolution frames includes identifying at least two frames in the first set spanning the indicated time period.

16. The method of claim 13 , further comprising:

capturing the first video using a first camera; and

capturing the second video using a second camera;

wherein first and second cameras are provided on a same side of a video capture device.

17. The method of claim 13 , wherein the second frame rate is substantially constant for all of the second set of video frames.

18. The method of claim 13 , further comprising storing the first and second sets of video frames in a memory in association with each other.

19. The method of claim 13 , further comprising:

obtaining a third set of video frames, the third set capturing the video view at a third resolution and at a third frame rate, the third resolution being lower than the second resolution, and the third frame rate being greater than the second frame rate for at least a portion of the third set;

in response to instructions to generate the slow motion video segment:

generating the slow motion video segment recursively by:

generating a set of intermediate slow motion video frames, having an intermediate resolution, using the second and third sets of video frames; and

generating the slow motion video segment using the set of intermediate slow motion video frames and the first set of video frames.

20. A non-transitory computer-readable medium having machine-executable instructions stored thereon, the machine-executable instructions, when executed by a processor of a device, cause the device to:

receive a first set of video frames capturing a video view at a first resolution and at a first frame rate;

receive a second set of video frames capturing the video view at a second resolution, lower than the first resolution, and at a second frame rate, the second frame rate being a variable frame rate;

estimate a speed of motion of an object in the video view;

in response to the estimated speed being at or above a defined speed threshold, control the second frame rate to be at a high frame rate that is greater than the first frame rate;

vary the second frame rate to be proportional to the estimated speed;

store the received first set of video frames as a first set of video frames;

store the received second set of video frames as a second set of video frames; and

in response to instructions to generate a slow motion video segment:

identify at least two high resolution frames in the first set for generating the slow motion video segment;

identify whether there is at least one low resolution frame in the second set corresponding to an inter-frame time period between the identified at least two high resolution frames;

when there is at least one low resolution frame corresponding to the inter-frame time period,

generate the slow motion video segment by generating at least one high resolution frame corresponding to the inter-frame time period using interpolation based on the identified at least two high resolution frames and the at least one low resolution frame; and

provide the at least one generated high resolution frame to an output device to display the slow motion video segment.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 17, 2021
From: LU, JUWEI; LI, WEI; TANG, JIN
To: HUAWEI TECHNOLOGIES CO., LTD.
Reel/Frame 056258/0049 →
Continuity (2)
Continuation PCTCN2018109359 · Oct 8, 2018
Related Publication 20210152739A1 · May 20, 2021
Cited By (1)
US 12,342,073