IP Library › Granted Patent US 10,404,916
Granted Patent B2
US 10,404,916 · App. 15/691,240 · Granted Sep 3, 2019

Multi-source video stabilization

Inventors: Pasi Johannes Parkkila (San Diego, CA); Cullum James Baldwin (San Diego, CA)
Assignee: QUALCOMM Incorporated
H04N5/23267H04N5/2258H04N5/23258H04N7/0127H04N13/25
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,404,916
App. No.
15/691,240
Granted
Sep 3, 2019
Kind
B2
Abstract

Producing stabilized video can include: (a) accessing a first video including a sequence of first frames, (b) accessing a second video including a sequence of second frames, and (c) building the stabilized video based on the accessed first and second videos. The stabilized video can include a sequence of stabilized frames. At least one stabilized frame can include pixels derived from at least one of the first frames and at least one of the second frames.

Claims (55)

1. A method of producing a stabilized video, the stabilized video comprising a sequence of stabilized frames, the method comprising:

accessing a first video comprising a sequence of first frames captured by a first camera subject to shake, each first frame having a first field of view;

accessing a second video comprising a sequence of second frames captured by a second camera subject to shake, each second frame having a second field of view; and

generating a first stabilized frame by forming a first composite frame comprising pixels derived from at least one first frame and pixels derived from at least one second frame, wherein generating the first stabilized frame comprises:

mapping a first stabilized field of view of the first stabilized frame with respect to the at least one second frame to define a margin of the at least one second frame; and

populating the margin with pixels derived from the at least one first frame.

2. The method of claim 1 , wherein at least one second field of view is equal to at least one stabilized field of view.

3. The method of claim 2 , wherein a second stabilized frame has the at least one stabilized field of view; the second stabilized frame not comprising pixels derived from any first frames.

4. The method of claim 3 , wherein the at least one stabilized field of view is equal to the first stabilized field of view and the second stabilized frame is earlier in the sequence of stabilized frames than the first stabilized frame.

5. The method of claim 1 , wherein the margin consists of pixels in the at least one second frame external to the first stabilized field of view.

6. The method of claim 1 , wherein generating the first stabilized frame further comprises:

identifying a margin of the first composite frame based on a first stabilized field of view of the first composite frame; and

cropping the margin.

7. The method of claim 6 , wherein at least one second field of view is equal to at least one stabilized field of view.

8. The method of claim 7 , wherein a second stabilized frame has the at least one stabilized field of view; the second stabilized frame not comprising pixels derived from any first frames.

9. The method of claim 8 , wherein the at least one stabilized field of view is equal to the first stabilized field of view and the second stabilized frame is earlier in the sequence of stabilized frames than the first stabilized frame.

10. The method of claim 6 , wherein the margin consists of pixels in the first composite frame external to the first stabilized field of view.

11. The method of claim 1 , wherein a two-dimensional area of each stabilized field of view is greater than or equal to a two-dimensional area of each second field of view.

12. The method of claim 1 , wherein a second stabilized frame with a second stabilized field of view comprises pixels derived from at least one of the second frames, but none of the first frames.

13. The method of claim 12 , wherein each stabilized field of view, including the first stabilized field of view and the second stabilized field of view, is the same.

14. The method of claim 1 , wherein the stabilized fields of view are the same and the method comprises:

presenting a plurality of different second frames;

receiving a user command; and

setting the stabilized field of view as being equal to the second frame presented at the time of receiving the user command.

15. A processing system rising one or more processors configured to:

access a first video comprising a sequence of first frames captured by a first camera subject to shake, each first frame having a first field of view;

access a second video comprising a sequence of second frames captured by a second camera subject to shake, each second frame having a second field of view; and

generating a first stabilized frame by forming a first composite frame comprising pixels derived from at least one of the first frames and pixels derived from at least one of the second frames, wherein generating the first stabilized frame comprises:

mapping the first stabilized field of view with respect to the at least one second frame to define a margin of the at least one second frame; and

populating the margin with pixels derived from the at least one first frame.

16. The processing system of claim 15 , wherein the one or more processors are configured to:

set at least one stabilized field of view as equal to at least one second field of view; and

generate a second stabilized frame that does not comprise pixels derived from any first frames.

17. The processing claim of 15 , wherein the one or more processors are configured to define the margin such that the margin consists of pixels in the at least one second frame external to the first stabilized field of view.

18. The processing system of claim 15 , wherein the one or more processors are configured to prepare the first stabilized frame by:

identifying a first margin of the first composite frame based on the first stabilized field of view; and

cropping the first margin.

19. The processing system of claim 18 , wherein the one or more processors are configured to:

set at least one stabilized field of view as equal to at least one second field of view; and

prepare a second stabilized frame having the at least one stabilized field of view, the second stabilized frame not comprising pixels derived from any first frames.

20. The processing system of claim 18 , wherein the one or more processors are configured to define the margin such that the margin consists of pixels in the first composite frame external to the first stabilized field of view.

21. The processing system of claim 15 , wherein the one or more processors are configured to:

(a) make a two-dimensional area of each stabilized field of view greater than or equal to a two-dimensional area of each second field of view;

(b) prepare a second stabilized frame having a second stabilized field of view where the second stabilized frame comprises pixels derived from at least one of the second frames, but none of the first frames; and

(c) make each stabilized field of view the same.

22. The processing system of claim 15 , wherein the stabilized fields of view are the same and the one or more processors are configured to:

present a plurality of different second frames;

receive a user command; and

set the stabilized field of view as being equal to the second frame presented at the time of receiving the user command.

23. A non-transitory computer-readable medium comprising program code, which, when executed by one or more processors, causes the one or more processors to perform operations, the program code comprising code for:

accessing a first video comprising a sequence of first frames captured by a first camera subject to shake, each first frame having a first field of view; and

accessing a second video comprising a sequence of second frames captured by a second camera subject to shake, each second frame having a second field of view;

generating a first stabilized frame by forming a first composite frame comprising pixels derived from at least one second frame and pixels derived from at least one first frame, wherein generating the first stabilized frame comprises:

mapping the first stabilized field of view with respect to the at least one second frame to define a margin of the at least one second frame;

populating the margin with pixels derived from the at least one first frame.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 20, 2017
From: PARKKILA, PASI JOHANNES; BALDWIN, CULLUM JAMES
To: QUALCOMM INCORPORATED
Reel/Frame 043915/0650 →
Continuity (1)
Related Publication 20190068885A1 · Feb 28, 2019
Cited By (1)
US 12,501,158