Granted Patent B2

US 12,256,147 · App. 18/345,229 · Granted Mar 18, 2025

Inventors: Thomas Derbanne (Paris, FR); Cesar Douady (Orsay, FR); Maxim Karpushin (Paris, FR)

Assignee: GoPro, Inc.

H04N23/6812H04N23/683H04N23/51

Abstract

Claims (39)

1. A system for stabilizing videos, the system comprising:

one or more physical processors configured by machine-readable instructions to:

obtain images captured by an image capture device during a capture duration, rotational positions of the image capture device during the capture duration reflected within a trajectory of the image capture device;

stabilize the trajectory of the image capture device based on a look ahead of the trajectory of the image capture device and a set of constraints, wherein changes in the rotational positions of the image capture device during the capture duration are smoothed in the stabilized trajectory of the image capture device, wherein the trajectory of the image capture device is stabilized to generate a path that minimizes a combination of rotational velocity and rotational acceleration of the image capture device while respecting the set of constraints, the rotational velocity of the image capture device including a time derivative of the rotational positions of the image capture device and the rotational acceleration of the image capture device including a second time derivative of the rotational positions of the image capture device; and

generate video frames of a stabilized video based on the images and the stabilized trajectory of the image capture device.

2. The system of claim 1 , wherein the stabilization of the trajectory of the image capture device based on the look ahead of the trajectory of the image capture device includes use of a subsequent portion of the trajectory of the image capture device to determine whether motion of the image capture device in a preceding portion of the trajectory of the image capture device is intended or not.

3. The system of claim 2 , wherein intended motion of the image capture device is preserved in the stabilized trajectory of the image capture device.

4. The system of claim 1 , wherein the generation of the video frames of the stabilized video based on the images and the stabilized trajectory of the image capture device includes:

determination of punch-outs of the images based on the stabilized trajectory of the image capture device; and

generation of the video frames of the stabilized video based on the punch-outs of the images.

5. The system of claim 1 , wherein the rotational velocity and the rotational acceleration of the image capture device have different affect on the minimization of the combination of the rotational velocity and the rotational acceleration of the image capture device for the generation of the path.

6. The system of claim 1 , wherein the set of constraints includes a margin constraint, the margin constraint providing a limitation on deviation of the stabilized trajectory of the image capture device from the trajectory of the image capture device.

7. The system of claim 1 , wherein the set of constraints includes a target constraint, the target constraint requiring a target in the images to be within punch-outs of the images.

8. The system of claim 1 , wherein the images are stored in a buffer during the stabilization of the trajectory of the image capture device such that individual ones of the images corresponding to a portion of the trajectory of the image capture device are stored in the buffer during the stabilization of the portion of the trajectory of the image capture device.

9. A method for stabilizing videos, the method performed by a computing system including one or more processors, the method comprising:

obtaining, by the computing system, images captured by an image capture device during a capture duration, rotational positions of the image capture device during the capture duration reflected within a trajectory of the image capture device;

stabilizing, by the computing system, the trajectory of the image capture device based on a look ahead of the trajectory of the image capture device and a set of constraints, wherein changes in the rotational positions of the image capture device during the capture duration are smoothed in the stabilized trajectory of the image capture device, wherein the trajectory of the image capture device is stabilized to generate a path that minimizes a combination of rotational velocity and rotational acceleration of the image capture device while respecting the set of constraints, the rotational velocity of the image capture device including a time derivative of the rotational positions of the image capture device and the rotational acceleration of the image capture device including a second time derivative of the rotational positions of the image capture device; and

generating, by the computing system, video frames of a stabilized video based on the images and the stabilized trajectory of the image capture device.

10. The method of claim 9 , wherein stabilizing the trajectory of the image capture device based on the look ahead of the trajectory of the image capture device includes using a subsequent portion of the trajectory of the image capture device to determine whether motion of the image capture device in a preceding portion of the trajectory of the image capture device is intended or not.

11. The method of claim 10 , wherein intended motion of the image capture device is preserved in the stabilized trajectory of the image capture device.

12. The method of claim 9 , wherein generating the video frames of the stabilized video based on the images and the stabilized trajectory of the image capture device includes:

determining punch-outs of the images based on the stabilized trajectory of the image capture device; and

generating the video frames of the stabilized video based on the punch-outs of the images.

13. The method of claim 9 , wherein the rotational velocity and the rotational acceleration of the image capture device have different affect on the minimization of the combination of the rotational velocity and the rotational acceleration of the image capture device for the generation of the path.

14. The method of claim 9 , wherein the set of constraints includes a margin constraint, the margin constraint providing a limitation on deviation of the stabilized trajectory of the image capture device from the trajectory of the image capture device.

15. The method of claim 9 , wherein the set of constraints includes a target constraint, the target constraint requiring a target in the images to be within punch-outs of the images.

16. The method of claim 9 , wherein the images are stored in a buffer during the stabilization of the trajectory of the image capture device such that individual ones of the images corresponding to a portion of the trajectory of the image capture device are stored in the buffer during the stabilization of the portion of the trajectory of the image capture device.

17. A system for stabilizing videos, the system comprising:

one or more physical processors configured by machine-readable instructions to:

obtain images captured by an image capture device during a capture duration, rotational positions of the image capture device during the capture duration reflected within a trajectory of the image capture device, the trajectory of the image capture device including a first portion corresponding to a first moment within the capture duration and a second portion corresponding to a second moment subsequent to the first moment within the capture duration;

stabilize the trajectory of the image capture device based on a set of constraints and a look ahead of the trajectory of the image capture device by using a subsequent portion of the trajectory of the image capture device to stabilize a preceding portion of the trajectory of the image capture device, wherein the first portion of the trajectory of the image capture device corresponding to the first moment within the capture duration is stabilized based on the second portion of the trajectory of the image capture device corresponding to the second moment subsequent to the first moment within the capture duration, wherein changes in the rotational positions of the image capture device during the capture duration are smoothed in the stabilized trajectory of the image capture device, wherein the trajectory of the image capture device is stabilized to generate a path that minimizes a combination of rotational velocity and rotational acceleration of the image capture device while respecting the set of constraints, the rotational velocity of the image capture device including a time derivative of the rotational positions of the image capture device and the rotational acceleration of the image capture device including a second time derivative of the rotational positions of the image capture device; and

generate video frames of a stabilized video based on the images and the stabilized trajectory of the image capture device.

18. The system of claim 17 , wherein:

the images are stored in a buffer during the stabilization of the trajectory of the image capture device such that individual ones of the images corresponding to a portion of the trajectory of the image capture device are stored in the buffer during the stabilization of the portion of the trajectory of the image capture device; and

the generation of the video frames of the stabilized video based on the images and the stabilized trajectory of the image capture device includes:

determination of punch-outs of the images based on the stabilized trajectory of the image capture device; and

generation of the video frames of the stabilized video based on the punch-outs of the images.

19. The system of claim 17 , wherein the rotational velocity and the rotational acceleration of the image capture device have different affect on the minimization of the combination of the rotational velocity and the rotational acceleration of the image capture device for the generation of the path.

20. The system of claim 17 , wherein the set of constraints includes a margin constraint and a target constraint, the margin constraint providing a limitation on deviation of the stabilized trajectory of the image capture device from the trajectory of the image capture device, the target constraint requiring a target in the images to be within punch-outs of the images.

Assignments (3)

Continuity (9)

References Cited (281)