IP Library Granted Patent US 10,681,332
Granted Patent B1
US 10,681,332 · App. 16/047,472 · Granted Jun 9, 2020

Systems and methods for stabilizing views of videos

Inventors: Daryl Stimm (Encinitas, CA); William Edward MacDonald (San Mateo, CA); Kyler William Schwartz (Valley Center, CA)
Assignee: GoPro, Inc.
H04N13/167H04N13/117
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,681,332
App. No.
16/047,472
Granted
Jun 9, 2020
Kind
B1
Abstract

A viewing direction may define an angle/visual portion of a spherical video at which a viewing window is directed. A trajectory of viewing direction may include changes in viewing directions for playback of spherical video. Abrupt changes in the viewing directions may result in jerky or shaky views of the spherical video. Changes in the viewing directions may be stabilized to provide stabilized views of the spherical video. Amount of stabilization may be limited by a margin constraint.

Claims (42)

1. A system that stabilizes views of videos, the system comprising:

one or more physical processors configured by machine-readable instructions to:

obtain video information defining spherical video content, the spherical video content having a progress length and including spherical video frames that define visual content viewable from a point of view as a function of progress through the progress length;

obtain viewing information for the spherical video content, the viewing information defining a trajectory of viewing directions for the spherical video content, the trajectory of viewing directions including viewing directions from the point of view as the function of progress through the progress length;

obtain margin information, the margin information defining a margin constraint that limits deviations from the trajectory of viewing directions;

determine stabilized viewing information based on the viewing information and the margin information, the stabilized viewing information defining a stabilized trajectory of viewing directions for the spherical video content, the stabilized trajectory of viewing directions including stabilized viewing directions from the point of view as the function of progress through the progress length, wherein differences between the trajectory of viewing directions and the stabilized trajectory of viewing directions are limited by the margin constraint; and

present the spherical video content on a display based on the stabilized viewing information.

2. The system of claim 1 , wherein the one or more physical processors are, to determine the stabilized viewing information based on the viewing information, further configured by the machine-readable instructions to determine at least a portion of the stabilized trajectory of viewing directions based on a subsequent portion of the trajectory of viewing directions.

3. The system of claim 2 , wherein:

the trajectory of viewing directions includes a first portion corresponding to a first moment within the progress length and a second portion corresponding to a second moment within the progress length, the second moment subsequent to the first moment;

a portion of the stabilized trajectory of viewing directions corresponding to the first portion of the trajectory of viewing directions is determined based on the second portion of the trajectory of viewing directions.

4. The system of claim 1 , wherein changes in the stabilized viewing directions are smoother than changes in the viewing directions.

5. The system of claim 1 , wherein the viewing directions from the point of view are defined based on rotations about the point of view.

6. The system of claim 5 , wherein the rotations about the point of view include rotations about one or more axes running through the point of view, the one or more axes including a yaw axis, a pitch axis, and/or a roll axis.

7. The system of claim 6 , wherein the margin constraint includes a yaw axis constraint, a pitch axis constraint, and/or a roll axis constraint.

8. The system of claim 6 , wherein the trajectory of viewing directions is defined based on a user's interaction with a display during presentation of the spherical video content on the display.

9. The system of claim 1 , wherein the margin constraint includes a target constraint determined based on positions of a target in the spherical video frames.

10. A method for stabilizing views of videos, the method performed by a computing system including one or more processors, the method comprising:

obtaining, by the computing system, video information defining spherical video content, the spherical video content having a progress length and including spherical video frames that define visual content viewable from a point of view as a function of progress through the progress length;

obtaining, by the computing system, viewing information for the spherical video content, the viewing information defining a trajectory of viewing directions for the spherical video content, the trajectory of viewing directions including viewing directions from the point of view as the function of progress through the progress length;

obtaining, by the computing system, margin information, the margin information defining a margin constraint that limits deviations from the trajectory of viewing directions;

determining, by the computing system, stabilized viewing information based on the viewing information and the margin information, the stabilized viewing information defining a stabilized trajectory of viewing directions for the spherical video content, the stabilized trajectory of viewing directions including stabilized viewing directions from the point of view as the function of progress through the progress length, wherein differences between the trajectory of viewing directions and the stabilized trajectory of viewing directions are limited by the margin constraint; and

presenting, by the computing system, the spherical video content on a display based on the stabilized viewing information.

11. The method of claim 10 , wherein determining the stabilized viewing information based on the viewing information includes determining at least a portion of the stabilized trajectory of viewing directions based on a subsequent portion of the trajectory of viewing directions.

12. The method of claim 11 , wherein:

the trajectory of viewing directions includes a first portion corresponding to a first moment within the progress length and a second portion corresponding to a second moment within the progress length, the second moment subsequent to the first moment;

a portion of the stabilized trajectory of viewing directions corresponding to the first portion of the trajectory of viewing directions is determined based on the second portion of the trajectory of viewing directions.

13. The method of claim 10 , wherein changes in the stabilized viewing directions are smoother than changes in the viewing directions.

14. The method of claim 10 , wherein the viewing directions from the point of view are defined based on rotations about the point of view.

15. The method of claim 14 , wherein the rotations about the point of view include rotations about one or more axes running through the point of view, the one or more axes including a yaw axis, a pitch axis, and/or a roll axis.

16. The method of claim 15 , wherein the margin constraint includes a yaw axis constraint, a pitch axis constraint, and/or a roll axis constraint.

17. The method of claim 15 , wherein the trajectory of viewing directions is defined based on a user's interaction with a display during presentation of the spherical video content on the display.

18. The method of claim 10 , wherein the margin constraint includes a target constraint determined based on positions of a target in the spherical video frames.

19. A system that stabilizes views of videos, the system comprising:

one or more physical processors configured by machine-readable instructions to:

obtain video information defining spherical video content, the spherical video content having a progress length and including spherical video frames that define visual content viewable from a point of view as a function of progress through the progress length;

obtain viewing information for the spherical video content, the viewing information defining a trajectory of viewing directions for the spherical video content, the trajectory of viewing directions including viewing directions from the point of view as the function of progress through the progress length, wherein the viewing directions from the point of view are defined based on rotations about one or more axes running through the point of view, the one or more axes including a yaw axis, a pitch axis, and/or a roll axis;

obtain margin information, the margin information defining a margin constraint that limits deviations from the trajectory of viewing directions, the margin constraint including a yaw axis constraint, a pitch axis constraint, and/or a roll axis constraint:

determine a stabilized viewing information based on the viewing information and the margin information, the stabilized viewing information defining a stabilized trajectory of viewing directions for the spherical video content, the stabilized trajectory of viewing directions including stabilized viewing directions from the point of view as the function of progress through the progress length, wherein differences between the trajectory of viewing directions and the stabilized trajectory of viewing directions are limited by the margin constraint and changes in the stabilized viewing directions are smoother than changes in the viewing directions; and

present the spherical video content on a display based on the stabilized viewing information.

20. The system of claim 19 , wherein the one or more physical processors are, to determine a stabilized viewing information based on the viewing information, further configured by the machine-readable instructions to determine at least a portion of the stabilized trajectory of viewing directions based on a subsequent portion of the trajectory of viewing directions, wherein:

the trajectory of viewing directions includes a first portion corresponding to a first moment within the progress length and a second portion corresponding to a second moment within the progress length, the second moment subsequent to the first moment; a portion of the stabilized trajectory of viewing directions corresponding to the first portion of the trajectory of viewing directions is determined based on the second portion of the trajectory of viewing directions.

Assignments (6)
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: FARALLON CAPITAL MANAGEMENT, L.L.C., AS AGENT
Reel/Frame 072340/0676 →
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 072358/0001 →
RELEASE OF PATENT SECURITY INTEREST Recorded Jan 25, 2021
From: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
To: GOPRO, INC.
Reel/Frame 055106/0434 →
SECURITY INTEREST Recorded Oct 19, 2020
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 054113/0594 →
SECURITY INTEREST Recorded Dec 3, 2018
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 047713/0309 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 20, 2018
From: STIMM, DARYL; MACDONALD, WILLIAM EDWARD; SCHWARTZ, KYLER WILLIAM
To: GOPRO, INC.
Reel/Frame 047555/0791 →
Cited By (2)
US 12,289,522 US 12,519,917