IP Library › Granted Patent US 11,729,364
Granted Patent B2
US 11,729,364 · App. 17/018,679 · Granted Aug 15, 2023

Circular stitching of images

Inventors: Bruno César Douady (Orsay, FR); Alexis Lefebvre (Paris, FR)
Assignee: GoPro, Inc.
H04N13/156G06T3/4038G06T7/593H04N13/239H04N23/45G06T2207/20221G06T2207/20224
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,729,364
App. No.
17/018,679
Granted
Aug 15, 2023
Kind
B2
Abstract

Systems and methods are disclosed for circular stitching of images. For example, methods may include accessing a first image captured using a first image sensor; accessing a second image captured using a second image sensor; determining a cost table for a circular stitching boundary that includes overlapping regions of the first image and the second image; determining an extended disparity profile based on a periodic extension of the cost table and a smoothness criterion, wherein the extended disparity profile has a length greater than the width of the cost table; determining a binocular disparity profile of a length equal to the width of the cost table based on a contiguous subsequence of the extended disparity profile; and stitching the first image and the second image using the binocular disparity profile to obtain a combined image.

Claims (51)

1. A system comprising:

a first image sensor configured to capture images;

a second image sensor configured to capture images; and

a processing apparatus that is configured to:

access a first image captured using the first image sensor;

access a second image captured using the second image sensor;

determine a cost table for a circular stitching boundary that includes overlapping regions of the first image and the second image, wherein the cost table is indexed by a longitude around the circular stitching boundary and indexed by a binocular disparity value, and wherein the cost table has a width corresponding to a circumference of the circular stitching boundary;

determine an extended disparity profile based on a periodic extension of the cost table and a smoothness criterion, wherein the extended disparity profile has a length greater than the width of the cost table;

determine a binocular disparity profile of a length equal to the width of the cost table based on a contiguous subsequence of the extended disparity profile; and

stitch the first image and the second image using the binocular disparity profile to obtain a combined image.

2. The system of claim 1 , wherein the extended disparity profile is constrained to start at a disparity value of zero and end at a disparity value of zero.

3. The system of claim 1 , wherein the processing apparatus that is configured to:

determine a subsequence selection index as a smallest non-negative index such that the

extend disparity profile has a value at the subsequence selection index that is greater than or equal to the value of the extended disparity profile at the subsequence selection index plus the width of the cost table; and

determine the binocular disparity profile based on the subsequence selection index.

4. The system of claim 1 , wherein the length of the extended disparity profile is between ten percent and thirty percent greater than the width of the cost table.

5. The system of claim 1 , wherein the smoothness criterion is a sum of differences between consecutive elements of the extended disparity profile.

6. The system of claim 1 , wherein the processing apparatus that is configured to:

determine the extended disparity profile subject to a constraint that all differences between consecutive elements of the extended disparity profile are less than a maximum disparity change.

7. The system of claim 1 , wherein the first image sensor is attached to the second image sensor and oriented in an opposite direction with respect to the second image sensor, such that the first image sensor and the second image sensor have partially overlapping fields of view.

8. The system of claim 7 , comprising:

a first fisheye lens attached to the first image sensor such that the first image sensor is positioned to detect light incident through the first fisheye lens; and

a second fisheye lens attached to the second image sensor such that the second image sensor is positioned to detect light incident through the second fisheye lens.

9. The system of claim 1 , wherein a height of the cost table corresponds to a range of binocular disparity values to be searched.

10. The system of claim 1 , wherein the combined image is a spherical image.

11. A method comprising:

accessing a first image captured using a first image sensor;

accessing a second image captured using a second image sensor;

determining a cost table for a circular stitching boundary that includes overlapping regions of the first image and the second image, wherein the cost table is indexed by a longitude around the circular stitching boundary and indexed by a binocular disparity value, and wherein the cost table has a width corresponding to a circumference of the circular stitching boundary;

determining an extended disparity profile based on a periodic extension of the cost table and a smoothness criterion, wherein the extended disparity profile has a length greater than the width of the cost table, wherein the extended disparity profile is constrained to start and end at a same disparity value;

determining a binocular disparity profile of a length equal to the width of the cost table based on a contiguous subsequence of the extended disparity profile; and

stitching the first image and the second image using the binocular disparity profile to obtain a combined image.

12. The method of claim 11 , wherein the same disparity value for the start and end of the extended disparity profile is zero.

13. The method of claim 11 , comprising:

determining a subsequence selection index as a smallest non-negative index such that the extend disparity profile has a value at the subsequence selection index that is greater than or equal to the value of the extended disparity profile at the subsequence selection index plus the width of the cost table; and

determining the binocular disparity profile based on the subsequence selection index.

14. The method of claim 11 , wherein the length of the extended disparity profile is between ten percent and thirty percent greater than the width of the cost table.

15. The method of claim 11 , wherein the smoothness criterion is a sum of differences between consecutive elements of the extended disparity profile.

16. The method of claim 11 , wherein determining the extended disparity profile comprises:

determining the extended disparity profile subject to a constraint that all differences between consecutive elements of the extended disparity profile are less than a maximum disparity change.

17. The method of claim 11 , wherein a height of the cost table corresponds to a range of binocular disparity values to be searched.

18. The method of claim 11 , wherein the combined image is a spherical image.

19. The method of claim 11 , comprising:

storing, displaying, or transmitting an output image that is based on the combined image.

20. A non-transitory computer-readable storage medium storing executable instructions that, when executed by a processor, cause performance of operations, comprising:

accessing a first image captured using a first image sensor;

accessing a second image captured using a second image sensor;

determining a cost table for a circular stitching boundary that includes overlapping regions of the first image and the second image, wherein the cost table is indexed by a longitude around the circular stitching boundary and indexed by a binocular disparity value, and wherein the cost table has a width corresponding to a circumference of the circular stitching boundary;

determining an extended disparity profile based on a periodic extension of the cost table and a smoothness criterion, wherein the extended disparity profile has a length greater than the width of the cost table, wherein the extended disparity profile is constrained to start at a disparity value of zero and end at a disparity value of zero;

determining a binocular disparity profile of a length equal to the width of the cost table based on a contiguous subsequence of the extended disparity profile; and

stitching the first image and the second image using the binocular disparity profile to obtain a combined image.

Assignments (3)
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: FARALLON CAPITAL MANAGEMENT, L.L.C., AS AGENT
Reel/Frame 072340/0676 →
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 072358/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 19, 2021
From: DOUADY, BRUNO CÉSAR; LEFEBVRE, ALEXIS
To: GOPRO, INC.
Reel/Frame 055954/0876 →
Continuity (2)
Provisional Application 62901918 · Sep 18, 2019
Related Publication 20210084281A1 · Mar 18, 2021