IP Library Granted Patent US 11,158,025
Granted Patent B2
US 11,158,025 · App. 16/237,089 · Granted Oct 26, 2021

Methods and apparatus for optimized stitching of overcapture content

Inventors: Daryl Stimm (Encinitas, CA); David Newman (San Diego, CA); Kyler William Schwartz (Valley Center, CA); William Edward MacDonald (San Diego, CA)
Assignee: GoPro, Inc.
G06T3/4038G06T3/0062G06T3/0093G06T7/13H04N5/23238H04N7/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,158,025
App. No.
16/237,089
Granted
Oct 26, 2021
Kind
B2
Abstract

Apparatus and methods for optimized stitching of overcapture content. In one embodiment, the optimized stitching of the overcapture content includes capturing the overcapture content; producing overlap bands associated with the captured overcapture content; downsampling the produced overlap bands; generating derivative images from the downsampled overlap bands; generating a cost map associated with the generated derivative images; determining shortest path information for the generated cost map; generating a warp file based on the determined shortest path information, the generated warp file being utilized for the optimized stitching of the overcapture content. Camera apparatus and a non-transitory computer-readable apparatus are also disclosed.

Claims (48)

1. A method for optimized stitching of overcapture content, the method comprising:

capturing the overcapture content;

producing overlap bands associated with the captured overcapture content;

downsampling the produced overlap bands to generate downsampled overlap bands with reduced dimensions;

performing feature detection based on the downsampled overlap bands with reduced dimensions within an integrated memory of a camera device;

generating a cost map based on the performed feature detection;

determining shortest path information for the generated cost map; and

generating a warp file based on the determined shortest path information, the generated warp file being utilized for the optimized stitching of the overcapture content.

2. The method of claim 1 , further comprising generating a plurality of fisheye images from the captured overcapture content.

3. The method of claim 2 , further comprising projecting the produced overlap bands into an equirectangular projection.

4. The method of claim 2 , further comprising transmitting the generated warp file and the generated plurality of fisheye images to another device.

5. The method of claim 2 , further comprising stitching the plurality of fisheye images from the captured overcapture content and transmitting the stitched fisheye images and the generated warp file to another device.

6. The method of claim 5 , wherein the transmitting of the generated warp file to the another device enables re-stitching of the stitched fisheye images.

7. The method of claim 1 , wherein the generating of the warp file comprises generating a plurality of integer values, the generated plurality of integer values being associated with displacement values associated with a stitch line for the captured overcapture content.

8. A camera apparatus comprising:

two or more image capture devices;

a network interface in data communication with the two or more image capture devices; and

an image processing pipeline that is configured to:

receive captured content from the two or more image capture devices;

perform edge detection on the received captured content; and

generate a warp track file comprising a string of unsigned character values representing a path value corresponding to a location of a stitch in a cost map of the captured content,

wherein the warp track file enables an another device to stitch the captured content by performing a look-up in the warp track file.

9. The camera apparatus of claim 8 , wherein the generation of the warp track file by the image processing pipeline comprises:

production of overlap bands associated with the received captured content;

downsample of the produced overlap bands to generate downsampled overlap bands with reduced dimensions;

generation of derivative images from the downsampled overlap bands with reduced dimensions;

generation of a cost map associated with the generated derivative images; and

determination of shortest path information for the generated cost map.

10. The camera apparatus of claim 9 , wherein the generated warp track file and the captured content from the two or more image capture devices are transmitted via the network interface to another device.

11. The camera apparatus of claim 10 , wherein the transmission of the generated warp track file and the transmission of the captured content to the another device enables re-stitching of the captured content.

12. The camera apparatus of claim 10 , wherein the generation of the warp track file comprises generation of a plurality of integer values, the plurality of integer values being associated with displacement values associated with a stitch line for the captured content.

13. The camera apparatus of claim 8 , wherein the receipt of the captured content comprises receipt of fisheye projections.

14. The camera apparatus of claim 13 , wherein the image processing pipeline is further configured to stitch the fisheye projections and the camera apparatus is configured to transmit the stitched fisheye projections and the generated warp file to another device via the network interface.

15. The camera apparatus of claim 14 , wherein the transmission of the generated warp track file and the transmission of the stitched fisheye projections to the another device enables a re-stitch of the stitched fisheye projections.

16. A non-transitory computer-readable apparatus comprising a storage medium having a computer program stored thereon, the computer program comprising a plurality of instructions configured to, when executed by a processor apparatus, cause an apparatus to:

receive captured content from two or more image capture devices;

perform edge detection on the received captured content; and

generate a warp file comprising a string of unsigned character values representing a path value corresponding to a location of a stitch in a cost map of the captured content,

wherein the warp file enables an another device to stitch the captured content by performing a look-up in the warp file.

17. The non-transitory computer-readable apparatus of claim 16 , wherein the receipt of the captured content comprises receipt of fisheye projections.

18. The non-transitory computer-readable apparatus of claim 17 , wherein the plurality of instructions, when executed by the processor apparatus, are further configured to stitch the fisheye projections and transmit the stitched fisheye projections and the generated warp file to another device via a network interface.

19. The non-transitory computer-readable apparatus of claim 18 , wherein the transmission of the generated warp file and the transmission of the stitched fisheye projections to the another device enables a re-stitch of the stitched fisheye projections.

20. The non-transitory computer-readable apparatus of claim 16 , wherein the generation of the warp file comprises:

production of overlap bands associated with the received captured content;

downsample of the produced overlap bands to generate downsampled overlap bands with reduced dimensions;

generation of derivative images from the downsampled overlap bands with reduced dimensions;

generation of a cost map associated with the generated derivative images; and

determination of shortest path information for the generated cost map.

Assignments (5)
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: FARALLON CAPITAL MANAGEMENT, L.L.C., AS AGENT
Reel/Frame 072340/0676 →
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 072358/0001 →
RELEASE OF PATENT SECURITY INTEREST Recorded Jan 25, 2021
From: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
To: GOPRO, INC.
Reel/Frame 055106/0434 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 27, 2019
From: STIMM, DARYL; NEWMAN, DAVID; SCHWARTZ, KYLER WILLIAM; MACDONALD, WILLIAM EDWARD
To: GOPRO, INC.
Reel/Frame 051378/0532 →
SECURITY INTEREST Recorded Mar 5, 2019
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 048508/0728 →
Continuity (1)
Related Publication 20210082085A1 · Mar 18, 2021