Granted Patent B2

US 12,614,246 · App. 17/795,523 · Granted Apr 28, 2026

Inventors: Sylvain Leroy (Bures-sur-Yvette, FR); Guillaume Matthieu Guerin (Renens, CH); Yoël Taïeb (Paris, FR)

Assignee: GoPro, Inc.

G06T3/4038G06T7/292G06V40/161H04N23/61H04R5/04

Abstract

Claims (63)

1 . A system comprising:

an array of two or more image sensors configured to capture images; and

a processing apparatus that is configured to:

select an image sensor as a prioritized sensor from among the array of two or more image sensors, wherein the array of two or more image sensors have different respective fields of view that jointly provide a larger combined field of view;

access one or more images captured using the prioritized sensor;

determine one or more image processing parameters based on the one or more images captured using the prioritized sensor, wherein the one or more image processing parameters include at least one of an auto exposure algorithm parameter, an auto white balance algorithm parameter, or a global tone mapping algorithm parameter;

access a second image captured using a second image sensor of the array of two or more image sensors, wherein the second image sensor is other than the prioritized sensor; and

apply image processing using the one or more image processing parameters to the second image to obtain a processed image.

2 . The system of claim 1 , wherein the one or more image processing parameters are determined without consideration of image data from image sensors other than the prioritized sensor.

3 . The system of claim 1 , wherein the processed image is a first processed image, and the processing apparatus is configured to:

apply image processing using the one or more image processing parameters to an image captured using the prioritized sensor to obtain a second processed image; and

stitch the first processed image to the second processed image to obtain a composite image.

4 . The system of claim 3 , wherein the composite image is a spherical image.

5 . The system of claim 1 , wherein the prioritized sensor and the second image sensor are positioned back-to-back, and further comprising:

a first hyper-hemispherical lens positioned to cover the prioritized sensor; and

a second hyper-hemispherical lens positioned to cover the second image sensor.

6 . The system of claim 1 , wherein the processing apparatus is configured to:

receive a user input signal identifying the prioritized sensor; and

select the prioritized sensor from among the array of two or more image sensors based on the user input signal.

7 . The system of claim 1 , wherein the processing apparatus is configured to:

detect a face in an image captured using the prioritized sensor; and

select the prioritized sensor from among the array of two or more image sensors based on detection of the face.

8 . The system of claim 1 , wherein the processing apparatus is configured to:

track an object appearing in images captured using the array of two or more image sensors; and

select the prioritized sensor from among the array of two or more image sensors based on appearance of the object in an image captured using the prioritized sensor.

9 . The system of claim 1 , comprising an array of two or more microphones that is attached to the array of two or more image sensors, and wherein the processing apparatus that is configured to:

determine a direction of arrival of an audio signal based on audio recordings captured using the array of two or more microphones; and

select the prioritized sensor from among the array of two or more image sensors based on correspondence between the direction of arrival of the audio signal and a field of view the prioritized sensor.

10 . The system of claim 9 , wherein the audio signal is a human speech signal.

11 . The system of claim 1 , wherein the one or more image processing parameters include parameters of an auto exposure algorithm, and the image processing applied includes auto exposure processing.

12 . The system of claim 1 , wherein the one or more image processing parameters include parameters of a global tone mapping algorithm, and the image processing applied includes global tone mapping processing.

13 . A method comprising:

selecting an image sensor as a prioritized sensor from among an array of two or more image sensors, wherein the array of two or more image sensors have different respective fields of view that jointly provide a larger combined field of view;

determining one or more image processing parameters based on one or more images captured using the prioritized sensor, wherein the one or more image processing parameters include at least one of an auto exposure algorithm parameter, an auto white balance algorithm parameter, or a global tone mapping algorithm parameter;

applying image processing using the one or more image processing parameters to images captured with each image sensor in the array of two or more image sensors to obtain respective processed images for the array of two or more image sensors; and

stitching the respective processed images for the array of two or more image sensors to obtain a composite image.

14 . The method of claim 13 , wherein the one or more image processing parameters are determined without consideration of image data from image sensors other than the prioritized sensor.

15 . The method of claim 13 , wherein selecting the prioritized sensor from among the array of two or more image sensors comprises:

detecting a face in an image captured using the prioritized sensor; and

selecting the prioritized sensor from among the array of two or more image sensors based on detection of the face.

16 . The method of claim 13 , wherein selecting the prioritized sensor from among the array of two or more image sensors comprises:

tracking an object appearing in images captured using the array of two or more image sensors; and

selecting the prioritized sensor from among the array of two or more image sensors based on appearance of the object in an image captured using the prioritized sensor.

17 . The method of claim 13 , wherein selecting the prioritized sensor from among the array of two or more image sensors comprises:

determining a direction of arrival of an audio signal based on audio recordings captured using an array of two or more microphones that is attached to the array of two or more image sensors; and

selecting the prioritized sensor from among the array of two or more image sensors based on correspondence between the direction of arrival of the audio signal and a field of view the prioritized sensor.

18 . An image capture device, comprising:

a first image sensor configured to capture images;

a second image sensor configured to capture images;

a memory that stores instructions for producing spherical images based on images captured using the first image sensor and the second image sensor; and

a processor that executes the instructions to:

select, from among the first image sensor and the second image sensor, an image sensor as a prioritized sensor and an other image sensor as a deprioritized sensor;

determine an average global luminance value for a spherical image to be produced using a first image captured using the first image sensor and a second image captured using the second image sensor, wherein the average global luminance value is determined as a weighted average using weights determined based on the selection of the prioritized sensor to weight pixel values captured using the prioritized sensor more heavily than pixel values captured using the deprioritized sensor;

determine luminance values for each of the first image and the second image;

determine delta luminance values for each of the first image and the second image based on the average global luminance value and the luminance values;

update the first image and the second image using the delta luminance values; and

produce the spherical image based on the updated first image and the updated second image.

19 . The image capture device of claim 18 , wherein the instructions to determine the delta luminance values for each of the first image and the second image based on the average global luminance value and the luminance values include instructions to:

determine clipped luminance variance values for the delta luminance values based on a threshold value representative of a maximum luminance variance between the first image sensor and the second image sensor; and

produce smoothed delta luminance values by applying temporal smoothing against the clipped luminance variance values.

20 . The image capture device of claim 19 , wherein the instructions to update the first image and the second image using the delta luminance values include instructions to:

determine constrained luminance values for the first image and the second image based on the average global luminance value and the smoothed delta luminance values; and

limit the update to the first image and the second image based on the constrained luminance values.

Assignments (3)

Continuity (2)

References Cited (56)