IP Library Granted Patent US 11,610,338
Granted Patent B2
US 11,610,338 · App. 17/214,424 · Granted Mar 21, 2023

Method of controlling a camera

Inventor: Piotr Stec (Galway, IE)
G06T7/80H04N5/232127G06T2207/10016G06T2207/10028
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,610,338
App. No.
17/214,424
Granted
Mar 21, 2023
Kind
B2
Abstract

A method comprises displaying a first image acquired from a camera having an input camera projection model including a first focal length and an optical axis parameter value. A portion of the first image is selected as a second image associated with an output camera projection model in which either a focal length and/or an optical axis parameter value differ from the parameters of the input camera projection model. The method involves iteratively: adjusting either the focal length and/or an optical axis parameter value for the camera lens so that it approaches the corresponding value of the output camera projection model; acquiring a subsequent image using the adjusted focal length or optical axis parameter value; mapping pixel coordinates in the second image, through a normalized 3D coordinate system to respective locations in the subsequent image to determine respective values for the pixel coordinates; and displaying the second image.

Claims (71)

1. A method of controlling a camera comprising:

acquiring, with a camera comprising a lens at a first focal length, a first image with a first field of view, the camera further comprising an input camera projection model including (i) the first focal length and (ii) a first optical axis parameter value;

displaying the first image;

receiving a selection of at least a portion of the first field of view to be displayed as a second image associated with an output camera projection model comprising a point of view in common with the input camera projection model and in which at least one of (i) a second focal length and (ii) a second optical axis parameter value differ from parameters of the input camera projection model; and

iteratively:

a) adjusting at least one of (i) the first focal length and (ii) the first optical axis parameter value for the lens so that the lens approaches a corresponding at least one of (i) the second focal length and (ii) the second optical axis parameter value of the output camera projection model;

b) acquiring a subsequent image with the camera using at least one of (i) an adjusted first focal length or (ii) an adjusted optical axis parameter value;

c) mapping pixel coordinates in the second image, using the output camera projection model, through a normalized 3-dimensional coordinate system to respective locations in the subsequent image, using the at least one of (i) the adjusted first focal length or (ii) the adjusted optical axis parameter value in the input camera projection model, to determine respective values for the pixel coordinates; and

d) displaying the second image using the respective values for pixel coordinates.

2. The method of claim 1 , wherein the mapping further comprises:

acquiring an indication of movement of the camera in a time between acquiring the subsequent image and acquiring a previous image; and

rotating mapped pixel coordinates in the normalized 3-dimensional coordinate system to counter camera movement.

3. The method of claim 1 , wherein the input camera projection model comprises one of (i) a pinhole projection model, (ii) a rectilinear projection model, or (iii) a fish eye projection model.

4. The method of claim 3 , wherein the output camera projection model comprises one of (i) a pinhole projection model, (ii) a rectilinear projection model, or (iii) a fish eye projection model.

5. The method of claim 1 , wherein:

the output camera projection model comprises a rectilinear projection model; and

the input camera projection model comprises a different projection model.

6. The method of claim 1 , wherein:

the selecting comprises selecting a sub-region of the first image that is displayed; and

the adjusting comprises adjusting the first focal length of the lens.

7. The method of claim 1 , wherein:

the selecting comprises selecting an off-center region of the first image that is displayed; and

the adjusting comprises adjusting a pan or tilt angle of the lens so that the adjusted optical axis parameter value approaches the second optical axis parameter value.

8. The method of claim 1 , wherein:

the selecting comprises selecting an off-center sub-region of the first image that is displayed; and

the adjusting comprises adjusting the first focal length of the lens.

9. The method of claim 1 , wherein:

the selecting comprises selecting an off-center sub-region of the first image that is displayed; and

the adjusting comprises adjusting the first focal length of the lens and adjusting a pan or tilt angle of the lens so that the adjusted optical axis parameter value approaches the second optical axis parameter value.

10. The method of claim 1 , wherein steps a) to d) are repeated until the first adjusted focal length and the adjusted optical axis parameter value for the lens match the corresponding the second focal length and the second optical axis parameter value.

11. The method of claim 1 , wherein steps a) to d) are repeated only to a point that all of the portion of the first field of view that is selected can be displayed.

12. A handheld camera comprising:

a lens at a first focal length;

a lens actuator configured to change the first focal length of the lens;

an input camera projection model including (i) the first focal length and (ii) a first optical axis parameter value; and

a processor configured to (i) run a camera application and (ii) control the handheld camera to perform:

acquiring a first image with a first field of view;

displaying the first image;

receiving a selection of at least a portion of the first field of view to be displayed as a second image associated with an output camera projection model comprising a point of view in common with the input camera projection model and in which at least one of (i) a second focal length and (ii) a second optical axis parameter value differ from parameters of the input camera projection model; and

iteratively:

a) adjusting at least one of (i) the first focal length and (ii) the first optical axis parameter value for the lens so that the lens approaches a corresponding at least one of (i) the second focal length and (ii) the second optical axis parameter value of the output camera projection model;

b) acquiring a subsequent image using at least one of (i) an adjusted first focal length or (ii) an adjusted optical axis parameter value;

c) mapping pixel coordinates in the second image, using the output camera projection model, through a normalized 3-dimensional coordinate system to respective locations in the subsequent image, using the at least one of (i) the adjusted first focal length or (ii) the adjusted optical axis parameter value in the input camera projection model, to determine respective values for the pixel coordinates; and

d) displaying the second image using the respective values for pixel coordinates.

13. The handheld camera of claim 12 , wherein steps b) and c) are performed in camera firmware prior to writing the respective values for the pixel coordinates for the second image to memory.

14. The handheld camera of claim 12 , wherein steps b) and c) are performed in either in camera firmware or by the camera application after writing the respective values for the pixel coordinates for the subsequent image to memory.

15. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform:

acquiring, with a camera comprising a lens at a first focal length, a first image with a first field of view, the camera further comprising an input camera projection model including (i) the first focal length and (ii) a first optical axis parameter value;

displaying the first image;

receiving a selection of at least a portion of the first field of view to be displayed as a second image associated with an output camera projection model comprising a point of view in common with the input camera projection model and in which at least one of (i) a second focal length and (ii) a second optical axis parameter value differ from parameters of the input camera projection model; and

iteratively:

a) adjusting at least one of (i) the first focal length and (ii) the first optical axis parameter value for the lens so that the lens approaches a corresponding at least one of (i) the second focal length and (ii) the second optical axis parameter value of the output camera projection model;

b) acquiring a subsequent image with the camera using at least one of (i) an adjusted first focal length or (ii) an adjusted optical axis parameter value;

c) mapping pixel coordinates in the second image, using the output camera projection model, through a normalized 3-dimensional coordinate system to respective locations in the subsequent image, using the at least one of (i) the adjusted first focal length or (ii) the adjusted optical axis parameter value in the input camera projection model, to determine respective values for the pixel coordinates; and

d) displaying the second image using the respective values for pixel coordinates.

16. A system comprising:

a fixed mount camera comprising a lens at a first focal length, an input camera projection model including (i) the first focal length and (ii) a first optical axis parameter value; and

one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors operably connected to the fixed mount camera, cause the one or more processors to perform:

acquiring, with the fixed mount camera, a first image with a first field of view;

displaying the first image;

receiving a selection of at least a portion of the first field of view to be displayed as a second image associated with an output camera projection model comprising a point of view in common with the input camera projection model and in which at least one of (i) a second focal length and (ii) a second optical axis parameter value differ from parameters of the input camera projection model; and

iteratively:

a) adjusting at least one of (i) the first focal length and (ii) the first optical axis parameter value for the lens so that the lens approaches a corresponding at least one of (i) the second focal length and (ii) the second optical axis parameter value of the output camera projection model;

b) acquiring a subsequent image with the fixed mount camera using at least one of (i) an adjusted first focal length or (ii) an adjusted optical axis parameter value;

c) mapping pixel coordinates in the second image, using the output camera projection model, through a normalized 3-dimensional coordinate system to respective locations in the subsequent image, using the at least one of (i) the adjusted first focal length or (ii) the adjusted optical axis parameter value in the input camera projection model, to determine respective values for the pixel coordinates; and

d) displaying the second image using the respective values for pixel coordinates.

17. The system of claim 16 , wherein the fixed mount camera comprises a lens actuator configured to change a focal length and a pan and tilt angle of the lens.

18. The system of claim 16 , wherein steps b) and c) are performed in camera firmware prior to writing the respective values for the pixel coordinates for the second image to memory.

19. The system of claim 16 , wherein:

the fixed mount camera comprises a processor configured to run a non-interactive camera application; and

the computer-executable instructions are configured to (i) display the first image, (ii) enable a user of a computing device to select the portion of the first field of view to be displayed as the second image, and (iii) cause the computing device to iteratively display the second image.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 12, 2025
From: TOBII TECHNOLOGIES LTD
To: ADEIA MEDIA HOLDINGS LLC
Reel/Frame 071572/0855 →
CONVERSION Recorded Jun 12, 2025
From: ADEIA MEDIA HOLDINGS LLC
To: ADEIA MEDIA HOLDINGS INC.
Reel/Frame 071577/0875 →
SECURITY INTEREST Recorded May 28, 2025
From: ADEIA INC. (F/K/A XPERI HOLDING CORPORATION); ADEIA HOLDINGS INC.; ADEIA MEDIA HOLDINGS INC.; ADEIA IMAGING LLC; ADEIA MEDIA LLC; ADEIA MEDIA SOLUTIONS INC.; ADEIA SEMICONDUCTOR BONDING TECHNOLOGIES INC.; ADEIA TECHNOLOGIES INC.; ADEIA GUIDES INC.; ADEIA SOLUTIONS LLC; ADEIA SEMICONDUCTOR ADVANCED TECHNOLOGIES INC.; ADEIA SEMICONDUCTOR SOLUTIONS LLC; ADEIA SEMICONDUCTOR INTELLECTUAL PROPERTY LLC; ADEIA SEMICONDUCTOR TECHNOLOGIES LLC; ADEIA PUBLISHING INC.
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 071454/0343 →
CHANGE OF NAME Recorded Dec 5, 2024
From: FOTONATION LIMITED
To: TOBII TECHNOLOGIES LIMITED
Reel/Frame 069516/0394 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 26, 2021
From: STEC, PIOTR
To: FOTONATION LIMITED
Reel/Frame 055738/0785 →
Continuity (1)
Related Publication 20220309709A1 · Sep 29, 2022