IP Library › Granted Patent US 11,769,222
Granted Patent B2
US 11,769,222 · App. 17/713,285 · Granted Sep 26, 2023

Image processor and a method therein for providing a target image

Inventor: Håkan Ardö (Lund, SE)
Assignee: AXIS AB
G06T3/0062G06V10/255G06V10/758G06V20/52G06V2201/07
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,769,222
App. No.
17/713,285
Granted
Sep 26, 2023
Kind
B2
Abstract

An image processor and a method therein to provide a target image for evaluation with an object detector. The method comprises: obtaining a source image captured by a camera and depicting an object, and applying an inverse pixel transform to each target pixel of a target image to determine one or more source pixels located at a position in the source image corresponding to a position of each target pixel in the target image. The method further comprises assigning, to each target pixel, a target pixel value determined based on one or more source pixel values of the determined one or more source pixels located at the corresponding position, thereby is a size c in the target image of the depicted object of a specific object type normalized in at least one size dimension. Thereafter, the target image is fed to an object detector for evaluation.

Claims (56)

1. A method to provide a target image for evaluation with an object detector configured to detect objects of a specific object type, the method comprises:

obtaining a source image captured by a fisheye camera and depicting one or more objects in a scene, wherein the camera is mounted to capture images of the scene from above;

combining an inverse dewarping transform T −1 dewarp with an inverse pixel transform T −1 (d t ) into a full inverse transform T −1 dewarp (T −1 (d t )), wherein the inverse dewarping transform T −1 dewarp depends on the type of fisheye camera and wherein the inverse pixel transform T −1 (d t ) as a function of a distance d t from a principal point Pt of the target image to the target pixel to which the inverse pixel transform T −1 (d t ) is applied, is given by:

T −1 (d t )=exp[(d t -b)/a] if d t >d cutoff and

T −1 (d t )=d t if d t ≤d cutoff ; and wherein:

a=c/[log(z 1 /z 2 )]:

b=d cutoff −a log(d cutoff );

c is a parameter specifying a size in the target image of an object of the specific object type and is set to c=[(d cutoff )(z 0 /z 2 )]−[(d cutoff )(z 0 /z 1 )] to make the object at the distance d cutoff in the source image and in the target image of equal size;

z 0 is an assumed height above ground of a center of an object of the specific type and is given by (z 1 +z 2 )/2;

z 1 is the camera mounting height above ground;

z 2 is the camera mounting height above ground minus the assumed object height; and

d cutoff is a distance threshold value at which an object of the specific object type in the source image will have the same size as the object of the specific object type in the target image;

applying the full inverse transform T −1 dewarp (T −1 (d t )) to each target pixel of a target image to determine one or more source pixels located at a position in the source image corresponding to a position of each target pixel in the target image,

assigning, to each target pixel, a target pixel value determined based on one or more source pixel values of the determined one or more source pixels located at the corresponding position, thereby is a size c in the target image of depicted one or more objects of a specific object type normalized in at least one size dimension of the depicted one or more objects of the specific object type; and

providing the target image to the object detector for evaluation.

2. The method of claim 1 , wherein the target pixels reproduce the depicted one or more objects of the specific object type in the target image with a constant size in the at least one size dimension irrespective of the location of each of the depicted one or more objects of the specific object type in the target image.

3. The method of claim 2 , wherein the inverse pixel transform T −1 is rotationally symmetric around a principal point Pt of the target image.

4. An image processor configured to provide a target image for evaluation with an object detector configured to detect objects of a specific object type; wherein the image processor is configured to:

obtain a source image captured by an image capturing module comprising a fisheye lens and depicting one or more objects in a scene, wherein the image capturing module is mounted to capture images of the scene from above,

combine an inverse dewarping transform T −1 dewarp with an inverse pixel transform T −1 (d t ) into a full inverse transform T −1 dewarp (T −1 (d t )), wherein the inverse dewarping transform T −1 dewarp depends on the type of fisheye lens, and wherein the inverse pixel transform T −1 as a function of a distance d t from a principal point Pt of the target image to the target pixel to which the inverse pixel transform T −1 is applied, is given by

T −1 (d t )=exp[(d t -b)/a] if d t >d cutoff and

T −1 (d t )=d t if d t ≤d cutoff ; and wherein:

a=c/[log(z 1 /z 2 )]:

b=d cutoff −a log(d cutoff );

c is a parameter specifying a size in the target image of an object of the specific object type and is set to c=[(d cutoff ) (z 0 /z 2 )]−[(d cutoff )(z 0 /z 1 )] to make the object at the distance d cutoff in the source image and in the target image of equal size;

z 0 is an assumed height above ground of a center of an object of the specific type and is given by (z 1 +z 2 )/2;

z 1 is the camera mounting height above ground;

z 2 is the camera mounting height above ground minus the assumed object height; and

d cutoff is a distance threshold value at which an object of the specific object type in the source image will have the same size as the object of the specific object type in the target image;

apply the full inverse transform T −1 dewarp (T −1 (d t )) to each target pixel of a target image to determine one or more source pixels located at a position in the source image corresponding to a position of each target pixel in the target image;

assign, to each target pixel, a target pixel value determined based on one or more source pixel values of the determined one or more source pixels located at the corresponding position, thereby is the size c in the target image of depicted one or more objects of the specific object type normalized in at least one size dimension of the depicted one or more objects of the specific object type; and

feed provide the target image to the object detector for evaluation.

5. A fisheye camera comprising the image processor of claim 4 , and configured to:

capture the source image depicting the one or more objects in the scene;

provide the source image to the image processor; and wherein the camera further comprises the object detector configured to receive the target image and to evaluate the target image by performing object detection on the target image.

6. The camera of claim 5 , wherein the camera is a monitoring camera.

7. A non-transitory computer-readable medium having stored thereon computer code instructions adapted to carry out a method when executed by a device having processing capability, the method providing a target image for evaluation with an object detector, the method comprising:

obtaining a source image captured by a fisheye camera and depicting one or more objects in a scene, wherein the camera is mounted to capture images of the scene from above;

combining an inverse dewarping transform T −1 dewarp with an inverse pixel transform T −1 (d t ) into a full inverse transform T −1 dewarp (T −1 (d t )), wherein the inverse dewarping transform T −1 dewarp depends on the type of fisheye camera and wherein the inverse pixel transform T −1 (d t ) as a function of a distance d t from a principal point Pt of the target image to the target pixel to which the inverse pixel transform T −1 (d t ) is applied, is given by

T −1 (d t )=exp[(d t −b)/a] if d t >d cutoff and

T −1 (d t )=d t if d t ≤d cutoff ; and wherein:

a=c/[log(z 1 /z 2 )];

b=d cutoff −a log(d cutoff );

c is a parameter specifying a size in the target image of an object of the specific object type and is set to c=[(d cutoff )(z 0 /z 2 )]−[d cutoff ) (z 0 /z 1 )] to make the object at the distance d cutoff in the source image and in the target image of equal size;

z 0 is an assumed height above ground of a center of an object of the specific type and is given by (z 1 +z 2 )/2;

z 1 is the camera mounting height above ground;

z 2 is the camera mounting height above ground minus the assumed object height; and

d cutoff is a distance threshold value at which an object of the specific object type in the source image will have the same size as the object of the specific object type in the target image;

applying the full inverse transform T −1 dewarp (T −1 (d t ))

to each target pixel of a target image to determine one or more source pixels located at a position in the source image corresponding to a position of each target pixel in the target image,

assigning, to each target pixel, a target pixel value determined based on one or more source pixel values of the determined one or more source pixels located at the corresponding position, thereby is a size c in the target image of depicted one or more objects of a specific object type normalized in at least one size dimension of the depicted one or more objects of the specific object type; and

providing the target image to an object detector for evaluation.

8. The non-transitory computer-readable medium of claim 7 , wherein the method further comprises:

wherein the target pixels reproduce the depicted one or more objects of the specific object type in the target image with a constant size in the at least one size dimension irrespective of the location of each of the depicted one or more objects of the specific object type in the target image.

9. The non-transitory computer-readable medium of claim 7 , wherein:

the inverse pixel transform T −1 is rotationally symmetric around a principal point Pt of the target image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 5, 2022
From: ARDÖ, HÅKAN
To: AXIS AB
Reel/Frame 059502/0356 →
Priority Claims (1)
EP 21175491 · May 24, 2021 · regional
Continuity (1)
Related Publication 20220375025A1 · Nov 24, 2022