Method and system for calibrating a camera and localizing objects within the camera field of view
A computer-implemented method of localizing an image of a person captured using a camera is provided, the person in the field of view of a camera, comprising: obtaining the image captured using a camera, the image comprising the person within a bounding box; determining at least one slant value associated with the person within the bounding box; determining head image coordinates and feet image coordinates for the person using the at least one slant value; and localizing the person by projecting the head image coordinates to a head plane and the feet image coordinates for the person to a ground plane.
1. A computer-implemented method of localizing an image of a person captured using a camera, the person in a field of view of a camera, comprising:
obtaining the image captured using a camera, the image comprising the person within a bounding box;
determining at least one slant value associated with the person within the bounding box;
determining head image coordinates and feet image coordinates for the person using the at least one slant value;
localizing the person by projecting the head image coordinates to a head plane and the feet image coordinates for the person to a ground plane; and
wherein the at least one slant value is selected from δ x or δ y , center coordinates of the bounding box are represented as (x c , y c ), the head image coordinates are represented as (x c +δ x , y c +δ y ) and the feet image coordinates are represented as (x c +δ x , y c +δ y ).
2. The method of claim 1 , further comprising determining a distance between the person and an object in the field of view of the camera.
3. The method of claim 1 wherein the head coordinates and the feet coordinates model the person using a feet-to-head homology equation:
H
[
x
feet
y
feet
1
]
=
w
[
x
head
y
head
1
]
wherei
n:
H
=
(
I
-
H
human
H
cam
p
2
(
p
0
×
p
1
)
T
p
2
T
(
p
0
×
p
1
)
)
which depends on camera parameters associated with the camera and wherein H human is a height of the person, wherein the camera parameters further comprise a height of the camera (H cam ), a camera focal length (f), a roll (θ), and a pitch (ρ), w is a scalar, T is a transpose operation, and p 0 , p 1 , p 2 are the first three columns of a homography projection matrix P:
P
=
[
f
cos
ρ
-
f
sin
ρ
cos
θ
f
sin
ρ
sin
θ
-
f
H
c
a
m
sin
ρ
sin
θ
f
sin
ρ
f
cos
ρ
cos
θ
-
f
cos
ρ
sin
θ
f
H
c
a
m
cos
ρ
sin
θ
0
sin
θ
cos
θ
-
H
c
a
m
cos
θ
]
.
4. The method of claim 3 wherein a slant model homology formula is used to model the person, the slant model homology formula comprising:
H
[
x
c
-
δ
x
y
c
-
δ
y
1
]
=
w
[
x
c
+
δ
x
y
c
+
δ
y
1
]
.
5. The method of claim 4 wherein the projecting the head image coordinates and the feet image coordinates comprises using homography projection equations:
[
x
c
-
δ
x
y
c
-
δ
y
1
]
w
=
P
[
X
Y
0
1
]
and
[
x
c
+
δ
x
y
c
+
δ
y
1
]
w
=
P
[
X
Y
H
human
1
]
wherein (X, Y, Z=0) are world coordinates of the feet coordinates and (X, Y, Z=H human ) are world coordinates of the head coordinates.
6. The method of claim 1 wherein the δ x and δ y values are predetermined for a plurality of pixels in the field of view, and wherein the δ x and δ y values are determined by selected the δ x and δ y values associated with a pixel in the plurality of pixels that is closest (x c , y c ).
7. A security system comprising:
a camera having a field of view including a person; and
a processor communicatively coupled to the camera; and
a memory device having stored thereon computer program code that is executable by the processor, and when executed by the processor, causes the processor to:
obtain an image of the person captured using the camera, the image comprising the person within a bounding box;
determine at least one slant value associated with the person within the bounding box;
determine head image coordinates and feet image coordinates for the person using the at least one slant value;
localize the person by projecting the head image coordinates to a head plane and the feet image coordinates for the person to a ground plane; and
wherein the at least one slant value is selected from δ x or δ y , the center coordinates of the bounding box are represented as (x c , y c ), the head image coordinates are represented as (x c +δ x , y c +δ y ) and the feet image coordinates are represented as (x c −δ x , y c −δ y ).
8. The system of claim 7 , wherein the processor is further caused to determine a distance between the person and an object.
9. The system of claim 7 wherein the head coordinates and the feet coordinates model the person using a feet-to-head homology equation:
H
[
x
feet
y
feet
1
]
=
w
[
x
head
y
head
1
]
wherei
n:
H
=
(
I
-
H
human
H
cam
p
2
(
p
0
×
p
1
)
T
p
2
T
(
p
0
×
p
1
)
)
which depends on camera parameters associated with the camera and wherein H human is a height of the person, wherein the camera parameters comprise a height of the camera (H cam ), a camera focal length (f), a roll (θ), and a pitch (ρ), T is a transpose operation, and p 0 , p 1 , p 2 are the first three columns of a homography projection matrix P:
P
=
[
f
cos
ρ
-
f
sin
ρ
cos
θ
f
sin
ρ
sin
θ
-
f
H
c
a
m
sin
ρ
sin
θ
f
sin
ρ
f
cos
ρ
cos
θ
-
f
cos
ρ
sin
θ
f
H
c
a
m
cos
ρ
sin
θ
0
sin
θ
cos
θ
-
H
c
a
m
cos
θ
]
.
10. The system of claim 9 wherein the processor is further caused to use a slant model homology formula to model the person, the slant model homology formula comprising:
H
[
x
c
-
δ
x
y
c
-
δ
y
1
]
=
w
[
x
c
+
δ
x
y
c
+
δ
y
1
]
.
11. The system of claim 10 wherein the processor is further caused to project the head image coordinates and the feet image coordinates using homography projection equations:
[
x
c
-
δ
x
y
c
-
δ
y
1
]
w
=
P
[
X
Y
0
1
]
and
[
x
c
+
δ
x
y
c
+
δ
y
1
]
w
=
P
[
X
Y
H
human
1
]
wherein (X, Y, Z=0) are the world coordinates of the feet coordinates and (X, Y, Z=H human ) are the world coordinates of the head coordinates.
12. The system of claim 7 wherein the δ x and δ y values are predetermined for a plurality of pixels in the field of view of the camera, and wherein the δ x and δ y values are determined by selected the δ x and δ y values associated with a pixel in the plurality of pixels that is closest (x c , y c ).