Human head detection in depth images
View Patent ↗Systems, devices and methods are described including receiving a depth image and applying a template to pixels of the depth image to determine a location of a human head in the depth image. The template includes a circular shaped region and a first annular shaped region surrounding the circular shaped region. The circular shaped region specifies a first range of depth values. The first annular shaped region specifies a second range of depth values that are larger than depth values of the first range of depth values.
1. A computer-implemented method for detecting a human head in an image, comprising:
receiving a depth image, the depth image comprising a foreground portion and the foreground portion comprising a first plurality of pixels;
selecting a second plurality of pixels of the foreground portion of the depth image, the second plurality of pixels being equal to or less than the first plurality of pixels;
for each selected pixel of the second plurality of pixels,
centering a center point M(x,y) of a template with respect to the selected pixel, the template comprising the center point and a plurality of template points M(i,j), the template further comprising a circular-shaped region IR and a first annular-shaped region MR surrounding the circular shaped region, a depth of a template point M(i,j) in the circular-shaped region IR and in the first annular region MR being defined as:
for
M
(
i
,
j
)
=
∈
IR
:
{
∑
d
≤
R
c
R
p
2
-
d
2
/
R
p
Num
(
IR
)
·
(
∑
d
≤
R
c
(
R
p
2
-
d
2
/
R
p
)
+
∑
R
c
≤
d
≤
R
p
(
R
p
2
-
d
2
/
R
p
)
)
and
for
M
(
i
,
j
)
∈
MR
:
{
∑
R
c
≤
d
≤
R
p
(
R
p
2
-
d
2
/
R
p
)
Num
(
MR
)
·
(
∑
d
≤
R
c
(
R
p
2
-
d
2
/
R
p
)
+
∑
R
c
≤
d
≤
R
p
(
R
p
2
-
d
2
/
R
p
)
)
wherein, R p comprises a projection of a radius of the head in a view plane of the depth image, d comprises a distance between a template point M(i,j) and the center point M(x,y) of the template, R c comprises a radius of the circular-shaped region IR, Num(IR) comprises a total number of template points within the circular-shaped region IR, Num(MR) comprises a total number of template points within the first annular-shaped region MR, i and j comprise rectangular coordinate indices associated with pixels of the depth image, and x and y comprise rectangular coordinate indices associated with the template, index i corresponding to index x, and index j corresponding to index y; and
determining a depth value for at least one pixel neighboring the selected pixel;
determining a score H(x,y) associated with the selected pixel based on a depth value for at least one neighboring pixel in comparison to a depth value associated with a template point corresponding to the neighboring pixel; and
determining a highest score of the determined scores; and
selecting the pixel associated with the highest score as a center of the human head.
2. The method according to claim 1 , the projection of the radius R p of the head in the image view plane comprises a radius of the first annular-shaped region, and
wherein R p comprises a value of 1.25 times R c .
3. The method according to claim 1 , wherein the template further comprises a second annular-shaped region OR surrounding the first annular-shaped region MR, a depth of a template point M(i,j) in the circular-shaped region OR being defined as:
for
M
(
i
,
j
)
∈
OR
:
-
1
Num
(
OR
)
wherein, Num(OR) comprises a total number of template points within second annular-shaped region OR.
4. The method according to claim 3 , wherein a radius of the second annular-shape region comprises a value of 1.25 times R p .
5. The method according to claim 3 , wherein determining the score H(x,y) comprises determining
H
(
x
,
y
)
=
(
∑
I
(
i
,
j
)
∈
IR
p
I
(
i
,
j
)
·
depth
(
M
(
x
,
y
)
)
+
∑
I
(
i
,
j
)
∈
MR
p
I
(
i
,
j
)
·
depth
(
M
(
x
,
y
)
)
+
∑
I
(
i
,
j
)
∈
OR
p
I
(
i
,
j
)
·
depth
(
M
(
X
,
j
)
)
)
/
SUM
wherein, I(i,j) comprises a depth value of each of the at least one neighboring pixel of the selected pixel, IR p comprises a projection of circular-shaped region IR into the view plane of the depth image, MR p comprises a projection of the first annular-shaped region MR into the view plane of the depth image, OR p comprises a projection of the second annual-shape region OR into the view plane of the depth image, and
SUM
=
∑
I
(
i
,
j
)
∈
IR
p
I
(
i
,
j
)
+
∑
I
(
i
,
j
)
∈
MR
p
I
(
i
,
j
)
+
∑
I
(
i
,
j
)
∈
OR
p
I
(
i
,
j
)
.
6. The method according to claim 1 , wherein the first annular-shaped MR region is arranged concentrically with respect to the circular-shaped region IR.
7. The method according to claim 1 , further comprising scaling the template so that the template points M(x,y) comprise a same pitch and a same orientation as pixels of the depth image.
8. The method according to claim 1 , wherein the depth image comprises a monochrome depth image.
9. The method according to claim 1 , wherein the depth image comprises a color depth image.
10. A system, comprising:
a processor and a memory coupled to the processor, the memory comprising instructions to configure the processor to:
receive a depth image, the depth image comprising a foreground portion and the foreground portion comprising a first plurality of pixels;
select a second plurality of pixels of the foreground portion of the depth image, the second plurality of pixels being equal to or less than the first plurality of pixels;
for each selected pixel of the second plurality of pixels,
center a center point M(x,y) of a template with respect to the selected pixel, the template comprising the center point and a plurality of template points M(i,j), the template further comprising a circular-shaped region IR and a first annular-shaped region MR surrounding the circular shaped region, a depth of a template point M(i,j) in the circular-shaped region IR and in the first annular region MR being defined as:
for
M
(
i
,
j
)
=
∈
IR
:
{
∑
d
≤
R
c
R
p
2
-
d
2
/
R
p
Num
(
IR
)
·
(
∑
d
≤
R
c
(
R
p
2
-
d
2
/
R
p
)
+
∑
R
c
≤
d
≤
R
p
(
R
p
2
-
d
2
/
R
p
)
)
and
for
M
(
i
,
j
)
∈
MR
:
{
∑
R
c
≤
d
≤
R
p
(
R
p
2
-
d
2
/
R
p
)
Num
(
MR
)
·
(
∑
d
≤
R
c
(
R
p
2
-
d
2
/
R
p
)
+
∑
R
c
≤
d
≤
R
p
(
R
p
2
-
d
2
/
R
p
)
)
wherein, R p comprises a projection of a radius of the head in a view plane of the depth image, d comprises a distance between a template point M(i,j) and the center point M(x,y) of the template, R c comprises a radius of the circular-shaped region IR, Num(IR) comprises a total number of template points within the circular-shaped region IR, Num(MR) comprises a total number of template points within the first annular-shaped region MR, i and j comprise rectangular coordinate indices associated with pixels of the depth image, and x and y comprise rectangular coordinate indices associated with the template, index i corresponding to index x, and index j corresponding to index y; and
determine a depth value for at east one pixel neighboring the selected pixel;
determine a score H(x,y) associated with the selected pixel based on a depth value for at least one neighboring pixel in comparison to a depth value associated with a template point corresponding to the neighboring pixel; and
determine a highest score of the determined scores; and
select the pixel associated with the highest score as a center of the human tread.
11. The system according to claim 10 , the projection of the radius R p of the head in the image view plane comprises a radius of the first annular-shaped region, and
wherein R p comprises a value of 1.25 times R c .
12. The system according to claim 11 , wherein the template further comprises a second annular-shaped region OR surrounding the first annular-shaped region MR, a depth of a template point M(i,j) in the circular-shaped region OR being defined as:
for
M
(
i
,
j
)
∈
OR
:
-
1
Num
(
OR
)
wherein, Num(OR) comprises a total number of template points within second annular-shaped region OR.
13. The system according to claim 12 , wherein a radius of the second annular-shape region comprises a value of 1.25 times R p .
14. The system according to claim 12 , wherein determining the score H(x,y) comprises determining
H
(
x
,
y
)
=
(
∑
I
(
i
,
j
)
∈
IR
p
I
(
i
,
j
)
·
depth
(
M
(
x
,
y
)
)
+
∑
I
(
i
,
j
)
∈
MR
p
I
(
i
,
j
)
·
depth
(
M
(
x
,
y
)
)
+
∑
I
(
i
,
j
)
∈
OR
p
I
(
i
,
j
)
·
depth
(
M
(
X
,
j
)
)
)
/
SUM
wherein, I(i,j) comprises a depth value of each of the at least one neighboring pixel of the selected pixel, IR p comprises a projection of circular-shaped region IR into the view plane of the depth image, MR p comprises a projection of the first annular-shaped region MR into the view plane of the depth image, OR p comprises a projection of the second annual-shape region OR into the view plane of the depth image, and
SUM
=
∑
I
(
i
,
j
)
∈
IR
p
I
(
i
,
j
)
+
∑
I
(
i
,
j
)
∈
MR
p
I
(
i
,
j
)
+
∑
I
(
i
,
j
)
∈
OR
p
I
(
i
,
j
)
.
15. The system according to claim 10 , wherein the first annular-shaped MR region is arranged concentrically with respect to the circular-shaped region IR.
16. The system according to claim 10 , wherein the processor is further configured to scale the template so that the template points M(x,y) comprise a same pitch and a same orientation as pixels of the depth image.
17. The system according to claim 10 , wherein the depth image comprises a monochrome depth image.
18. The system according to claim 10 , wherein the depth image comprises a color depth image.
19. At least one non-transitory machine-readable medium comprising a plurality of instructions that in response to being executed on a computing device, cause the computing device to:
receive a depth image, the depth image comprising a foreground portion and the foreground portion comprising a first plurality of pixels;
select a second plurality of pixels of the foreground portion of the depth image, the second plurality of pixels being equal to or less than the first plurality of pixels;
for each selected pixel of the second plurality of pixels,
center a center point M(x,y) of a template with respect to the selected pixel, the template comprising the center point and a plurality of template points M(i,j), the template further comprising a circular-shaped region IR and a first annular-shaped region MR surrounding the circular shaped region, a depth of a template point M(i,j) in the circular-shaped region IR and in the first annular region MR being defined as:
for
M
(
i
,
j
)
=
∈
IR
:
{
∑
d
≤
R
c
R
p
2
-
d
2
/
R
p
Num
(
IR
)
·
(
∑
d
≤
R
c
(
R
p
2
-
d
2
/
R
p
)
+
∑
R
c
≤
d
≤
R
p
(
R
p
2
-
d
2
/
R
p
)
)
and
for
M
(
i
,
j
)
∈
MR
:
{
∑
R
c
≤
d
≤
R
p
(
R
p
2
-
d
2
/
R
p
)
Num
(
MR
)
·
(
∑
d
≤
R
c
(
R
p
2
-
d
2
/
R
p
)
+
∑
R
c
≤
d
≤
R
p
(
R
p
2
-
d
2
/
R
p
)
)
wherein, R p comprises a projection of a radius of the head in a view plane of the depth image, d comprises a distance between a template point M(i,j) and the center point M(x,y) of the template, R c comprises a radius of the circular-shaped region IR, Num(IR) comprises a total number of template points within the circular-shaped region IR, Num(MR) comprises a total number of template points within the first annular-shaped region MR, i and j comprise rectangular coordinate indices associated with pixels of the depth image, and x and y comprise rectangular coordinate indices associated with the template, index i corresponding to index x, and index j corresponding to index y; and
determine a depth value for at least one pixel neighboring the selected pixel;
determine a score H(x,y) associated with the selected pixel based on a depth value for at least one neighboring pixel in comparison to a depth value associated with a template point corresponding to the neighboring pixel; and
determine a highest score of the determined scores; and
select the pixel associated with the highest score as a center of the human head.
20. The at least one machine-readable medium according to claim 19 , the projection of the radius R p of the head in the image view plane comprises a radius of the first annular-shaped region, and
wherein R p comprises a value of 1.25 times R c .
21. The at least one machine-readable medium according to claim 19 , wherein the template further comprises a second annular-shaped region OR surrounding the first annular-shaped region MR, a depth of a template point M(i,j) in the circular-shaped region OR being defined as:
for
M
(
i
,
j
)
∈
OR
:
-
1
Num
(
OR
)
wherein, Num(OR) comprises a total number of template points within second annular-shaped region OR.
22. The at least one machine-readable medium according to claim 21 , the projection of the radius R p of the head in the image view plane comprises a radius of the first annular-shaped region, and
wherein R p comprises a value of 1.25 times R c .
23. The at least one machine-readable medium according to claim 21 , wherein determining the score H(x,y) comprises determining
H
(
x
,
y
)
=
(
∑
I
(
i
,
j
)
∈
IR
p
I
(
i
,
j
)
·
depth
(
M
(
x
,
y
)
)
+
∑
I
(
i
,
j
)
∈
MR
p
I
(
i
,
j
)
·
depth
(
M
(
x
,
y
)
)
+
∑
I
(
i
,
j
)
∈
OR
p
I
(
i
,
j
)
·
depth
(
M
(
X
,
j
)
)
)
/
SUM
wherein, I(i,j) comprises a depth value of each of the at least one neighboring pixel of the selected pixel, IR p comprises a projection of circular-shaped region IR into the view plane of the depth image, MR p comprises a projection of the first annular-shaped region MR into the view plane of the depth image, OR p comprises a projection of the second annual-shape region OR into the view plane of the depth image, and
SUM
=
∑
I
(
i
,
j
)
∈
IR
p
I
(
i
,
j
)
+
∑
I
(
i
,
j
)
∈
MR
p
I
(
i
,
j
)
+
∑
I
(
i
,
j
)
∈
OR
p
I
(
i
,
j
)
.
24. The at least one machine-readable medium according to claim 20 , further comprising instructions that in response to being executed on a computing device, cause the computing device to scale the template so that the template points M(x,y) comprise a same pitch and a same orientation as pixels of the depth image.
25. The at least one machine-readable medium according to claim 19 , wherein the depth image comprises a monochrome depth image or a color depth image.