Method and a device for generating an occupancy map of an environment of a vehicle
A method of generating an occupancy map representing free and occupied space around a vehicle, wherein the occupancy map is divided into a plurality of cells Mx,y, the method includes: capturing two consecutive images by a camera mounted on the vehicle; generating optical flow vectors from the two consecutive images; estimating 3D points in the space around the vehicle from the optical flow vectors; generating rays from the camera to each of the estimated 3D points, wherein intersection points of the rays with the cells Mx,y defining further 3D points; determining for each of the cells Mx,y a function L_(x,y)^t for a time step t; and determining an occupancy probability from the function L_(x,y)^t for each of the cells Mx,y.
1. A method of generating an occupancy map representing free and occupied space around a vehicle, wherein the occupancy map is divided into a plurality of cells M x,y , the method comprising:
capturing two consecutive images by a camera ( 3 ) mounted on the vehicle;
generating optical flow vectors from the two consecutive images;
estimating 3D points in the space around the vehicle from the optical flow vectors;
generating rays from the camera ( 3 ) to each of the estimated 3D points, wherein the cells M x,y structure the space around the vehicle and intersection points of the rays with the cells M x,y in the space around the vehicle define further 3D points;
determining for each of the cells M x,y a function L x,y t for a time step t:
L
x
,
y
t
=
L
x
,
y
t
-
1
+
Σ
i
log
P
(
M
x
,
y
|
h
i
,
d
i
)
1
-
P
(
M
x
,
y
|
h
i
,
d
i
)
,
wherein i denotes the 3D points located within the respective cell M x,y and each of the 3D points i is defined by a height h i above ground and a distance d i between the 3D point i and a 3D point on the respective ray where the ray hits the ground or an object,
wherein P(M x,y |h i ,d i ) is a function of
Σ
S
P
(
M
x
,
y
|
S
,
h
i
)
L
(
S
|
d
i
)
Σ
S
L
(
S
|
d
i
)
,
wherein P(M x,y |S,h i ) is an occupancy probability contribution of a 3D point i dependent on a status S of the 3D point i and the height h i , L(S|d i ) is a likelihood of the status of the 3D point i dependent on the distance d i , and S={hit, pass, covered}, wherein the status S of a 3D point i is S=hit if the 3D point hits the ground or an object, S=pass if the 3D point is located between the camera ( 3 ) and the 3D point on the respective ray where the ray hits the ground or an object, and S=covered if the 3D point is located on the respective ray beyond the 3D point on the respective ray where the ray hits an object; and
determining an occupancy probability from the function L x,y t for each of the cells M x,y .
2. The method as claimed in claim 1 , wherein P (M x,y |h 1 ,d i ) is given by
P ( M x,y |h i ,d i )ϕ Th ϕ Td +ϕ Dh ϕ Dd +ϕ Vh ϕ Vd
with functions ϕ Th , ϕ Dh and ϕ Vh each generating a probability and functions ϕ Td , ϕ Dd and ϕ Vd each generating a likelihood:
ϕ
Th
=
P
(
M
x
,
y
|
S
=
hit
,
h
i
)
,
ϕ
Dh
=
P
(
M
x
,
y
|
S
=
pass
,
h
i
)
,
ϕ
Vh
=
P
(
M
x
,
y
|
S
=
covered
,
h
i
)
,
ϕ
Td
=
L
(
S
=
hit
|
d
i
)
L
(
S
=
hit
|
d
i
)
+
L
(
S
=
pass
|
d
i
)
+
L
(
S
=
covered
|
d
i
)
,
ϕ
Dd
=
L
(
S
=
pass
|
d
i
)
L
(
S
=
hit
|
d
i
)
+
L
(
S
=
pass
|
d
i
)
+
L
(
S
=
covered
|
d
i
)
,
and
ϕ
Vd
=
L
(
S
=
covered
|
d
i
)
L
(
S
=
hit
|
d
i
)
+
L
(
S
=
pass
|
d
i
)
+
L
(
S
=
covered
|
d
i
)
.
3. The method as claimed in claim 2 , wherein the function ϕ Th is an inverted Gaussian function with a minimum at h i =0.
4. The method as claimed in claim 3 , wherein the function ϕ Dh has a minimum in a first range ( 50 ) for h i around h i =0 and assumes 1 for values of h i that are smaller than the first range ( 50 ) and assumes the constant value for values of h i that are greater than the first range ( 50 ).
5. The method as claimed in in claim 2 , wherein the function ϕ Vh assumes a constant value for all values of h i .
6. The method as claimed in claim 2 , wherein the function ϕ Dd is a Gaussian function with a maximum at h i =0.
7. The method as claimed in claim 2 , wherein the function ϕ Dd has a transition area for values of d i in a second range ( 51 ) around d i =0 where the function ϕ Dd rises from 0 to 1, wherein the function ϕ Dd assumes 0 for values of d i that are smaller than the second range ( 51 ) and assumes 1 for values of d i that are greater than the second range ( 51 ).
8. The method as claimed in claim 2 , wherein the function ϕ Vd has a transition area for values of d i in a second range ( 51 ) around d i =0 where the function ϕ Vd falls from 1 to 0, wherein the function ϕ Vd assumes 1 for values of d i that are smaller than the second range ( 51 ) and assumes 0 for values of d i that are greater than the second range ( 51 ).
9. The method as claimed in claim 1 , wherein the occupancy probability is determined from the function L x,y t for each of the cells M x,y by
P
t
(
M
x
,
y
)
=
1
-
1
1
+
e
L
x
,
y
t
.
10. The method as claimed in claim 1 , wherein the occupancy map is used as an input for a driver assistance system or a system for autonomous driving, in particular a system for path planning for autonomous driving, a system for obstacle detection, a system for lane departure warning, a system for forward collision warning or a system for autonomous emergency braking.
11. A device for generating an occupancy map representing free and occupied space around a vehicle, wherein the occupancy map is divided into a plurality of cells M x,y , the device is configured to
receive two consecutive images captured by a camera mounted on the vehicle,
generate optical flow vectors from the two consecutive images,
estimate 3D points in the space around the vehicle from the optical flow vectors,
generate rays from the camera to each of the estimated 3D points, wherein the cells M x,y structure the space around the vehicle and intersection points of the rays with the cells M x,y in the space around the vehicle define further 3D points,
determine for each of the cells M x,y a function L x,y t for a time step t:
L
x
,
y
t
=
L
x
,
y
t
-
1
+
Σ
i
log
P
(
M
x
,
y
|
h
i
,
d
i
)
1
-
P
(
M
x
,
y
|
h
i
,
d
i
)
,
wherein i denotes the 3D points located within the respective cell M x,y and each of the 3D points i is defined by a height h i above ground and a distance d i between the 3D point i and a 3D point on the respective ray where the ray hits the ground or an object,
wherein P(M x,y |h i ,d i ) is a function of
Σ
S
P
(
M
x
,
y
|
S
,
h
i
)
L
(
S
|
d
i
)
Σ
S
L
(
S
|
d
i
)
,
wherein P(M x,y |S,h i ) is an occupancy probability contribution of a 3D point i dependent on a status S of the 3D point i and the height h i , L (S|d i ) is a likelihood of the status of the 3D point i dependent on the distance d i , and S={hit, pass, covered}, wherein the status S of a 3D point i is S=hit if the 3D point hits the ground or an object, S=pass if the 3D point is located between the camera ( 3 ) and the 3D point on the respective ray where the ray hits the ground or an object, and S=covered if the 3D point is located on the respective ray beyond the 3D point on the respective ray where the ray hits an object; and
determine an occupancy probability from the function L x,y t for each of the cells M x,y .
12. A system comprising: a device for generating an occupancy map as claimed in claim 11 , and a driver assistance system or a system for autonomous driving, wherein the occupancy map is input in the driver assistance system or the system for autonomous driving.