Continuous occlusion models for road scene understanding
View Patent ↗Systems and methods are disclosed for road scene understanding of vehicles in traffic by capturing images of traffic with a camera coupled to a vehicle; generating a continuous model of occlusions with a continuous occlusion mode for traffic participants to enhance point track association accuracy without distinguishing between moving and static objects; applying the continuous occlusion model to handle visibility constraints in object tracks; and combining point track association and soft object track modeling to improve 3D localization accuracy.
1. A method for road scene understanding of vehicles in traffic, comprising
capturing images of traffic with a camera coupled to a vehicle;
generating a continuous model of occlusions with a continuous occlusion mode for traffic participants to enhance point track association accuracy without distinguishing between moving and static objects;
applying the continuous occlusion model to handle visibility constraints in object tracks;
combining the point track association and soft object track modeling to improve 3D localization accuracy; and
modelling objects as translucent 3D ellipsoids whose opacity is maximum at the center and falls off towards the edges, further comprising modeling occupancy at location x corresponding to a traffic participant-centered at p as:
ƒ occ ( x )= L ( x;p, Σ)
where L(•) is a logistic given by
L
(
x
;
p
,
Σ
)
=
1
1
+
e
-
k
(
1
-
d
(
x
,
p
)
)
,
with d(x,p)=(x−p) T Σ(x−p) being a Mahalanobis distance.
2. The method of claim 1 , comprising applying physical insights to model occlusion relationships.
3. The method of claim 1 , comprising applying an occupancy model for traffic participants (TPs) that treats the TPs as translucent objects reflected by an occupancy function.
4. The method of claim 1 , comprising applying a reflection model for handling visible points.
5. The method of claim 1 , comprising applying a transmission model for representing occlusions from all other TPs in a scene.
6. The method of claim 1 , comprising applying models to determine soft assignments of point tracks to TPs.
7. The method of claim 1 , comprising applying models to account for occlusion relationships in object detection scores.
8. The method of claim 1 , comprising applying 3D localization using the energies from point tracks and detection models, along with energies that incorporate transition and collision constraints.
9. The method of claim 1 , comprising defining an image formation model that accounts for reflection and transmission as a soft way to model occlusions.
10. The method of claim 1 , comprising defining a reflection probability model that determines when a point is visible in the image.
11. The method of claim 1 , comprising defining transmission probability model proposes a soft way to deal with occlusions due to intermediate objects in the path of a back-projected ray.
12. The method of claim 1 , comprising defining a point tracks association energy to assign points in a soft fashion to various traffic participants.
13. The method of claim 1 , comprising defining an object tracks energy to probabilistically adjust detection bounding boxes and scores to account for occlusions.
14. A method for road scene understanding of vehicles in traffic, comprising
capturing images of traffic with a camera coupled to a vehicle;
generating a continuous model of occlusions with a continuous occlusion mode for traffic participants to enhance point track association accuracy without distinguishing between moving and static objects;
applying the continuous occlusion model to handle visibility constraints in object tracks;
combining the point track association and soft object track modeling to improve 3D localization accuracy;
defining energy by a dot product between car orientation and tangent to a lane at a point; and
determining
E
lane
it
=
∑
m
∈
M
close
(
1
-
ω
i
(
t
)
·
TAN
(
L
m
(
k
)
,
p
i
(
t
)
)
)
Σ
L
m
(
p
i
(
t
)
)
where M close ={m:DIST(L m (k),p i (t))<50} is the set of nearby lanes and
Σ
L
m
(
p
i
(
t
)
)
=
1
1
+
exp
(
-
q
(
w
road
-
DIST
(
L
m
(
k
)
,
p
i
(
t
)
)
)
)
for a constant that represents a road width.
15. A method for road scene understanding of vehicles in traffic, comprising
capturing images of traffic with a camera coupled to a vehicle;
generating a continuous model of occlusions with a continuous occlusion mode for traffic participants to enhance point track association accuracy without distinguishing between moving and static objects;
applying the continuous occlusion model to handle visibility constraints in object tracks;
combining the point track association and soft object track modeling to improve 3D localization accuracy;
applying a localization framework that combines a plurality of energies along with collision and transition energies and size priors; and
determining prior energy
E prior it =( B i - {circumflex over (B)} ) T Σ {circumflex over (B)} −1 ( B i − {circumflex over (B)} )
where {circumflex over (B)} is mean TP dimensionns and Σ {circumflex over (B)} is a correspondence covariance matrix.