IP Library Granted Patent US 9,460,343
Granted Patent B2
US 9,460,343 · App. 14/696,903 · Granted Oct 4, 2016

Method and system for proactively recognizing an action of a road user

Inventors: Daniel Suerth (Munich, DE); Magdalena Szczot (Zorneding, DE); Joe Lallemand (Munich, DE)
Assignee: Bayerische Motoren Werke Aktiengesellschaft
G06K9/00382G06K9/6202G06T7/0044G06T2207/30196G06T2207/30261
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,460,343
App. No.
14/696,903
Granted
Oct 4, 2016
Kind
B2
Abstract

A method and system are provided to proactively recognize an action of a road user in road traffic, wherein an image of the road user, which is structured in a pixel-wise manner, is captured by way of at least one camera, and corresponding image data is generated. Image data of multiple pixels is grouped in each case by cells, wherein the image comprises multiple cells. A respective centroid is determined based on the image data within a cell. For each of the pixels, the respective distance from the centroids of a plurality of cells is ascertained, wherein a feature vector that is assigned to the pixel is formed based on coordinates of the respective pixel and the centroids. The feature vector is compared to at least one reference vector cluster, and a pose is associated with the road user based on the comparison.

Claims (34)

1. A method for proactively recognizing an action of a road user in road traffic, the method comprising the acts of:

capturing an image of the road user, which is structured in a pixel-wise manner, by way of at least one camera, and generating corresponding image data;

grouping pixels from the image data into a corresponding plurality of cells, wherein the image comprises the plurality of cells;

determining a respective cell centroid based on the image data for each of the plurality of cells;

for each of the pixels, determining a distance from the cell centroids of the plurality of cells, wherein a feature vector is assigned to each pixel based on coordinates of the respective pixel and the cell centroids; and

comparing the feature vector for each of the pixels to at least one reference vector cluster;

associating a pose, from a predetermined number of poses, with the road user based on said comparing; and

determining an expected action of the road user based at least in part on the associated pose.

2. The method according to claim 1 , wherein for determining the respective cell centroid, the method further comprises the acts of:

filtering the image data captured by the camera to the effect that data of a contiguous image area is created, wherein the road user is depicted in the image area and at least some of the other image components captured by the camera are filtered out; and

wherein the plurality of cells comprise the contiguous image area, and further wherein each of the plurality of cells comprises at least one pixel of the contiguous area.

3. The method according to claim 2 , wherein a respective pixel-wise orientation of the road user relative to the camera is associated in the course of the comparison of the respective feature vectors of the respective pixels to the respective at least one reference vector cluster and, based on the associated respective pixel-wise orientation, an orientation of the road user is associated.

4. The method according to claim 3 , wherein, based on the associated orientation, the feature vectors are compared pixel-wise to the respective reference vector clusters thereof and/or further reference vector clusters, which are classified according to possible poses of the road user, and in particular according to the associated orientation, and the pose is associated by way of the result of the comparison.

5. The method according to claim 1 , wherein the comparison of the feature vectors to the reference vector clusters is carried out by way of a random forest method, wherein decision trees constructed during a learning process are used, which comprise the reference vector clusters.

6. The method according to claim 1 , wherein further data is detected and used to proactively recognize the action of the road user, such further data comprising surroundings data, geographic data, measurement data regarding the road user and/or data of the motor vehicle.

7. The method according to claim 1 , wherein a stereo camera system is provided, which comprises first and second cameras, wherein the image of the road user is captured three-dimensionally using image data.

8. A system for proactively recognizing an action of a road user in road traffic, comprising:

at least one camera, which is usable to capture an image of the road user that is structured in a pixel-wise manner, and which generates corresponding image data; and

a data processing system configured to:

group pixels from the image data into a corresponding plurality of cells, wherein the image comprises the plurality of cells;

determine a respective cell centroid based on the image data for each of the plurality of cells;

ascertain, for each of the pixels, the respective distance from the cell centroids of the plurality of cells, wherein a feature vector is assigned to each pixel based on coordinates of the respective pixel and the cell centroids; and

compare the feature vector for each of the pixels to at least one reference vector cluster;

associate a pose, from a predetermined number of poses, with the road user based on the comparison; and

determine an expected action of the road user based at least in part on the associated pose.

9. The method of claim 1 , further comprising issuing a warning to a vehicle driver in response to the determined expected action of the road user.

10. The method of claim 1 , further comprising associating poses, from the predetermined number of poses, with the road user for each of the pixels comprising the image data,

wherein the method further comprises determining a final pose based on the associated poses for each of the pixels, and

wherein determining the expected action comprises determining the expected action of the road user based at least in part on the final pose.

11. The system of claim 8 , wherein the data processing system is further configured to issue a warning to a vehicle driver in response to the determined expected action of the road user.

12. The system of claim 8 , wherein the data processing system is further configured to:

associate poses, from the predetermined number of poses, with the road user for each of the pixels comprising the image data,

determine a final pose based on the associated poses for each of the pixels, and

determine the expected action based at least in part on the final pose.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 21, 2015
From: SUERTH, DANIEL; SZCZOT, MAGDALENA; LALLEMAND, JOE
To: BAYERISCHE MOTOREN WERKE AKTIENGESELLSCHAFT
Reel/Frame 036139/0499 →
Priority Claims (1)
DE 10 2014 207 802 · Apr 25, 2014 · national
Continuity (1)
Related Publication 20150310265A1 · Oct 29, 2015