IP Library Granted Patent US 10,565,468
Granted Patent B2
US 10,565,468 · App. 15/000,730 · Granted Feb 18, 2020

Object tracking system with radar/vision fusion for automated vehicles

Inventor: Jan K. Schiffmann (Newbury Park, CA)
Assignee: Aptiv Technologies Limited
G06K9/6215B60R1/00G01S13/66G01S13/867G01S13/931G06K9/00791B60R2300/301
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,565,468
App. No.
15/000,730
Granted
Feb 18, 2020
Kind
B2
Abstract

An object tracking system suitable for use on an automated vehicle includes a camera, a radar-sensor and a controller. The controller is configured to assign a vision-identification to each vision-track associated with an instance of an object detected using the camera, and assign a radar-identification to each radar-glob associated with an instance of grouped-tracklets indicated detected using the radar-sensor. The controller is further configured to determine probabilities that a vision-track and a radar-glob indicate the same object. If the combination has a reasonable chance of matching it is includes in a further screening of the data to determine a combination of pairings of each vision-track to a radar-track that has the greatest probability of being the correct combination.

Claims (30)

1. An object tracking system suitable for use on an automated vehicle, said system comprising:

a camera used to capture an image of a vision-field-of-view (VFOV) of the camera;

a radar-sensor used to detect a radar-signal reflected from a radar-field-of-view (RFOV) that overlaps the VFOV; and

a controller in communication with the camera and the radar-sensor, said controller configured to assign a vision-identification to each vision-track that indicates a vision-motion-status associated with an instance of an object detected in the image by the controller,

assign a radar-identification to each radar-glob that indicates a radar-motion-status associated with an instance of grouped-tracklets indicated by the radar-signal,

determine a match-feasibility for each combination of vision-track and radar-glob, wherein each match-feasibility is a binary-value set equal to one (1) when at least one of

a) an azimuth-difference between a vision-angle of the vision-track and a radar-angle of a radar-glob is less than an azimuth-threshold,

b) a distance-difference between a vision-distance of the vision-track and a radar-distance of a radar-glob is less than a distance-threshold, and

c) a speed-difference between a vision-speed of the vision-track and a radar-speed of a radar-glob is less than a speed-threshold,

select from all possible vision-track and radar-glob pairing-combinations a list of feasible-match-patterns and determine a fused track based on the vision-motion-status and the radar-motion-status, wherein a pairing-combination is a feasible-match-pattern when the match-feasibility of each combination of vision-track and radar-glob in the pairing-combination is equal to one,

determine a normalized-value for each instance of the feasible-match-pattern, and

calculate a raw-probability for each combination of vision-track and radar-glob by summing the normalized-values of the feasible-match-patterns that indicate a match for that combination of vision-track and radar-glob.

2. The system in accordance with claim 1 , wherein the controller is configured to

determine a match-probability for each combination of vision-track and radar-glob by low-pass filtering of the raw-probability obtained for that combination,

determine a need for a re-organize test of the existing associations between vision-tracks and radar-globs by comparing the match-probabilities for combinations of vision-track and radar-glob to a threshold, and

implement a re-organize test of associations between vision-tracks and radar-globs when one of the match-probabilities transitions the threshold.

3. The system in accordance with claim 1 , wherein the match-feasibility for a combination of vision-track and radar-glob is set equal to zero (0) when the azimuth-difference is greater than the azimuth-threshold.

4. The system in accordance with claim 1 , wherein each match-feasibility is set equal to one (1) only when

a) the azimuth-difference is less than the azimuth-threshold,

b) the distance-difference is less than the distance-threshold, and

c) the speed-difference between is less than the speed-threshold.

5. The system in accordance with claim 1 , wherein each match-feasibility is set equal to one (1) when at least one of

a) the azimuth-difference is less than the azimuth-threshold,

b) the distance-difference is less than the distance-threshold,

c) the speed-difference between is less than the speed-threshold, and

d) a vision-motion-status of the vision-track matches a radar-motion-status of the radar glob, wherein the vision-motion-status and the radar-motion-status are each characterized by one of oncoming, stationary, and receding.

6. The system in accordance with claim 1 , wherein the vision-speed and the radar-speed are indicative of a longitudinal-velocity of the vision-track and the radar-glob, respectively.

7. The system in accordance with claim 1 , wherein the vision-distance and the radar-distance are indicative of a longitudinal-position of the vision-track and the radar-glob, respectively.

8. The system in accordance with claim 1 , wherein the quality-value is based on a reciprocal of a square-root of a weighted-summation of squares of the azimuth-difference, the distance-difference, and the speed-difference.

9. The system in accordance with claim 1 , wherein the controller is configured to perform an occlusion-test on radar-globs characterized by substantially equal values of azimuth-angle to null the match-probability for any combination of vision-track and radar-glob that the occlusion-test indicates is not possible.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 11, 2024
From: APTIV MANUFACTURING MANAGEMENT SERVICES S.À R.L.
To: APTIV TECHNOLOGIES AG
Reel/Frame 066551/0219 →
MERGER Recorded Feb 11, 2024
From: APTIV TECHNOLOGIES (2) S.À R.L.
To: APTIV MANUFACTURING MANAGEMENT SERVICES S.À R.L.
Reel/Frame 066566/0173 →
ENTITY CONVERSION Recorded Feb 11, 2024
From: APTIV TECHNOLOGIES LIMITED
To: APTIV TECHNOLOGIES (2) S.À R.L.
Reel/Frame 066746/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 26, 2018
From: DELPHI TECHNOLOGIES INC.
To: APTIV TECHNOLOGIES LIMITED
Reel/Frame 047153/0902 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 19, 2016
From: SCHIFFMANN, JAN K.
To: DELPHI TECHNOLOGIES, INC.
Reel/Frame 037561/0182 →
Continuity (1)
Related Publication 20170206436A1 · Jul 20, 2017
Cited By (4)
US 12,190,601 US 12,282,089 US 12,380,684 US 12,545,249