IP Library › Granted Patent US 10,710,579
Granted Patent B2
US 10,710,579 · App. 15/611,397 · Granted Jul 14, 2020

Collision prediction system

Inventors: Wan-Yen Lo (Sunnyvale, CA); Abhijit Ogale (Sunnyvale, CA); David Ferguson (El Dorado Hills, CA)
Assignee: Waymo LLC
B60W30/09B60W50/0097B60W50/14G05D1/0088G08G1/096725G08G1/165G08G1/166G05D1/024
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,710,579
App. No.
15/611,397
Filed
Jun 1, 2017
Granted
Jul 14, 2020
Kind
B2
Art Unit
3666
USPC
701/301
Abstract

In some implementations, an autonomous or semi-autonomous vehicle is capable of using a collision prediction system to determine a confidence that any objects detected within a vicinity of the vehicle are on a trajectory that will collide with the vehicle. Laser obstacle points derived from recent sensor readings of one or more sensors of a vehicle are initially obtained. The laser obstacle points are projected into a pose coordinate system to generate an occupancy grid of a vicinity of the vehicle. A confidence that any objects represented by the laser obstacle points are on a trajectory that will collide with the vehicle is determined by applying a particle filter to the occupancy grid.

Claims (76)

1. A computer-implemented method comprising:

obtaining, by a computer system on-board a vehicle, laser obstacle points derived from recent sensor readings of one or more sensors of the vehicle;

obtaining, by the computer system on-board the vehicle, vehicle movement data representing a vehicle trajectory of the vehicle;

projecting, by the computer system on-board the vehicle, the laser obstacle points into a pose coordinate system to generate an occupancy grid of a vicinity of the vehicle, wherein the occupancy grid comprises multiple frames, and wherein each of the multiple frames comprises a plurality of laser obstacle points;

clustering, by the computer system on-board the vehicle, the projected laser obstacle points in the occupancy grid into one or more object clusters that each represents one or more corresponding objects;

generating, by the computer system on-board the vehicle and for each object cluster, an object trajectory from movement of the object cluster within the multiple frames of the occupancy grid;

for each object cluster:

assigning, by the computer system on-board the vehicle and using particle filtering, a respective weight to each of a plurality of individual particles that are sampled from within the object cluster based on correspondences between the trajectories of the individual particles that are sampled from within the object cluster and the object trajectories of the object clusters;

determining, by the computer system on-board the vehicle, a respective particle collision score for each of the plurality of individual particles from the object cluster that represents a likelihood that the trajectory of the individual particle will collide with the vehicle trajectory of the vehicle at a future time point; and

determining, by the computer system on-board the vehicle, a cluster collision score for the object cluster based on the respective weights and respective particle collision score for the individual particles from the cluster; and

determining, by the computer system on-board the vehicle and from the cluster collision scores, a confidence that any objects represented by the laser obstacle points are on a trajectory that will collide with the vehicle at the future time point.

2. The method of claim 1 , further comprising:

when the confidence exceeds a threshold confidence, providing a collision alert to a control system for the vehicle.

3. The method of claim 1 , wherein:

the sensor readings are collected over a particular time period; and

wherein the multiple frames of the occupancy grid each correspond to a different time point within the particular time period.

4. The method of claim 1 , wherein generating, for each object cluster, an object trajectory from movement of the object cluster within the multiple frames of the occupancy grid comprises:

determining a respective coordinate location of the object cluster within each of the multiple frames of the occupancy grid; and

determining a respective object trajectory for the object cluster based on the determined respective locations of the object cluster within each of the multiple frames of the occupancy grid.

5. The method of claim 1 , wherein clustering the projected laser obstacle points in the occupancy grid into one or more object clusters that each represents one or more corresponding objects comprises:

clustering the laser obstacle points into one or more candidate object clusters; and

discarding any candidate object clusters having a size that does not satisfy predetermined size criteria.

6. The method of claim 1 , wherein assigning, using particle filtering, a respective weight to each of a plurality of individual particles that are sampled from within the object cluster based on correspondences between the trajectories of the individual particles that are sampled from within the object cluster and the object trajectories of the object clusters comprises:

determining a plurality of particles that are within the vicinity of the vehicle in the occupancy grid;

for each of the plurality of particles:

determining a location of the particle within each of the multiple frames of the occupancy grid;

determining a particle trajectory based on the determined locations of the particle within each of the multiple frames of the occupancy grid; and

assigning a weight to the particle based on a correspondence between the particle trajectory and an object trajectory of an object cluster that includes the particle.

7. The method of claim 6 , wherein assigning, using particle filtering, a respective weight to each of a plurality of individual particles that are sampled from within the object cluster based on correspondences between the trajectories of the individual particles that are sampled from within the object cluster and the object trajectories of the object clusters further comprises, for each of the one or more object clusters:

combining the particle trajectories using the respective weights assigned to the particles included in the object cluster to generate one or more combined particle trajectories.

8. A system comprising:

one or more computers; and

one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:

obtaining laser obstacle points derived from recent sensor readings of one or more sensors of a vehicle;

obtaining vehicle movement data representing a vehicle trajectory of the vehicle;

projecting the laser obstacle points into a pose coordinate system to generate an occupancy grid of a vicinity of the vehicle, wherein the occupancy grid comprises multiple frames, and wherein each of the multiple frames comprises a plurality of laser obstacle points;

clustering the projected laser obstacle points in the occupancy grid into one or more object clusters that each represents one or more corresponding objects;

generating, for each object cluster, an object trajectory from movement of the object cluster within the multiple frames of the occupancy grid;

for each object cluster:

assigning, using particle filtering, a respective weight to each of a plurality of individual particles that are sampled from within the object cluster based on correspondences between the trajectories of the individual particles that are sampled from within the object cluster and the object trajectories of the object clusters;

determining a respective particle collision score for each of the plurality of individual particles from the object cluster that represents a likelihood that the trajectory of the individual particle will collide with the vehicle trajectory of the vehicle at a future time point; and

determining a cluster collision score for the cluster based on the respective weights and respective particle collision score for the individual particles from the cluster; and

determining, from the cluster collision scores, a confidence that any objects represented by the laser obstacle points are on a trajectory that will collide with the vehicle at the future time point.

9. The system of claim 8 , wherein the operations further comprise:

when the confidence exceeds a threshold confidence, providing a collision alert to a control system for the vehicle.

10. The system of claim 8 , wherein:

the sensor readings are collected over a particular time period; and

wherein the multiple frames of the occupancy grid each correspond to a different time point within the particular time period.

11. The system of claim 8 , wherein the operations of generating, for each object cluster, an object trajectory from movement of the object cluster within the multiple frames of occupancy grid comprise:

determining a respective coordinate location of the object cluster within each of the multiple frames of the occupancy grid; and

determining a respective object trajectory based on the determined respective locations of the object cluster within each of the multiple frames of the occupancy grid.

12. The system of claim 8 , wherein the operations of assigning, using particle filtering, a respective weight to each of a plurality of individual particles that are sampled from within the object cluster based on correspondences between the trajectories of the individual particles that are sampled from within the object cluster and the object trajectories of the object clusters comprises:

determining a plurality of particles that are included in each of the one or more object clusters; and

for each of the plurality of particles:

determining a location of the particle within each of the multiple frames of the occupancy grid;

determining a particle trajectory based on the determined locations of the particle within each of the multiple frames of the occupancy grid; and

assigning a weight to the particle based on a correspondence between the particle trajectory and an object trajectory of an object cluster that includes the particle.

13. A non-transitory computer-readable storage device encoded with computer program instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:

obtaining laser obstacle points derived from recent sensor readings of one or more sensors of a vehicle;

obtaining vehicle movement data representing a vehicle trajectory of the vehicle;

projecting the laser obstacle points into a pose coordinate system to generate an occupancy grid of a vicinity of the vehicle, wherein the occupancy grid comprises multiple frames, and wherein each of the multiple frames comprises a plurality of laser obstacle points;

clustering the projected laser obstacle points in the occupancy grid into one or more object clusters that each represents one or more corresponding objects;

generating, for each object cluster, an object trajectory from movement of the object cluster within the multiple frames of the occupancy grid;

for each cluster:

assigning, using particle filtering, a respective weight to each of a plurality of individual particles that are sampled from within the object cluster based on correspondences between the trajectories of the individual particles that are sampled from within the object cluster and the object trajectories of the object clusters;

determining a respective particle collision score for each of the plurality of individual particles from the object cluster that represents a likelihood that the trajectory of the individual particle will collide with the vehicle trajectory of the vehicle at a future time point; and

determining a cluster collision score for the cluster based on the respective weights and respective particle collision score for the individual particles from the cluster; and

determining, from the cluster collision scores, a confidence that any objects represented by the laser obstacle points are on a trajectory that will collide with the vehicle at the future time point.

14. The device of claim 13 , wherein the operations further comprise:

when the confidence exceeds a threshold confidence, providing a collision alert to a control system for the vehicle.

15. The device of claim 13 , wherein:

the sensor readings are collected over a particular time period; and

wherein the multiple frames of the occupancy grid each correspond to a different time point within the particular time period.

16. The device of claim 13 , wherein the operations of generating, for each object cluster, an object trajectory from movement of the object cluster within the multiple frames of occupancy grid comprise:

determining a respective coordinate location of the object cluster within each of the multiple frames of the occupancy grid; and

determining a respective object trajectory based on the determined respective locations of the object cluster within each of the multiple frames of the occupancy grid.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 11, 2017
From: LO, WAN-YEN; OGALE, ABHIJIT; FERGUSON, DAVID
To: WAYMO LLC
Reel/Frame 043831/0110 →
Continuity (1)
Related Publication 20180345958A1 · Dec 6, 2018
Cited By (1)
US 12,387,615