Granted Patent B2

US 12,227,208 · App. 17/841,382 · Granted Feb 18, 2025

Inventors: Juan Fasola (San Francisco, CA); Eugene Lo (Redwood City, CA); Xiaoyu Zhou (San Francisco, CA); Shreyans Kushwaha (Burlingame, CA)

Assignee: GM Cruise Holdings LLC

B60W60/0027B60W30/09B60W50/023G06N5/01B60W2554/4044B60W2555/60

Abstract

Claims (74)

1. A method comprising:

collecting sensor data generated by one or more sensors of a vehicle;

detecting, by a fallback control system of the vehicle based on the sensor data, an imminent collision between the vehicle and an object in an environment of the vehicle using a first process different from a second process of a primary control system of the vehicle,

wherein the detecting of the imminent collision is performed repeatedly,

wherein the first process and the second process are associated with at least one of a perception or a prediction,

wherein the first process includes: predicting a trajectory of the vehicle and a trajectory of the object in the environment of the vehicle is at least partially overlapping, and

classifying the object to determine an object type of the object,

determining, by the fallback control system and based on the object type of the object, whether to suppress a trigger, which indicates the detected imminent collision, from being transmitted to the primary control system;

suppressing transmission of the trigger when the object type of the object corresponds to an assumption that the object is observant of a traffic rule that would result in avoidance of the detected imminent collision between the vehicle and the object;

transmitting, by the fallback control system to the primary control system, the trigger when the object type of the object does not correspond to the assumption that the object is observant of the traffic rule that would result in avoidance of the detected imminent collision between the vehicle and the object;

wherein the transmitting of the trigger is further based on a number of consecutive detected imminent collisions satisfying a threshold number of consecutive detections;

adjusting the threshold number of consecutive detections for determining whether to transmit the trigger based on the object type of the object; and

determining, by the primary control system based on the sensor data, using the second process a path for the vehicle in the environment, wherein the second process utilizes machine learning to determine the path, and

wherein the primary control system and the fallback control system operate within a multi-layer autonomous vehicle software stack that autonomously controls operation of the vehicle.

2. The method of claim 1 ,

wherein the detecting the imminent collision between the vehicle and the object is further based on the path information.

3. The method of claim 1 , wherein the transmitting the indication of the detected imminent collision is further based on a timer associated with the detected imminent collision.

4. The method of claim 1 , wherein the first process used by the fallback control system to detect the imminent collision is based on heuristic rules.

5. The method of claim 1 , wherein the detecting the imminent collision comprises:

predicting a trajectory of the object based on the sensor data; and

adjusting the trajectory based on the object type of the object.

6. The method of claim 1 , wherein the detecting the imminent collision comprises:

processing the sensor data to generate motion information associated with at least the object that participates in the detected imminent collision; and

computing a trajectory of the object based on the motion information.

7. The method of claim 6 , wherein the processing the sensor data to generate the motion information comprises:

processing the sensor data to generate a series of occupancy grid maps associated with a time window, each including a plurality of grids and an indication of whether a respective grid is occupied, free, or unknown; and

generating the motion information based on the series of occupancy grid maps.

8. One or more non-transitory, computer-readable media encoded with instructions that, when executed by one or more processing units of a vehicle, perform a method comprising:

receiving, by a fallback control system from a primary control system, path information associated with a path for the vehicle, the path based on at least one of a first perception or a first prediction from the primary control system;

collecting sensor data generated by one or more sensors of a vehicle;

detecting, by the fallback control system of the vehicle, based on the sensor data and the received path information, an imminent collision between the vehicle and an object in an environment of the vehicle using at least one of a second perception different from the first perception or a second prediction different from the first prediction,

wherein the detecting of the imminent collision is performed repeatedly,

wherein the second perception or the second prediction includes: predicting a trajectory of the vehicle and a trajectory of the object in the environment of the vehicle is at least partially overlapping, and

classifying the object to determine an object type of the object,

determining, by the fallback control system and based on the object type of the object, whether to suppress a trigger, which indicates the detected imminent collision, from being transmitted to the primary control system;

suppressing transmission of the trigger when the object type of the object corresponds to an assumption that the object is observant of a traffic rule that would result in avoidance of the detected imminent collision between the vehicle and the object;

transmitting to the primary control system the trigger when the object type of the object does not correspond to the assumption that the object is observant of the traffic rule that would result in avoidance of the detected imminent collision between the vehicle and the object;

wherein the transmitting of the trigger is further based on a number of consecutive detected imminent collisions satisfying a threshold number of consecutive detections;

adjusting the threshold number of consecutive detections for determining whether to transmit the trigger based on the object type of the object; and

determining, by the primary control system based on the sensor data, using machine learning, a path for the vehicle in the environment, and

wherein the primary control system and the fallback control system operate within a multi-layer autonomous vehicle software stack that autonomously controls operation of the vehicle.

9. The one or more non-transitory, computer-readable media of claim 8 , wherein:

the at least one of the first perception or the first prediction on which the path is based includes machine learning; and

the at least one of the second perception or the second prediction for detecting the imminent collision is based on rules.

10. The one or more non-transitory, computer-readable media of claim 8 , wherein:

the detecting the imminent collision is performed repeatedly; and

the transmitting the indication of the detected imminent collision is further based on a number of consecutive detected imminent collisions satisfying a threshold.

11. A vehicle comprising:

a hardware processor; and

a set of memory units coupled to the hardware processor, which includes instructions that when performed by the processor cause the vehicle to:

collect sensor data generated by one or more sensors of a vehicle;

detect, by a fallback control system of the vehicle based on the sensor data, an imminent collision between the vehicle and an object in an environment of the vehicle using a first process different from a second process of a primary control system of the vehicle,

wherein the detecting of the imminent collision is performed repeatedly,

wherein the first process and the second process are associated with at least one of a perception or a prediction,

wherein the first process includes: predict a trajectory of the vehicle and a trajectory of the object in the environment of the vehicle is at least partially overlapping, and

classifying the object to determine an object type of the object,

determine, by the fallback control system and based on the object type of the object, whether to suppress a trigger, which indicates the detected imminent collision, from being transmitted to the primary control system;

suppress transmission of the trigger when the object type of the object corresponds to an assumption that the object is observant of a traffic rule that would result in avoidance of the detected imminent collision between the vehicle and the object;

transmit, by the fallback control system to the primary control system, the trigger when the object type of the object does not correspond to the assumption that the object is observant of the traffic rule that would result in avoidance of the detected imminent collision between the vehicle and the object;

wherein the transmitting of the trigger is further based on a number of consecutive detected imminent collisions satisfying a threshold number of consecutive detections;

adjust the threshold number of consecutive detections for determining whether to transmit the trigger based on the object type of the object; and

determine, by the primary control system based on the sensor data, using the second process a path for the vehicle in the environment, wherein the second process utilizes machine learning to determine the path, and

wherein the primary control system and the fallback control system operate within a multi-layer autonomous vehicle software stack that autonomously controls operation of the vehicle.

12. The vehicle of claim 11 , wherein the detecting the imminent collision between the vehicle and the object is further based on the path information.

13. The vehicle of claim 11 , wherein the transmitting the indication of the detected imminent collision is further based on a timer associated with the detected imminent collision.

14. The vehicle of claim 11 , wherein the detecting the imminent collision comprises:

predicting a trajectory of the object based on the sensor data; and

adjusting the trajectory based on the object type of the object.

15. The vehicle of claim 11 , wherein the detecting the imminent collision comprises:

processing the sensor data to generate motion information associated with at least the object that participates in the detected imminent collision; and

computing a trajectory of the object based on the motion information.

16. The vehicle of claim 11 , wherein the processing the sensor data to generate the motion information comprises:

processing the sensor data to generate a series of occupancy grid maps associated with a time window, each including a plurality of grids and an indication of whether a respective grid is occupied, free, or unknown; and

generating the motion information based on the series of occupancy grid maps.

Assignments (1)

Continuity (1)

References Cited (9)