Granted Patent B1

US 12,269,462 · App. 16/820,378 · Granted Apr 8, 2025

Inventors: Gowtham Garimella (Burlingame, CA); Marin Kobilarov (Mountain View, CA); Kai Zhenyu Wang (Foster City, CA)

Assignee: Zoox, Inc.

B60W30/06B62D15/0285G06N3/08

Abstract

Claims (84)

1. A method comprising:

receiving sensor data associated with an environment of a first vehicle;

determining, based at least in part on the sensor data, a type of intent of a second vehicle proximate the first vehicle, wherein the intent is associated with a maneuver to be performed by the second vehicle during a future time, the maneuver comprising a plurality of trajectories within a first region in the environment;

determining, based at least in part on the intent, an entirety of time that the second vehicle performs the maneuver;

determining that the type of intent satisfies a temporal condition;

determining, based at least in part on the sensor data and the type of intent satisfying the temporal condition, a second region of the environment around and encompassing the second vehicle, wherein the second region includes areas in the environment that are likely to be occupied by the second vehicle for the entirety of time and within the first region;

determining, based at least in part on the sensor data and the second region of the environment, a blocked region corresponding to the first region that the first vehicle should not enter; and

determining an instruction for controlling the first vehicle based at least in part on the blocked region.

2. The method as claim 1 recites, further comprising:

generating, based at least in part on the sensor data, a multi-channel image comprising a top-down representation of the environment, a channel of the multi-channel image being associated with a feature of the second vehicle or the environment; and

determining the second region of the environment around the second vehicle based at least in part on analyzing the multi-channel image using a convolutional neural network.

3. The method as claim 1 recites, wherein the instruction for controlling the first vehicle comprises an instruction to cause the first vehicle to at least one of:

decelerate;

stop;

perform a lane change; or

send a signal to a remote operator.

4. The method as claim 1 recites, wherein determining the second region of the environment around the second vehicle is performed at a first time, the method further comprising:

receiving, at a second time, updated sensor data;

determining, based at least in part on the updated sensor data, an updated second region of the environment around the second vehicle that is to be occupied by the second vehicle; and

determining an updated instruction for controlling the first vehicle based at least in part on the updated second region of the environment.

5. A system comprising:

one or more processors; and

one or more non-transitory computer-readable media storing instructions, that when executed by the one or more processors, cause the system to perform operations comprising:

receiving sensor data associated with an environment of a vehicle;

determining, based at least in part on the sensor data, a type of action performed at a future time by an object proximate the vehicle;

determining, based at least in part on the action, an entirety of time that the object performs the action;

determining that the type of action satisfies a temporal condition;

receiving, based at least in part on the sensor data and the type of action satisfying the temporal condition, a first region of the environment encompassing the object and that the object proximate the vehicle is likely to occupy at the future time for the entirety of time that the object performs the action, the action comprising a plurality of trajectories within the first region;

determining, based at least in part on the sensor data and the first region of the environment, a blocked region corresponding to the first region that the vehicle should not enter; and

determining an instruction for controlling the vehicle based at least in part on the blocked region.

6. The system as claim 5 recites, the operations further comprising:

determining, based at least in part on the sensor data, a confidence score associated with an intent of the object; and

determining that the confidence score meets or exceeds a threshold, wherein the first region of the environment is determined based at least in part on the intent, and wherein the action is associated with the intent.

7. The system as claim 5 recites, the operations further comprising:

determining, based at least in part on the sensor data, a plurality of intents of the object, wherein a first intent of the plurality of intents is associated with a first confidence score and a second intent of the plurality of intents is associated with a second confidence score;

determining that the first confidence score and the second confidence score meet or exceed a threshold;

determining a third region of the environment associated with the object that is to be occupied by the object while the object performs a first action associated with the first intent; and

determining a fourth region of the environment associated with the object that is to be occupied by the object while the object performs a second action associated with the second intent, wherein the first region is determined based on at least one of the third region or the fourth region.

8. The system as claim 5 recites, the operations further comprising:

generating, based at least in part on the sensor data, an input comprising a top-down representation of the environment, wherein the top-down representation comprises a multi-channel image;

inputting the input into a convolutional neural network; and

receiving the first region of the environment associated with the object from the convolutional neural network.

9. The system as claim 8 recites, the operations further comprising selecting the convolutional neural network based at least in part on the action.

10. The system as claim 5 recites, the operations further comprising determining an occupancy grid, wherein a tile associated with the occupancy grid is associated with a confidence score indicative of whether a portion of the environment is likely to be occupied by the object, and wherein the first region of the environment is defined based at least in part on the occupancy grid.

11. The system as claim 10 recites, the operations further comprising inputting the occupancy grid into a planner component associated with the vehicle to determine the instruction for controlling the vehicle.

12. The system as claim 5 recites, wherein the vehicle is an autonomous vehicle and wherein the instruction for controlling the vehicle comprises an instruction to cause the vehicle to at least one of:

decelerate;

stop;

perform a lane change; or

send a signal to a remote operator.

13. The system as claim 5 recites, wherein determining the first region of the environment associated with the object is performed at a first time, the operations further comprising:

receiving, at a second time, updated sensor data;

determining, based at least in part on the updated sensor data and an associated intent, an updated first region of the environment associated with the object that is to be occupied by the object; and

determining an updated instruction for controlling the vehicle based at least in part on the updated first region of the environment.

14. One or more non-transitory computer-readable media storing instructions, that when executed by one or more processors, cause the one or more processors to perform operations comprising:

receiving sensor data associated with an environment of a vehicle;

determining, based at least in part on the sensor data, an action performed at a future time by an object proximate the vehicle, wherein the action is a parallel parking maneuver;

determining, based at least in part on the action, an entirety of time that the object performs the action;

receiving, based at least in part on the sensor data, a first region of the environment encompassing the object and that the object proximate the vehicle is likely to occupy at the future time for the entirety of time that the object performs the action, the action comprising a plurality of trajectories within the first region;

determining, based at least in part on the sensor data and the first region of the environment, a blocked region corresponding to the first region that the vehicle should not enter; and

determining an instruction for controlling the vehicle based at least in part on the blocked region.

15. The one or more non-transitory computer-readable media as claim 14 recites, the operations further comprising:

generating, based at least in part on the sensor data, an input comprising a top-down representation of the environment, wherein the top-down representation comprises a multi-channel image;

inputting the input into a convolutional neural network, wherein the convolutional neural network is selected based at least in part on the action; and

receiving the first region of the environment associated with the object from the convolutional neural network.

16. The one or more non-transitory computer-readable media as claim 15 recites, wherein the convolutional neural network is trained based at least in part on:

receiving log data;

detecting, based at least in part on the log data, a second object that performs the action;

determining a start time associated with the action and a stop time associated with the action;

determining a third region of the environment occupied by the second object during a period of time between the start time and the stop time; and

training the convolutional neural network based at least in part on the third region of the environment.

17. The one or more non-transitory computer-readable media as claim 14 recites, wherein the first region of the environment is determined based at least in part on the action and at least one other action, and wherein each action is associated with an intent that is associated with a confidence score that meets or exceeds a threshold.

18. The one or more non-transitory computer-readable media as claim 14 recites, the operations further comprising determining an occupancy grid, wherein a tile associated with the occupancy grid is associated with a confidence score indicative of whether a portion of the environment is likely to be occupied by the object, and wherein the first region of the environment is defined based at least in part on the occupancy grid.

19. The method of claim 1 , wherein:

the plurality of trajectories includes at least one trajectory for a current time and at least one subsequent trajectory for the future time, and

the instruction for controlling the first vehicle includes determining a trajectory for the first vehicle at the current time that avoids the blocked region.

20. The method of claim 1 , wherein:

the maneuver includes at least one of:

parallel parking,

a three-point turn,

a U-turn,

a K-turn, or

an N-point turn, and

the plurality of trajectories occur within the first region.

Assignments (1)

References Cited (25)

Cited By (5)

US 12,352,857 US 12,466,394 US 12,472,984 US 12,565,192 US 12,637,102