Granted Patent B1

US 11,526,721 · App. 16/798,073 · Granted Dec 13, 2022

Inventor: Bryan Matthew O'Malley (Oviedo, FL)

Assignee: Zoox, Inc.

G06N3/006B60W50/04G06N5/04G06N20/00B60W2050/0031G05D1/0088

Abstract

Claims (118)

1. A system comprising:

one or more processors; and

one or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause the one or more processors to perform operations comprising:

receiving log data based at least in part on sensor data captured by a sensor of an autonomous vehicle traversing an environment, the log data including:

a first attribute value for an object represented in the log data, the first attribute value associated with a first time;

a first confidence value associated with the first attribute value;

a second attribute value for the object represented in the log data, the second attribute value associated with a second time different from the first time; and

a second confidence value associated with the second attribute value;

determining a first distance between a first location associated with the object and a second location associated with the sensor at the first time;

determining a second distance between a third location associated with the object and a fourth location associated with the sensor at the second time;

determining, based at least in part on the first confidence value and the first distance, a first weight associated with the first attribute value;

determining, based at least in part on the second confidence value and the second distance, a second weight associated with the second attribute value;

determining, based at least in part on the first attribute value, the first weight, the second attribute value, and the second weight, a simulated object attribute for a simulated object;

generating a simulated scenario comprising a simulated environment;

instantiating, based at least in part on the simulated object attribute, the simulated object in the simulated environment; and

generating simulation data comprising data generated by an autonomous controller in response to the simulated scenario.

2. The system of claim 1 , wherein the first attribute value is a first object classification, wherein the second object attribute value is a second object classification that is different than the first object classification, the operations further comprising:

comparing the first weight and the second weight; and

determining, based at least in part on the first weight being greater than the second weight, the first object classification as the simulated object attribute.

3. The system of claim 1 , wherein the first attribute value is a first numerical value, the second attribute value is second numerical value that is different than the first numerical value, and

wherein determining the simulated object attribute for the simulated object comprises determining a third numerical value based at least in part on the first numerical value, the first weight, the second numerical value, and the second weight.

4. The system of claim 1 , wherein the first attribute value is associated with a changeable attribute type, and wherein instantiating the simulated object in the simulated environment comprises:

determining a first simulation time in the simulated scenario at which the simulated object is instantiated;

determining, based on the log data, a first observation time associated with the first simulation time in the simulated scenario;

determining a previous attribute value for the object represented in the log data, wherein the previous attribute value is associated with a second observation time before the first observation time;

determining a third distance associated with the previous attribute value;

determining a subsequent attribute value for the object represented in the log data, wherein the subsequent attribute value is associated with a third observation time after the first observation time; and

determining a fourth distance associated with the subsequent attribute value,

wherein the simulated object attribute for the simulated object is determined based at least in part on the previous attribute value for the object, the third distance, the subsequent attribute value for the object, and the fourth distance.

5. The system of claim 1 , wherein the first attribute value is associated with an attribute type indicating at least one of:

a size of the object,

a classification type of the object,

a pose of the object,

a trajectory of the object,

a velocity of the object,

a state of the object, or

a position of the object.

6. A method comprising:

receiving log data that is based at least in part on sensor data captured by a sensor in an environment, the log data including:

a first attribute value for an attribute of an object represented in the log data at a first time; and

a first confidence value associated with the first attribute value;

determining a first distance between a first location associated with the object represented in the log data and a second location associated with the sensor at the first time;

determining a simulated object attribute for a simulated object, based at least in part on the first attribute value, data and the first confidence value, and the first distance; and

executing a simulation based at least in part on the log data, wherein executing the simulation includes controlling the simulated object based at least in part on the simulated object attribute.

7. The method of claim 6 , wherein the log data further includes:

a second attribute value for the object represented in the log data, wherein the second attribute value is associated with a second time different from the first time; and

a second confidence value associated with the second attribute value;

wherein the method further comprises determining a second distance between a third location associated with the object and a fourth location associated with the sensor at the second time; and

wherein determining the simulated object attribute for the simulated object is further based at least in part on the second attribute value, the second confidence value, and the second distance.

8. The method of claim 7 , wherein the first attribute value is a first object classification, wherein the second attribute value is second object classification that is different than the first object classification, the method further comprising:

determining a first weight based at least in part on the first confidence value and the first distance;

determining a second weight based at least in part on the second confidence value and the second distance;

comparing the first weight and the second weight; and

determining, based at least in part on the first weight being greater than the second weight, the first object classification as the simulated object attribute.

9. The method of claim 7 , wherein the first attribute value is a first numerical value, the second attribute value is second numerical value that is different than the first numerical value, and wherein determining the simulated object attribute for the simulated object comprises:

determining a first weight based at least in part on the first confidence value and the first distance;

determining a second weight based at least in part on the second confidence value and the second distance; and

determining a third numerical value based at least in part on the first numerical value, the first weight, the second numerical value, and the second weight.

10. The method of claim 6 , wherein the first attribute value is associated with an attribute type indicating at least one of:

a size of the object,

a classification type of the object,

a pose of the object,

a trajectory of the object,

a velocity of the object,

a state of the object, or

a position of the object.

11. The method of claim 6 , wherein executing the simulation comprises:

generating a simulated environment;

instantiating a simulated autonomous vehicle in the simulated environment; and

instantiating, based at least in part on the simulated object attribute, the simulated object in the simulated environment.

12. The method of claim 11 , further comprising:

determining, based at least in part on the log data, a trigger region in the simulated environment associated with the simulated object,

wherein instantiating the simulated object is based at least in part on the trigger region and a position of the simulated autonomous vehicle within the simulated environment.

13. The method of claim 11 , wherein the first attribute value is associated with a changeable attribute type, and wherein instantiating the simulated object in the simulated environment comprises:

determining a first simulation time in the simulated environment at which the simulated object is instantiated;

determining, based on the log data, a first observation time associated with the first simulation time in the simulated environment;

determining a previous attribute value for the object represented in the log data, wherein the previous attribute value is associated with a second observation time before the first observation time;

determining a second distance associated with the previous attribute value;

determining a subsequent attribute value for the object represented in the log data, wherein the subsequent attribute value is associated with a third observation time after the first observation time; and

determining a third distance associated with the subsequent attribute value,

wherein the simulated object attribute for the simulated object is determined based at least in part on the previous attribute value, the second distance, the subsequent attribute value, and the third distance.

14. One or more non-transitory computer-readable media storing instructions executable by one or more processors, wherein the instructions, when executed, cause the one or more processors to perform operations comprising:

receiving log data that is based at least in part on sensor data captured by a sensor in an environment, the log data including;

a first attribute value for an attribute of an object represented in the log data at a first time; and

a first confidence value associated with the first attribute value;

determining a first distance between a first location associated with the object represented in the log data and a second location associated with the sensor at the first time;

determining a simulated object attribute for a simulated object, based at least in part on the first attribute value, the first confidence value, and the first distance; and

executing a simulation based at least in part on the log data, wherein executing the simulation includes controlling the simulated object based at least in part on the simulated object attribute.

15. The one or more non-transitory computer-readable media of claim 14 , the operations further comprising:

determining a weight value associated with the first attribute value,

wherein the weight value associated with the first attribute value is determined based at least in part on the first distance and the first confidence value.

16. The one or more non-transitory computer-readable media of claim 14 ,

wherein the log data further includes:

a second attribute value for the object represented in the log data, wherein the second attribute value is associated with a second time different from the first time; and

a second confidence value associated with the second attribute value

wherein the operations further comprise determining a second distance between a third location associated with the object and a fourth location associated with the sensor at the second time; and

wherein determining the simulated object attribute for the simulated object is further based at least in part on the second attribute value, the second confidence value, and the second distance.

17. The one or more non-transitory computer-readable media of claim 16 , wherein the first attribute value is a first object classification, wherein the second attribute value is second object classification that is different than the first object classification, the operations further comprising:

determining a first weight based at least in part on the first confidence value and the first distance;

determining a second weight based at least in part on the second confidence value and the second distance;

comparing the first weight and the second weight; and

determining, based at least in part on the first weight being greater than the second weight, the first object classification as the simulated object attribute.

18. The one or more non-transitory computer-readable media of claim 16 , wherein the first attribute value is a first numerical value, the second attribute value is second numerical value that is different than the first numerical value, and wherein determining the simulated object attribute for the simulated object comprises:

determining a first weight based at least in part on the first confidence value and the first distance;

determining a second weight based at least in part on the second confidence value and the second distance; and

determining a third numerical value based at least in part on the first numerical value, the first weight, the second numerical value, and the second weight.

19. The one or more non-transitory computer-readable media of claim 14 , wherein executing the simulation comprises:

generating a simulated environment;

instantiating a simulated autonomous vehicle in the simulated environment; and

instantiating, based at least in part on the simulated object attribute, the simulated object in the simulated environment.

20. The one or more non-transitory computer-readable media of claim 19 , wherein the first attribute value is associated with a changeable attribute type, and wherein instantiating the simulated object in the simulated environment comprises:

determining a first simulation time in the simulated environment at which the simulated object is instantiated;

determining, based on the log data, a first observation time associated with the first simulation time in the simulated environment;

determining a previous attribute value for the object represented in the log data, wherein the previous attribute value is associated with a second observation time before the first observation time;

determining a second distance associated with the previous attribute value;

determining a subsequent attribute value for the object represented in the log data, wherein the subsequent attribute value is associated with a third observation time after the first observation time; and

determining a third distance associated with the subsequent attribute value,

wherein the simulated object attribute for the simulated object is determined based at least in part on the previous attribute value, the second distance, the subsequent attribute value, and the third distance.

Assignments (1)

Cited By (10)

US 1,087,150 US 12,462,194 US 12,498,251 US 12,509,066 US 12,518,215 US 12,530,625 US 12,536,473 US 12,541,718 US 12,608,655 US 12,703,366