Granted Patent B2

US 12,243,327 · App. 18/054,844 · Granted Mar 4, 2025

Inventor: Gil Arditi (Palo Alto, CA)

Assignee: Lyft, Inc.

G06V20/58G01C21/32G01C21/3841G01C21/3878G05D1/0274G06F16/29G06N3/08G06N20/00

Abstract

Claims (64)

1. A method comprising, by a computing system:

collecting, by the computing system of a first vehicle, first sensor data, of a particular geographic location, from a first type of sensor of the first vehicle associated with a fleet of vehicles;

processing, by the computing system of the first vehicle, the first sensor data to identify one or more first objects at the particular geographic location;

accessing, by the computing system of the first vehicle, an existing high-definition (HD) map associated with the particular geographic location;

determining, by the computing system of the first vehicle, whether the one or more first objects are included in the existing HD map; and

in response to determining that the one or more first objects are not included in the existing HD map, sending, by the computing system of the first vehicle, the first sensor data to a server to transform the first sensor data into a latent representation, wherein transforming the first sensor data into the latent representation comprises:

encoding, using a trained machine-learning model, the first sensor data collected from the first type of sensor of the first vehicle into the latent representation in a common data space to minimize discrepancies between the first sensor data collected from the first type of sensor of the first vehicle and second sensor data collected from a second type of sensor of a second vehicle;

receiving, by the computing system of the first vehicle, the latent representation of the first sensor data from the server;

decoding, by the computing system of the first vehicle, the latent representation to generate map data corresponding to the first sensor data; and

associating, by the computing system of the first vehicle, the map data corresponding to the first sensor data with the existing HD map to generate an updated HD map.

2. The method of claim 1 , wherein:

the first sensor data comprises camera data and the first type of sensor is a camera sensor; and

the second sensor data comprises light detection and ranging (LiDAR) data and the second type of sensor is a LiDAR sensor.

3. The method of claim 1 , wherein the first type of sensor and the second type of sensor are same.

4. The method of claim 1 , wherein the first type of sensor and the second type of sensor are different.

5. The method of claim 1 , further comprising:

determining that the one or more first objects are inanimate or stationary objects.

6. The method of claim 5 , wherein determining that the one or more first objects are inanimate or stationary objects comprises:

receiving additional sensor data, of the particular geographic location, from other vehicles associated with the fleet of vehicles at different times within a certain time frame; and

determining that the one or more first objects identified at the particular geographic location are consistent across the first sensor data associated with the first vehicle, the second sensor data associated with the second vehicle, and the additional sensor data associated with the other vehicles of the fleet of vehicles.

7. The method of claim 1 , wherein the trained machine-learning model comprises a convolutional neural network and a deconvolutional neural network, wherein an output of the convolutional neural network is configured to be an input of the deconvolutional neural network.

8. The method of claim 7 , wherein:

the encoding the first sensor data into the latent representation is performed using the convolutional neural network; and

the decoding the latent representation is performed using the deconvolutional neural network.

9. The method of claim 1 , wherein processing the first sensor data to identify the one or more first objects at the particular geographic location comprises:

detecting a plurality of objects based at least in part on the first sensor data;

classifying, using an object classifier, the plurality of objects into different classification types; and

determining, based on the different classification types, stationary objects from the plurality of objects, wherein the one or more first objects are the stationary objects on a road.

10. The method of claim 1 , wherein determining whether the one or more first objects are included in the existing HD map comprises:

generating, for each first object of the one or more first objects, a confidence score representing a likelihood of the first object being included in the existing HD map; and

comparing, for each first object of the one or more first objects, the confidence score with a threshold.

11. The method of claim 10 , wherein confidence score generated for the first object is based on one or more of a similarity comparison of a measured size, dimensions, classification, or location of the first object with known objects in the existing HD map.

12. The method of claim 1 , wherein determining whether the one or more first objects are included in the existing HD map comprises:

determining whether there is a mismatch between the first sensor data of the particular geographic location and the existing HD map associated with particular geographic location based on comparing map data corresponding to the first sensor data with corresponding map data in the existing HD map.

13. A system, comprising:

one or more processors and one or more computer-readable non-transitory storage media in communication with the one or more processors, the one or more computer-readable non-transitory storage media comprising instructions operable when executed by the one or more processors to cause the system to perform operations comprising:

collecting, by a computing system of a first vehicle, first sensor data, of a particular geographic location, from a first type of sensor of the first vehicle associated with a fleet of vehicles;

processing, by the computing system of the first vehicle, the first sensor data to identify one or more first objects at the particular geographic location;

accessing, by the computing system of the first vehicle, an existing high-definition (HD) map associated with the particular geographic location;

determining, by the computing system of the first vehicle, whether the one or more first objects are included in the existing HD map; and

in response to determining that the one or more first objects are not included in the existing HD map, sending, by the computing system of the first vehicle, the first sensor data to a server to transform the first sensor data into a latent representation, wherein transforming the first sensor data into the latent representation comprises:

encoding, using a trained machine-learning model, the first sensor data collected from the first type of sensor of the first vehicle into the latent representation in a common data space to minimize discrepancies between the first sensor data collected from the first type of sensor of the first vehicle and second sensor data collected from a second type of sensor of a second vehicle;

receiving, by the computing system of the first vehicle, the latent representation of the first sensor data from the server;

decoding, by the computing system of the first vehicle, the latent representation to generate map data corresponding to the first sensor data; and

associating, by the computing system of the first vehicle, the map data corresponding to the first sensor data with the existing HD map to generate an updated HD map.

14. The system of claim 13 , wherein:

the first sensor data comprises camera data and the first type of sensor is a camera sensor; and

the second sensor data comprises light detection and ranging (LiDAR) data and the second type of sensor is a LiDAR sensor.

15. The system of claim 13 , wherein the first type of sensor and the second type of sensor are different.

16. The system of claim 13 , wherein the first type of sensor and the second type of sensor are same.

17. One or more computer-readable non-transitory storage media including instructions that are operable when executed to cause one or more processors to perform operations comprising:

collecting, by a computing system of a first vehicle, first sensor data, of a particular geographic location, from a first type of sensor of the first vehicle associated with a fleet of vehicles;

processing, by the computing system of the first vehicle, the first sensor data to identify one or more first objects at the particular geographic location;

accessing, by the computing system of the first vehicle, an existing high-definition (HD) map associated with the particular geographic location;

determining, by the computing system of the first vehicle, whether the one or more first objects are included in the existing HD map; and

in response to determining that the one or more first objects are not included in the existing HD map, sending, by the computing system of the first vehicle, the first sensor data to a server to transform the first sensor data into a latent representation, wherein transforming the first sensor data into the latent representation comprises:

encoding, using a trained machine-learning model, the first sensor data collected from the first type of sensor of the first vehicle into the latent representation in a common data space to minimize discrepancies between the first sensor data collected from the first type of sensor of the first vehicle and second sensor data collected from a second type of sensor of a second vehicle;

decoding, by the computing system of the first vehicle, the latent representation to generate map data corresponding to the first sensor data; and

associating, by the computing system of the first vehicle, the map data corresponding to the first sensor data with the existing HD map to generate an updated HD map.

18. The one or more computer-readable non-transitory storage media of claim 17 , wherein:

the first sensor data comprises camera data and the first type of sensor is a camera sensor; and

the second sensor data comprises light detection and ranging (LiDAR) data and the second type of sensor is a LiDAR sensor.

19. The one or more computer-readable non-transitory storage media of claim 17 , wherein the first type of sensor and the second type of sensor are different.

20. The one or more computer-readable non-transitory storage media of claim 17 , wherein the first type of sensor and the second type of sensor are same.

Continuity (2)

References Cited (9)

Cited By (1)

US 12,662,137