IP Library Granted Patent US 10,444,365
Granted Patent B1
US 10,444,365 · App. 16/166,950 · Granted Oct 15, 2019

Object detection based on Lidar intensity

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,444,365
App. No.
16/166,950
Granted
Oct 15, 2019
Kind
B1
Abstract

Aspects of the present disclosure involve systems, methods, and devices for determining reflectance properties of objects based on Lidar intensity values, A system includes one or more processors of a machine and a machine-storage medium storing instructions that, when executed by the one or more processors, cause the machine to perform operations comprising accessing an incoming data point output by a Lidar unit during operation of a vehicle. The operations may further include inferring, using a reflectance inference model, a reflectance value of an object based on the incoming data point. The reflectance inference model comprises a mapping of previously collected data points to a coordinate system using associated range values and raw intensity values. The operations may further include determining one or more characteristics of the object based on the inferred reflectance value.

Claims (69)

1. An autonomous vehicle (AV) computing system comprising:

a reflectance inference system comprising one or more processors, the reflectance inference system to perform operations comprising:

accessing an incoming data point output by a light detection and ranging (Lidar) unit during operation of the AV computing system;

determining, using a reflectance inference model generated by a Lidar calibration system, an inferred reflectance value of an object based on the incoming data point, the reflectance inference model comprising a mapping of a set of previously collected data points to a coordinate system using range and raw intensity values of the previously collected data points as coordinates, the inferred reflectance value being interpolated from at least two previously collected data points of the set of previously collected data points mapped to the coordinate system, each of the two previously collected data points corresponding to one of multiple known reflectance values; and

determining one or more characteristics of the object based on the inferred reflectance value; and

a motion planning system comprising one or more processors, the motion planning system to determine a motion plan for an AV based on the one or more characteristics of the object, the motion plan being used to control motion of the AV.

2. The AV computing system of claim 1 , wherein the determining of the inferred reflectance value of the object comprises:

mapping the incoming data point to the coordinate system;

identifying two or more nearest neighbors of the incoming data point mapped to the coordinate system, the two or more nearest neighbors including the at least two previously collected data points; and

interpolating the reflectance value from the two or more nearest neighbors.

3. The AV computing system of claim 1 , wherein:

the reflectance inference model comprises a look-up table comprising a plurality of reflectance values interpolated from the set of previously collected data points mapped to the coordinate system, each reflectance value corresponding to a particular combination of range and raw intensity values; and

the determining of the inferred reflectance value of the object comprises:

accessing the look-up table; and

determining the inferred reflectance value from the look-up table based on a raw intensity value and a range value of the incoming data point.

4. The AV computing system of claim 1 , wherein:

the Lidar unit comprises a plurality of channels, each channel being capable of operating at a plurality of power levels;

the reflectance inference model comprises multiple mappings of the previously collected data points to the coordinate system, each of the mappings corresponding to a particular power level of a particular channel of the Lidar unit; and

the determining of the inferred reflectance value of the object comprises selecting the mapping of the set of previously collected data points from the multiple mappings based on a channel of the Lidar unit that output the incoming data point and a power level of the channel.

5. The AV computing system of claim 1 , wherein the operations further comprise generating state data that describes the object, the state data comprising at least the inferred reflectance value.

6. The AV computing system of claim 1 , wherein the operations further comprise:

generating a reflectance map that includes an indication of the inferred reflectance value of the object; and

performing vehicle localization based on the reflectance map.

7. The AV computing system of claim 1 , wherein the operations further comprise:

collecting a data set comprising data points output by the Lidar unit, each of the data points corresponding to one of multiple targets, each target having one of the multiple known reflectance values, the data set corresponding to the set of previously collected data points; and

generating the reflectance inference model for the Lidar unit based on the data set, the generating of the reflectance inference model comprising mapping the set of previously collected data points to the coordinate system.

8. A light detection and ranging (Lidar) calibration system comprising:

one or more processors of a machine; and

a machine-storage medium storing instructions that, when executed by the one or more processors, cause the machine to perform operations comprising:

collecting a data set comprising a plurality of data points output by a Lidar unit, each of the data points corresponding to one of multiple targets, each target having a known reflectance value, each channel of the Lidar unit being capable of operating at a plurality of power levels, the collecting of the data set including collecting data points output by each channel at each power level; and

generating a reflectance inference model for the Lidar unit based on the data set, the generating of the reflectance inference model comprising mapping the plurality of data points to a coordinate system using range and raw intensity values of each data point as coordinates, the reflectance inference model operable to infer reflectance values from incoming data points output by the Lidar unit, the generating of the reflectance inference model includes generating multiple mappings of the plurality of data points to the coordinate system, each mapping corresponding to a particular power level of a particular channel.

9. The Lidar calibration system of claim 8 , wherein:

the generating of the reflectance inference model further comprises generating a look-up table comprising a plurality of reflectance values interpolated from the plurality of data points mapped to the coordinate system,

each reflectance value corresponding to a particular combination of range and raw intensity values.

10. The Lidar calibration system of claim 9 , wherein the operations further comprise:

compressing the look-up table to generate a compressed look-up table; and

storing the compressed look-up table in a memory device of a vehicle computing system.

11. A method comprising:

collecting, by a Lidar calibration system comprising at least a first hardware processor, a data set comprising a plurality of data points output by a light detection and ranging (Lidar) unit of an autonomous vehicle, each of the data points corresponding to one of multiple targets, each target having a known reflectance value;

generating, by the Lidar calibration system, a reflectance inference model for the Lidar unit based on the data set, the generating of the reflectance inference model comprising mapping the plurality of data points to a coordinate system using range and raw intensity values of each data point as coordinates;

accessing, by a reflectance inference system of a vehicle computing system comprising at least a second hardware processor, an incoming data point output by the Lidar unit during operation of the autonomous vehicle;

determining, by the reflectance inference system and using the reflectance inference model, an inferred reflectance value of an object based on the incoming data point, the inferred reflectance value being interpolated from at least two data points mapped to the coordinate system; and

determining, by a motion planning system of the vehicle computing system, a motion plan for the autonomous vehicle based on the inferred reflectance value, the motion plan of the autonomous vehicle being used to control motion of the autonomous vehicle.

12. The method of claim 11 , wherein the determining of the inferred reflectance value of the object comprises:

mapping the incoming data point to the coordinate system;

identifying two or more nearest neighbors of the incoming data point mapped to the coordinate system, the two or more nearest neighbors including the at least two data points; and

interpolating the inferred reflectance value from the two or more nearest neighbors.

13. The method of claim 11 , wherein:

the generating of the reflectance inference model further comprises:

generating a look-up table comprising a plurality of reflectance values interpolated from the plurality of data points mapped to the coordinate system, each reflectance value corresponding to a particular combination of range and raw intensity values; and

the determining of the inferred reflectance value of the object comprises:

accessing, by the vehicle computing system, the look-up table from a memory device of the vehicle computing system; and

determining, by the vehicle computing system, the inferred reflectance value from the look-up table based on a range value and a raw intensity value of the incoming data point.

14. The method of claim 11 , wherein:

the collecting of the data set includes collecting data points output by each channel of the Lidar unit; and

the generating of the reflectance inference model includes generating, for each channel, a mapping of data points to the coordinate system.

15. The method of claim 14 ,

each channel of the Lidar unit is capable of operating at a plurality of power levels;

the collecting of the data points output by each channel of the Lidar unit includes collecting data points output by each channel at each power level; and

the generating of the reflectance inference model includes generating multiple mappings of the data points to the coordinate system, each mapping corresponding to a particular power level of a particular channel.

16. The method of claim 15 , wherein:

the determining of the inferred reflectance value of the object includes selecting a mapping from the multiple mappings based on a channel of the Lidar unit that output the incoming data point and a power level of the channel.

17. The method of claim 11 , wherein the collecting of the data set comprises:

arranging a plurality of targets, each of the targets having a known reflectance; and

collecting at least one data point corresponding to each target.

18. The method of claim 17 , wherein the operations further comprise:

mounting the Lidar unit on a gimbal; and

using the gimbal to position the Lidar unit at multiple angles; and

wherein the collecting of the at least one data point corresponding to each target includes collecting at least one data point corresponding to each target at each angle of the multiple angles.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 2, 2024
From: UATC, LLC
To: AURORA OPERATIONS, INC.
Reel/Frame 066973/0513 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 11, 2019
From: UBER TECHNOLOGIES, INC.
To: UATC, LLC
Reel/Frame 050348/0743 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 22, 2018
From: JAIN, SHANTANU; YANG, GEHUA
To: UBER TECHNOLOGIES, INC.
Reel/Frame 047263/0808 →