IP Library Granted Patent US 11,821,990
Granted Patent B2
US 11,821,990 · App. 16/677,439 · Granted Nov 21, 2023

Scene perception using coherent doppler LiDAR

Inventors: Xu Chen (Livermore, CA); Tong Lin (Cupertino, CA); Zhenxiang Jian (Sunnyvale, CA); Hiu Hong Yu (Alameda, CA)
Assignee: NIO Technology (Anhui) Co., Ltd.
G01S17/931G01S17/89G06F18/251G06V10/454G06V10/82G06V20/58G06V20/59G06V2201/07
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,821,990
App. No.
16/677,439
Granted
Nov 21, 2023
Kind
B2
Abstract

Embodiments of the present disclosure are directed to providing scene perception display requiring reduced processing capabilities. Sensor data indicative of one or more targets from an imaging and ranging subsystem, location data from a positioning subsystem defining a geographical location of the imaging and ranging subsystem and orientation data from an orientation subsystem defining an orientation of the imaging and ranging subsystem are received. Doppler point cloud data is generated based on Doppler information and point cloud data from the sensor data, the location data and the orientation data. The targets are classified as either a static or dynamic target. Afterwards, the Doppler point cloud data is filtered by removing either the dynamic or the static targets from the Doppler point cloud data. The Doppler point cloud data of either the dynamic or the static targets are further processed and the further processed Doppler point cloud data is rendered.

Claims (55)

1. A scene perception system, comprising:

an imaging and ranging subsystem comprising:

a signal transmitter configured to transmit signals into a region; and

a receiver configured to receive reflected signals generated by reflection of the transmitted signals from one or more targets and to generate receive signals indicative of the one or more targets as sensor data;

a positioning subsystem configured to provide location data defining a geographical location of the imaging and ranging subsystem;

an orientation subsystem configured to provide orientation data defining an orientation of the imaging and ranging subsystem;

a processor configured to:

receive the sensor data, the location data and the orientation data and to process the sensor data, the location data and the orientation data to generate Doppler point cloud data based on Doppler information and point cloud data;

classify the one or more targets as either a static target or a dynamic target;

generate a Doppler point cloud including curved lines where the static targets are represented by points close to or on the curved lines and the dynamic targets are represented by points that lie away from the curved lines; and

filter out the static targets for further processing of the Doppler point cloud data; and

a display subsystem configured to display the further processed Doppler point cloud data.

2. The system of claim 1 , wherein the Doppler information includes a measured Doppler value determined from the imaging and ranging subsystem and a theoretical Doppler value calculated from the imaging and ranging subsystem, the positioning subsystem and the orientation subsystem.

3. The system of claim 2 , wherein the processor is further configured to compare a difference between the measured Doppler value and the theoretical Doppler value with a predetermined threshold value.

4. The system of claim 1 , wherein the positioning subsystem includes a global navigation satellite system (GNSS), including a global positioning system (GPS).

5. The system of claim 1 , wherein the orientation subsystem includes an inertial measurement unit (IMU).

6. The system of claim 1 , wherein the processor is further configured to process Doppler point cloud data representing the static targets separately from Doppler point cloud data representing the dynamic targets.

7. The system of claim 6 , wherein the processor is further configured to process the Doppler point cloud data representing the dynamic targets to generate map data.

8. A method for providing scene perception, the method comprising:

receiving, by a processor, sensor data indicative of one or more targets from an imaging and ranging subsystem, location data from a positioning subsystem defining a geographical location of the imaging and ranging subsystem and orientation data from an orientation subsystem defining an orientation of the imaging and ranging subsystem;

generating, by the processor, Doppler point cloud data based on Doppler information and point cloud data from the sensor data, the location data and the orientation data;

classifying, by the processor, the one or more targets as either a static target or a dynamic target;

generating, by the processor, a Doppler point cloud including curved lines where the static targets are represented by points close to or on the curved lines and the dynamic targets are represented by points that lie away from the curved lines;

filtering, by the processor, the Doppler point cloud data;

removing, by the processor, the static targets from the Doppler point cloud data;

further processing, by the processor, the Doppler point cloud data of the dynamic targets; and

rendering, by the processor, the further processed Doppler point cloud data.

9. The method of claim 8 , wherein generating Doppler point cloud data comprises:

receiving, by the processor, the point cloud data from the sensor data;

determining, by the processor, a measured Doppler value based on the received point cloud data from the sensor data; and

calculating, by the processor, a theoretical Doppler value based on the received point cloud data from the sensor data, the location data and the orientation data.

10. The method of claim 9 , further comprising comparing, by the processor, a difference between the measured Doppler value and the theoretical Doppler value with a predetermined threshold value.

11. The method of claim 8 , further comprising initializing, by the processor, the imaging and ranging subsystem, the positioning subsystem and the orientation subsystem prior to receiving the sensor data, the location data and the orientation data.

12. The method of claim 8 , wherein the location data is determined by a global navigation satellite system (GNSS), such as a global positioning system (GPS).

13. The method of claim 8 , wherein the orientation data is determined by an inertial measurement unit (IMU).

14. The method of claim 8 , further comprising further processing, by the processor, Doppler point cloud data representing the static targets separately from Doppler point cloud data representing the dynamic targets.

15. The method of claim 14 , further comprising generating, by the processor, map data from the further processed Doppler point cloud data representing dynamic targets.

16. A vehicle control system, comprising:

a processor; and

a memory coupled with and readable by the processor and storing therein a set of instructions, which when executed by the processor, cause the processor to provide a scene perception display by:

receiving sensor data indicative of one or more targets from an imaging and ranging subsystem, location data from a positioning subsystem defining a geographical location of the imaging and ranging subsystem and orientation data from an orientation subsystem defining an orientation of the imaging and ranging subsystem;

generating Doppler point cloud data based on Doppler information and point cloud data from sensor data, the location data and the orientation data;

classifying the one or more targets as either a static target or a dynamic target;

generating a Doppler point cloud including curved lines where the static targets are represented by points close to or on the curved lines and the dynamic targets are represented by points that lie away from the curved lines;

filtering the Doppler point cloud data;

removing the static targets from the Doppler point cloud data;

further processing the Doppler point cloud data of the dynamic targets; and

rendering the further processed Doppler point cloud data.

17. The system of claim 16 , wherein generating Doppler point cloud data comprises:

receiving the point cloud data from the sensor data;

determining a measured Doppler value based on the received point cloud data from the sensor data; and

calculating a theoretical Doppler value based on the received point cloud data from the sensor data, the location data and the orientation data.

18. The system of claim 17 , wherein generating Doppler point cloud data comprises comparing a difference between the measured Doppler value and the theoretical Doppler value with a predetermined threshold value.

19. The system of claim 16 , wherein the imaging and ranging subsystem, the positioning subsystem and the orientation subsystem are initialized prior to receiving the sensor data, the location data and the orientation data.

20. The system of claim 16 , wherein the location data is determined by a global navigation satellite system (GNSS), such as a global positioning system (GPS).

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 24, 2022
From: NIO USA, INC.
To: NIO TECHNOLOGY (ANHUI) CO., LTD.
Reel/Frame 060171/0724 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 30, 2019
From: CHEN, XU; LIN, TONG; JIAN, ZHENXIANG; YU, HIU HONG
To: NIO USA, INC.
Reel/Frame 051389/0417 →
Continuity (1)
Related Publication 20210141092A1 · May 13, 2021