IP Library › Granted Patent US 12,643,569
Granted Patent B2
US 12,643,569 · App. 18/896,342 · Granted Jun 2, 2026

Vehicle system lidar fog detection and compensation

Inventor: Arvind Srivastav (San Francisco, CA)
Assignee: Zoox, Inc.
B60W60/0011B60W60/0015G01S7/497G01S13/931G01S17/89G01S17/931B60W2420/408B60W2554/20B60W2554/40B60W2555/20B60W2556/25
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,643,569
App. No.
18/896,342
Granted
Jun 2, 2026
Kind
B2
Abstract

Techniques for detecting and determining a density for a non-impeding object based on correlation between radar and lidar returns are described herein. The techniques provide for receiving lidar data and radar data from a vehicle system operating in an environment. Portions of the lidar data and radar data are determined based on being associated with moving objects in the environment. The portions are then correlated to determine a similarity between the radar and lidar data. The similarity may then be used to determine an indication and density of the non-impeding object in the environment and cause the vehicle to operate within the environment accordingly.

Claims (72)

1 . A system comprising:

one or more processors; and

one or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause the system to perform operations comprising:

receiving radar data of an environment;

receiving lidar data of the environment;

determining, based at least in part on comparing the radar data and the lidar data, a presence of a non-impeding object in the environment;

determining, based at least in part on the presence of the non-impeding object, an effective range for a lidar sensor; and

controlling a vehicle based at least in part on the effective range.

2 . The system of claim 1 , wherein controlling the vehicle is further based at least in part on:

determining, based at least in part on the presence of the non-impeding object, a subset of the lidar data associated with the non-impeding object; and

controlling the vehicle based at least in part on the subset of the lidar data.

3 . The system of claim 1 , wherein determining the presence of the non-impeding object is further based at least in part on:

determining a first portion of the radar data that is associated with a moving object;

determining a first portion of the lidar data that is associated with the moving object; and

determining a similarity between the first portion of the radar data and the first portion of the lidar data, the similarity indicating the presence of the non-impeding object.

4 . The system of claim 3 , wherein:

receiving lidar data comprises receiving first lidar data comprising first point cloud data at a first time and second lidar data comprising second point cloud data at a second time; and

determining the first portion of the lidar data comprises:

determining static objects represented in the first point cloud data and the second point cloud data that are stationary between the first point cloud data and the second point cloud data; and

determining the moving object by removing the static objects.

5 . The system of claim 1 , wherein determining the effective range comprises:

determining an indication of the non-impeding object at a first location;

determining a first effective range in a first direction from the lidar sensor to the first location based on the indication of the non-impeding object; and

determining a second effective range in a second direction from the lidar sensor to a second location.

6 . The system of claim 1 , wherein determining the presence of the non-impeding object is further based at least in part on a velocity associated with the radar data and a velocity associated with the lidar data.

7 . One or more non transitory computer readable media storing instructions executable by one or more processors, wherein the instructions, when executed, cause a system to perform operations comprising:

receiving radar data of an environment;

receiving lidar data of the environment;

determining, based at least in part on comparing the radar data and the lidar data, a presence of a non-impeding object in the environment; and

controlling a vehicle based at least in part on removing a portion of the lidar data associated with the non-impeding object.

8 . The one or more non transitory computer readable media of claim 7 , wherein controlling the vehicle is further based at least in part on:

determining, based at least in part on the presence of the non-impeding object, an effective range for a lidar sensor; and

controlling the vehicle based at least in part on the effective range.

9 . The one or more non transitory computer readable media of claim 8 , wherein determining the effective range comprises:

determining an indication of the non-impeding object at a first location based on a similarity score between the lidar data and the radar data;

determining a first effective range in a first direction from the lidar sensor to the first location based on the indication of the non-impeding object; and

determining a second effective range in a second direction from the lidar sensor to a second location.

10 . The one or more non transitory computer readable media of claim 8 , wherein determining the effective range for the lidar sensor comprises:

determining a density of the non-impeding object based at least in part on a similarity score between the lidar data and the radar data; and

determining the effective range for the lidar sensor based on the density of the non-impeding object.

11 . The one or more non transitory computer readable media of claim 8 , wherein controlling the vehicle comprises modifying a perception system of the vehicle to reduce reliance on or ignore lidar returns beyond the effective range in the environment.

12 . The one or more non transitory computer readable media of claim 8 , the operations further comprising:

determining a confidence score for the lidar data based on the effective range; and

controlling the vehicle is based further on the confidence score.

13 . The one or more non transitory computer readable media of claim 7 , wherein determining the presence of the non-impeding object is further based at least in part on:

determining a first portion of the radar data that is associated with a moving object;

determining a first portion of the lidar data that is associated with the moving object; and

determining a similarity between the first portion of the radar data and the first portion of the lidar data, the similarity indicating the presence of the non-impeding object.

14 . The one or more non transitory computer readable media of claim 13 , wherein:

receiving lidar data comprises receiving first lidar data comprising first point cloud data at a first time and second lidar data comprising second point cloud data at a second time; and

determining the first portion of the lidar data comprises:

determining static objects represented in the first point cloud data and the second point cloud data that are stationary between the first point cloud data and the second point cloud data; and

determining the moving object by removing the static objects.

15 . The one or more non transitory computer readable media of claim 7 , wherein determining the presence of the non-impeding object is further based at least in part on a velocity associated with the radar data and a velocity associated with the lidar data.

16 . A method comprising:

receiving radar data of an environment;

receiving lidar data of the environment;

determining, based at least in part on comparing the radar data and the lidar data, a presence of a non-impeding object in the environment;

determining, based at least in part on the presence of the non-impeding object, an effective range for a lidar sensor; and

controlling a vehicle based at least in part on the effective range.

17 . The method of claim 16 , wherein controlling the vehicle is further based at least in part on:

determining, based at least in part on the presence of the non-impeding object, a subset of the lidar data associated with the non-impeding object; and

controlling the vehicle based at least in part on the subset of the lidar data.

18 . The method of claim 16 , wherein determining the presence of the non-impeding object is further based at least in part on:

determining a first portion of the radar data that is associated with a moving object;

determining a first portion of the lidar data that is associated with the moving object; and

determining a similarity between the first portion of the radar data and the first portion of the lidar data, the similarity indicating the presence of the non-impeding object.

19 . The method of claim 16 , wherein determining the effective range comprises:

determining an indication of the non-impeding object at a first location;

determining a first effective range in a first direction from the lidar sensor to the first location based on the indication of the non-impeding object; and

determining a second effective range in a second direction from the lidar sensor to a second location.

20 . The method of claim 16 , wherein determining the presence of the non-impeding object is further based at least in part on a velocity associated with the radar data and a velocity associated with the lidar data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 25, 2024
From: SRIVASTAV, ARVIND
To: ZOOX, INC.
Reel/Frame 068696/0580 →
Continuity (2)
Continuation 17845865 · Jun 21, 2022
Related Publication 20250010885A1 · Jan 9, 2025
References Cited (36)
US 9855894B1 · Khorasani · 2018 [cited by examiner]
US 10754037B2 · Englard et al. · 2020 [cited by applicant]
US 11594017B1 · Gupta et al. · 2023 [cited by applicant]
US 20100066587A1 · Yamauchi · 2010 [cited by examiner]
US 20140324266A1 · Zhu · 2014 [cited by examiner]
US 20160266256A1 · Allen · 2016 [cited by examiner]
US 20170031015A1 · Mei et al. · 2017 [cited by applicant]
US 20170248693A1 · Kim · 2017 [cited by applicant]
US 20170307735A1 · Rohani · 2017 [cited by examiner]
US 20180232947A1 · Nehmadi · 2018 [cited by examiner]
US 20190051063A1 · Tatourian · 2019 [cited by examiner]
US 20190294167A1 · Kutila · 2019 [cited by examiner]
US 20200174131A1 · Chen · 2020 [cited by examiner]
US 20200217952A1 · Rider · 2020 [cited by examiner]
US 20200301013A1 · Banerjee et al. · 2020 [cited by applicant]
US 20200309923A1 · Bhaskaran · 2020 [cited by examiner]
US 20200309957A1 · Bhaskaran · 2020 [cited by examiner]
US 20200379096A1 · Zhou et al. · 2020 [cited by applicant]
US 20210019537A1 · Hiroi · 2021 [cited by applicant]
US 20210094538A1 · Beller · 2021 [cited by examiner]
US 20210183088A1 · Gallo et al. · 2021 [cited by applicant]
US 20210278523A1 · Urtasun et al. · 2021 [cited by applicant]
US 20220083841A1 · Isele · 2022 [cited by examiner]
US 20220101635A1 · Koivisto · 2022 [cited by examiner]
US 20220120903A1 · Krause Perin · 2022 [cited by examiner]
US 20220178718A1 · Xia · 2022 [cited by examiner]
US 20220327719A1 · Shaag · 2022 [cited by examiner]
US 20230168359A1 · Rezvani et al. · 2023 [cited by applicant]
US 20230184931A1 · Wang · 2023 [cited by examiner]
US 20230214728A1 · Danilyuk · 2023 [cited by examiner]
US 20230227027A1 · Sung et al. · 2023 [cited by applicant]
US 20230228884A1 · Won · 2023 [cited by examiner]
US 20230244242A1 · Armstrong-Crews · 2023 [cited by examiner]
US 20230258805A1 · Won · 2023 [cited by applicant]
US 20240385315A1 · Lunn · 2024 [cited by examiner]
Office Action for U.S. Appl. No. 17/845,865, dated Apr. 30, 2024, 55 pages. [cited by applicant]