IP Library › Granted Patent US 12,283,063
Granted Patent B2
US 12,283,063 · App. 17/809,589 · Granted Apr 22, 2025

Ego motion correction of LIDAR output

Inventors: Kevin Rosenblum (Jerusalem, IL); Erez Dagan (Rishpon, IL); David Boublil (Jerusalem, IL)
Assignee: MOBILEYE VISION TECHNOLOGIES LTD.
G06T7/55B60W10/04B60W10/18B60W10/20B60W30/09B60W60/001G01B11/22G01C21/1652G01C21/1656G01C21/30G01C21/3885G01S7/4817G01S17/42G01S17/58G01S17/86G01S17/89G01S17/931H04N23/698H04N23/90B60W2420/403B60W2420/408B60W2554/20B60W2554/4042B60W2554/802B60W2556/45B60W2720/10B60W2720/24G06T2207/10028G06T2207/20081G06T2207/20084G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,283,063
App. No.
17/809,589
Granted
Apr 22, 2025
Kind
B2
Abstract

A navigation system for a host vehicle may include a processor programmed to determine at least one indicator of ego motion of the host vehicle. A processor may be also programmed to receive, from a LIDAR system, a first point cloud including a first representation of at least a portion of an object and a second point cloud including a second representation of the at least a portion of the object. The processor may further be programmed to determine a velocity of the object based on the at least one indicator of ego motion of the host vehicle, and based on a comparison of the first point cloud, including the first representation of the at least a portion of the object, and the second point cloud, including the second representation of the at least a portion of the object.

Claims (55)

1. A navigation system for a host vehicle, the navigation system comprising:

at least one processor programmed to:

determine at least one indicator of ego motion of the host vehicle;

receive, from a LIDAR system associated with the host vehicle and based on a first LIDAR scan of a field of view of the LIDAR system, a first point cloud including a first representation of at least a portion of an object;

receive, from the LIDAR system and based on a second LIDAR scan of the field of view of the LIDAR system, a second point cloud including a second representation of the at least a portion of the object; and

determine a velocity of the object based on the at least one indicator of ego motion of the host vehicle, and based on a comparison of the first point cloud, including the first representation of the at least a portion of the object, and the second point cloud, including the second representation of the at least a portion of the object, wherein the comparison of the first point cloud and the second point cloud comprises a subtraction of first point cloud data associated with the first point cloud from second point cloud data associated with the second point cloud.

2. The navigation system of claim 1 , wherein the at least one processor is further programmed to receive a plurality of images captured from an environment of the host vehicle by a camera onboard the host vehicle, and wherein the at least one indicator of ego motion of the host vehicle is determined based on an analysis of the plurality of images.

3. The navigation system of claim 1 , wherein the at least one indicator of ego motion of the host vehicle is determined based on an analysis of map information.

4. The navigation system of claim 1 , wherein the at least one indicator of ego motion of the host vehicle is determined based on a heading direction of the host vehicle.

5. The navigation system of claim 1 , wherein the at least one indicator of ego motion of the host vehicle is determined based on an analysis of an output from one or more of a speedometer, an accelerometer, and a GPS device.

6. The navigation system of claim 1 , wherein the first point cloud or the second point cloud includes three-dimensional coordinates for a plurality of locations in the field of view of the LIDAR system at which a plurality of laser reflections are detected by the LIDAR system.

7. The navigation system of claim 1 , wherein the velocity includes a speed and a direction of the object.

8. The navigation system of claim 1 , wherein the at least one processor is further programmed to determine at least one actual point cloud difference between the first and second point clouds.

9. The navigation system of claim 8 , wherein the at least one processor is further programmed to determine an expected point cloud difference between the first and second point clouds.

10. The navigation system of claim 9 , wherein the expected point cloud difference includes a change of a location of the at least a portion of the object relative to the field of view of the LIDAR system.

11. The navigation system of claim 9 , wherein the expected point cloud difference includes a change of a size of the second representation relative to the first representation.

12. The navigation system of claim 9 , wherein the at least one processor is further programmed to determine the velocity of the object based on a comparison of the actual and expected point cloud differences.

13. The navigation system of claim 12 , wherein the comparison of the actual and expected point cloud differences indicates a non-zero velocity of the object.

14. The navigation system of claim 12 , wherein the velocity of the object is further based on assigning a first weight to a result of the comparison of the actual and expected point cloud differences, and assigning a second weight to a result of the comparison of the actual and expected image differences.

15. The navigation system of claim 14 , wherein the first and second weights are different.

16. The navigation system of claim 14 , wherein the first and second weights are equal.

17. The navigation system of claim 1 , wherein the at least one processor is further programmed to determine whether the object is moving based on the velocity of the object.

18. The navigation system of claim 1 , wherein the at least one processor is further programmed to determine at least one navigational action for the host vehicle based on the velocity of the object.

19. The navigation system of claim 18 , wherein the at least one processor is further programmed to cause the host vehicle to implement the at least one navigational action.

20. The navigation system of claim 18 , wherein the at least one navigational action includes one of accelerating, braking, or turning the host vehicle.

21. The navigation system of claim 1 , wherein the at least one processor is further programmed to determine whether to cause an update to a map based on the velocity of the object.

22. The navigation system of claim 1 , wherein the at least one processor is further programmed to transmit a location of the object based on the velocity of the object.

23. The navigation system of claim 22 , wherein the location of the object is transmitted to a server configured to update a map.

24. The navigation system of claim 1 , wherein the at least one processor is further programmed to:

receive from a camera associated with the host vehicle a first image including a first image representation of the at least a portion of the object;

receive from the camera a second image including a second image representation of the at least a portion of the object; and

wherein the velocity of the object is further based on a comparison of the first image, including the first image representation of the at least a portion of the object, and the second image, including the second image representation of the at least a portion of the object.

25. The navigation system of claim 24 , wherein the at least one processor is further programmed to determine at least one actual image difference between the first and second images.

26. The navigation system of claim 25 , wherein the at least one processor is further programmed to determine an expected image difference between the first and second images.

27. The navigation system of claim 26 , wherein the expected image difference includes a change of a location of the at least a portion of the object relative to a field of view of the camera.

28. The navigation system of claim 26 , wherein the expected image difference includes a change of a size of the second image representation relative to the first image representation.

29. The navigation system of claim 1 , wherein the at least one processor is further programmed to determine the velocity of the object based on a comparison of the actual and expected image differences.

30. A method for detecting an object in an environment of a host vehicle, the method comprising:

determining at least one indicator of ego motion of the host vehicle;

receiving, from a LIDAR system associated with the host vehicle and based on a first LIDAR scan of a field of view of the LIDAR system, a first point cloud including a first representation of at least a portion of an object;

receiving, from the LIDAR system and based on a second LIDAR scan of the field of view of the LIDAR system, a second point cloud including a second representation of the at least a portion of the object; and

determining a velocity of the object based on the at least one indicator of ego motion of the host vehicle, and based on a comparison of the first point cloud, including the first representation of the at least a portion of the object, and the second point cloud, including the second representation of the at least a portion of the object, wherein the comparison of the first point cloud and the second point cloud comprises a subtraction of first point cloud data associated with the first point cloud from second point cloud data associated with the second point cloud.

31. A computer-readable medium storing instructions that when executed by at least one processor, are configured to cause the at least one processor programmed to:

determine at least one indicator of ego motion of a host vehicle; receive, from a LIDAR system associated with the host vehicle and based on a first LIDAR scan of a field of view of the LIDAR system, a first point cloud including a first representation of at least a portion of an object;

receive, from the LIDAR system and based on a second LIDAR scan of the field of view of the LIDAR system, a second point cloud including a second representation of the at least a portion of the object; and

determine a velocity of the object based on the at least one indicator of ego motion of the host vehicle, and based on a comparison of the first point cloud, including the first representation of the at least a portion of the object, and the second point cloud, including the second representation of the at least a portion of the object, wherein the comparison of the first point cloud and the second point cloud comprises a subtraction of first point cloud data associated with the first point cloud from second point cloud data associated with the second point cloud.

32. The navigation system of claim 1 , wherein each of the first point cloud data and the second point cloud data comprises at least one light intensity value.

33. The method of claim 30 , wherein the at least one indicator of ego motion of the host vehicle is determined based on an analysis of map information.

34. The method of claim 30 , wherein the at least one indicator of ego motion of the host vehicle is determined based on a heading direction of the host vehicle.

35. The method of claim 30 , wherein the at least one indicator of ego motion of the host vehicle is determined based on an analysis of an output from one or more of a speedometer, an accelerometer, and a GPS device.

36. The method of claim 30 , wherein the first point cloud or the second point cloud includes three-dimensional coordinates for a plurality of locations in the field of view of the LIDAR system at which a plurality of laser reflections are detected by the LIDAR system.

37. The computer-readable medium storing of claim 31 , wherein the at least one indicator of ego motion of the host vehicle is determined based on an analysis of map information.

38. The computer-readable medium storing of claim 31 , wherein the at least one indicator of ego motion of the host vehicle is determined based on a heading direction of the host vehicle.

39. The computer-readable medium storing of claim 31 , wherein the at least one indicator of ego motion of the host vehicle is determined based on an analysis of an output from one or more of a speedometer, an accelerometer, and a GPS device.

40. The computer-readable medium storing of claim 31 , wherein the first point cloud or the second point cloud includes three-dimensional coordinates for a plurality of locations in the field of view of the LIDAR system at which a plurality of laser reflections are detected by the LIDAR system.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 17, 2023
From: DAGAN, EREZ; BOUBLIL, DAVID
To: MOBILEYE VISION TECHNOLOGIES LTD.
Reel/Frame 065255/0993 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 14, 2022
From: ROSENBLUM, KEVIN
To: MOBILEYE VISION TECHNOLOGIES LTD.
Reel/Frame 061430/0873 →
Continuity (4)
Continuation PCTUS2020067753 · Dec 31, 2020
Provisional Application 63082619 · Sep 24, 2020
Provisional Application 62957000 · Jan 3, 2020
Related Publication 20220333927A1 · Oct 20, 2022
References Cited (30)
US 8064643B2 · Stein · 2011 [cited by examiner]
US 20080137908A1 · Stein et al. · 2008 [cited by applicant]
US 20160110878A1 · Chang · 2016 [cited by examiner]
US 20170371348A1 · Mou · 2017 [cited by examiner]
US 20180314253A1 · Mercep et al. · 2018 [cited by applicant]
US 20190025841A1 · Haynes et al. · 2019 [cited by applicant]
US 20200301013A1 · Banerjee · 2020 [cited by examiner]
CN 105752154A · 2016 [cited by applicant]
CN 107438754A · 2017 [cited by applicant]
CN 109643367A · 2019 [cited by applicant]
DE 102016214027A1 · 2018 [cited by applicant]
DE 102017206847A1 · 2018 [cited by applicant]
EP 3748606A1 · 2020 [cited by applicant]
JP 2002352226A · 2002 [cited by applicant]
JP 2016156973A · 2016 [cited by applicant]
JP 2019533810A · 2019 [cited by applicant]
WO WO2018015811A1 · 2018 [cited by applicant]
WO WO2018063245A1 · 2018 [cited by applicant]
WO WO2018127789A1 · 2018 [cited by applicant]
WO 2019151110A1 · 2019 [cited by applicant]
WO 2019188389A1 · 2019 [cited by applicant]
Office Action for German Patent Application No. 112020006426.3 dated Feb. 5, 2024. [cited by applicant]
The First Office Action for Chinese Application No. 202310031754.2, issuing No. 2023102001307930, issuing date Oct. 20, 2023. [cited by applicant]
International Search Report and Written Opinion of the International Searching Authority dated Jun. 8, 2021, PCT/US2020/067753. [cited by applicant]
Cho Kuk et al., “Pedestrian and car detection and classification for unmanned ground vehicle . . . ,” Unmanned Systems Technology, vol. 8045, No. 1, May 13, 2011. [cited by applicant]
Sergio A F Rodriguez et al., “An embedded multi-modal system for object localization . . . ,” IEEE Intelligent Transportation Systems Magazine, vol. 4, No. 4, pp. 42-53, Jan. 2012. [cited by applicant]
Extended European Search Report dated Jun. 13, 2024, for corresponding European Application No. 24157351.8-1009/4345419. [cited by applicant]
Japanese Office Action issued in the counterpart Japanese Application No. 2022-541307 issued on Sep. 10, 2024. [cited by applicant]
Communication pursuant Article 94(3) EPC issued on Sep. 12, 2024. [cited by applicant]
Sergio A. Rodriguez et al., “An Embedded Multi-Modal System for Object Localization and Tracking”, IEEE Intelligent Transportation Systems Magazine, Winter 2012. [cited by applicant]