IP Library Granted Patent US 11,852,498
Granted Patent B2
US 11,852,498 · App. 17/308,803 · Granted Dec 26, 2023

Lane marking localization

Inventors: Mingdong Wang (San Diego, CA); Chenzhe Qian (San Diego, CA); Xue Mei (San Diego, CA)
Assignee: TUSIMPLE, INC.
G01C21/3602G01C21/32G01S13/89G01S17/89G01S19/13
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,852,498
App. No.
17/308,803
Granted
Dec 26, 2023
Kind
B2
Abstract

Various embodiments of the present disclosure provide a system and method for lane marking localization that may be utilized by autonomous or semi-autonomous vehicles traveling within the lane. In the embodiment, the system comprises a locating device adapted to determine the vehicle's geographic location; a database; a region map; a response map; a camera; and a computer connected to the locating device, database, and camera, wherein the computer is adapted to: receive the region map, wherein the region map corresponds to a specified geographic location; generate the response map by receiving information form the camera, the information relating to the environment in which the vehicle is located; identifying lane markers observed by the camera; and plotting identified lane markers on the response map; compare the response map to the region map; and generate a predicted vehicle location based on the comparison of the response map and region map.

Claims (71)

1. A method implemented by a processor, comprising:

approximating a vehicle's region;

receiving a region map from a database, wherein the region map corresponds to the vehicle's approximated region and comprises a plurality of region points indicating an expected roadway lane;

receiving a response image generated by an imaging device of one or more imaging devices of the vehicle, the response image comprising information relating to the vehicle's environment;

generating a response map from the response image, the response map comprising a plurality of response points indicating the vehicle's location;

comparing the response map to the region map to determine (a) differences between the plurality of response points and the plurality of region points, and (b) a confidence score that comprises a variance that is computed using a logistic function; and

predicting the vehicle's roadway position based on the confidence score and the differences between the plurality of response points and the plurality of region points.

2. The method of claim 1 , wherein the vehicle's region is approximated using a Global Positioning System (GPS) device or an inertial measurement unit (IMU) device.

3. The method of claim 1 , wherein the generating the response map further comprises:

detecting lane markers in the response image, the lane markers pertaining to physical aspects contained in the response image; and

plotting the response points on the response map, the response points indicating locations of the lane markers.

4. The method of claim 1 , further comprising:

generating, based on the differences, the confidence score.

5. The method of claim 1 , wherein the response image is generated from radar sensing equipment, light detection and ranging (LIDAR) sensing equipment, Global Positioning System (GPS) sensing information, and/or images.

6. The method of claim 1 , wherein the region map and the response map are compared at a selected frequency.

7. The method of claim 6 , wherein the selected frequency is at least 20 cycles per second.

8. An apparatus, comprising:

a processor configured to:

approximate a vehicle's region;

receive a region map from a database, wherein the region map corresponds to the vehicle's approximated region and comprises a plurality of region points indicating an expected roadway lane;

receive a response image generated by an imaging device of one or more imaging devices of the vehicle, the response image comprising information relating to the vehicle's environment;

generate a response map from the response image, the response map comprising a plurality of response points indicating the vehicle's location;

compare the response map to the region map to determine (a) differences between the plurality of response points and the plurality of region points, and (b) a confidence score that comprises a variance that is computed using a logistic function; and

predict the vehicle's roadway position based on the confidence score and the differences between the plurality of response points and the plurality of region points.

9. The apparatus of claim 8 , wherein the processor is further configured to:

output the vehicle's predicted location to an advanced driver-assistance system (ADAS).

10. The apparatus of claim 8 , wherein each of the one or more imaging devices is adapted to perceive a different aspect of the vehicle's environment.

11. The apparatus of claim 8 , wherein the variance is computed as:

variance

=

x

min

+

(

x

max

-

x

min

)

1

+

e

S

(

G

x

-

m

)

,

wherein x is a matching score of the response map, x min is a minimum value of the response map, x max is a maximum value of the response map, S is a steepness parameter, G is a growth rate parameter, and m is a midpoint of the response map.

12. The apparatus of claim 8 , wherein the imaging device comprises a light detection and ranging (LIDAR) device.

13. The apparatus of claim 8 , wherein the imaging device comprises a camera that captures images at a rate of at least 10 images per second.

14. A non-transitory computer-readable storage medium having instructions stored thereupon, the storage medium comprising:

instructions for approximating a vehicle's region;

instructions for receiving a region map from a database, wherein the region map corresponds to the vehicle's approximated region and comprises a plurality of region points indicating an expected roadway lane;

instructions for receiving a response image generated by an imaging device of one or more imaging devices of the vehicle, the response image comprising information relating to the vehicle's environment;

instructions for generating a response map from the response image, the response map comprising a plurality of response points indicating the vehicle's location;

instructions for comparing the response map to the region map to determine (a) differences between the plurality of response points and the plurality of region points, and (b) a confidence score that comprises a variance that is computed using a logistic function; and

instructions for predicting the vehicle's roadway position based on the confidence score and the differences between the plurality of response points and the plurality of region points.

15. The storage medium of claim 14 , further comprising:

instructions for comparing the confidence score to a threshold; and

instructions for outputting the vehicle's predicted roadway position based on the confidence score being lower than the threshold.

16. The storage medium of claim 14 , wherein the response map is generated using a camera pointing in a direction opposite to a direction of travel of the vehicle.

17. The storage medium of claim 14 , wherein lane markers in the response map are based on one or more physical structures.

18. The storage medium of claim 17 , wherein the one or more physical structures comprise a bridge, a barrier, a sign, or a building.

19. The storage medium of claim 17 , wherein the lane markers comprise at least one of solid lines, striped lines, or reflectors.

20. The storage medium of claim 14 , wherein the region map further comprises information related to a speed of the vehicle, a direction of travel of the vehicle, or a velocity of the vehicle relative to the expected roadway lane.

Assignments (3)
CHANGE OF NAME Recorded Dec 3, 2025
From: TUSIMPLE, INC.
To: CREATEAI, INC.
Reel/Frame 073832/0485 →
CHANGE OF NAME Recorded Oct 4, 2021
From: TUSIMPLE
To: TUSIMPLE, INC.
Reel/Frame 057710/0602 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 5, 2021
From: WANG, MINGDONG; QIAN, CHENZHE; MEI, XUE
To: TUSIMPLE
Reel/Frame 056148/0330 →
Continuity (2)
Continuation 15896077 · Feb 14, 2018
Related Publication 20210278232A1 · Sep 9, 2021