IP Library Granted Patent US 11,216,004
Granted Patent B2
US 11,216,004 · App. 16/183,012 · Granted Jan 4, 2022

Map automation—lane classification

Inventors: Justin Jin-Wei Liang (Toronto, CA); Raquel Urtasun Sotil (Toronto, CA)
Assignee: UATC, LLC
G05D1/0246G01C21/32G05D1/0088G05D1/0274G06F16/285G06F16/29G06F16/51G06K9/00798G06K9/4628G06K9/6273G06N3/04G06N3/0454G06N3/08G05D2201/0213
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,216,004
App. No.
16/183,012
Granted
Jan 4, 2022
Kind
B2
Abstract

A computer system including one or more processors programmed or configured to receive image data associated with an image of one or more roads, where the one or more roads comprise one or more lanes, determine a lane classification of the one or more lanes based on the image data associated with the image of the one or more roads, and provide lane classification data associated with the lane classification of the one or more lanes.

Claims (80)

1. A method comprising:

receiving, with a computer system comprising one or more processors, image data associated with an image of a road, wherein the road comprises one or more lanes;

determining that the image data depicts a portion of the road comprising a curve;

determining a first line in the image data, the first line being perpendicular to the road at a first position on the curve;

determining a second line in the image data, the second line being perpendicular to the road at a second position on the curve, first position being different than the second position;

arranging a first portion of the image data along the first line and a second portion of the image data along the second line to generate warped road segment image data depicting a warped road segment;

determining, with the computer system, a lane classification of the one or more lanes based on the image data associated with the image of the road and the warped road segment image data; and

providing, with the computer system, lane classification data associated with the lane classification to an autonomous vehicle (AV) for controlling travel of the AV in the one or more lanes.

2. The method of claim 1 , further comprising:

providing the image data associated with the image of the road to a convolutional neural network; and

processing the image data associated with the image of the road by the convolutional neural network, the lane classification of the one or more lanes being based at least in part on the processing of the image data by the convolutional neural network.

3. The method of claim 1 , wherein determining the lane classification of the one or more lanes comprises:

determining one or more lane markings associated with the one or more lanes of the road based on the image data associated with the image of the road; and

determining, with the computer system, a lane classification of the one or more lanes based on the one or more lane markings associated with the one or more lanes.

4. The method of claim 3 , further comprising determining one or more lane attributes associated with the one or more lanes of the road based on the one or more lane markings associated with the one or more lanes of the road, the determining of the lane classification of the one or more lanes being based at least in part on the one or more lane attributes associated with the one or more lanes of the road.

5. The method of claim 1 , wherein the lane classification of the one or more lanes comprises an indication of a lane marking of the one or more lanes.

6. The method of claim 5 , wherein the lane marking comprises at least one of:

one or more lines,

one or more letters,

one or more symbols, or

any combination thereof.

7. The method of claim 1 , further comprising:

providing the warped road segment image data as an input to a machine learning algorithm; and

receiving the lane classification of the one or more lanes based on the warped road segment image data associated with the warped road segment.

8. A computing system comprising:

one or more processors programmed or configured to:

receive image data associated with an image of a road, wherein the road comprise one or more lanes;

determine that the image data depicts a portion of the road comprising a curve;

determine a first line in the image data, the first line being perpendicular to the road at a first position on the curve;

determine a second line in the image data, the second line being perpendicular to the road at a second position on the curve, first position being different than the second position;

arrange a first portion of the image data along the first line and a second portion of the image data along the second line to generate warped road segment image data depicting a warped road segment;

determine a lane classification of the one or more lanes based on the image data associated with the image of the road and the warped road segment image data; and

provide lane classification data associated with the lane classification of the one or more lanes to an autonomous vehicle (AV) for controlling travel of the AV in the one or more lanes.

9. The computing system of claim 8 , wherein the one or more processors are further programmed or configured to:

provide the image data associated with the image of the road to a convolutional neural network;

wherein the one or more processors, when determining a lane classification of the one or more lanes are further programmed or configured to: process the image data associated with the image of the road by the convolutional neural network; and

the one or more processors are further programmed or configured to: receive the lane classification of the one or more lanes of the road from the convolutional neural network based on processing the image data associated with the image of the road.

10. The computing system of claim 8 , wherein the one or more processors are further programmed or configured to:

determine one or more lane markings associated with the one or more lanes of the road based on the image data associated with the image of the road; and

determine a lane classification of the one or more lanes based on the one or more lane markings associated with the one or more lanes.

11. The computing system of claim 10 , wherein the one or more processors are further programmed or configured to:

determine one or more lane attributes associated with the one or more lanes of the road based on the one or more lane markings associated with the one or more lanes of the road; and

wherein the one or more processors, when determining the lane classification of the one or more lanes, are programmed or configured to:

determine the lane classification of the one or more lanes based on the one or more lane attributes associated with the one or more lanes of the road.

12. The computing system of claim 8 , wherein the lane classification of the one or more lanes comprises an indication of a lane marking of the one or more lanes.

13. The computing system of claim 12 , wherein the lane marking comprises at least one of:

one or more lines,

one or more letters,

one or more symbols, or

any combination thereof.

14. The computing system of claim 8 , wherein the one or more processors are further programmed or configured to:

provide the warped road segment image data as an input to a machine learning algorithm; and

receive the lane classification of the one or more lanes based on the warped road segment image data associated with the warped road segment.

15. An autonomous vehicle (AV) comprising:

one or more sensors for detecting an object in an environment surrounding the autonomous vehicle; and

a vehicle computing system comprising one or more processors, wherein the vehicle computing system is programmed or configured to:

receive AV map data associated with an AV map including a road, the AV map including one or more prediction scores associated with one or more areas of the AV map, wherein the AV map data is determined based on:

receiving image data associated with an image of the road, wherein the road comprises one or more lanes;

determining that the image data depicts a portion of the road comprising a curve;

determining a first line in the image data, the first line being perpendicular to the road at a first position on the curve;

determining a second line in the image data, the second line being perpendicular to the road at a second position on the curve, first position being different than the second position;

arranging a first portion of the image data along the first line and a second portion of the image data along the second line to generate warped road segment image data depicting a warped road segment;

determining a lane classification of the one or more lanes based on the image data associated with the image of the road and the warped road segment image data;

providing lane classification data associated with the lane classification of the one or more lanes; and

control travel of the autonomous vehicle based on sensor data from the one or more sensors and the AV map data associated with the AV map.

16. The autonomous vehicle of claim 15 , wherein the vehicle computing system is further programmed or configured to:

control the autonomous vehicle to travel with respect to the lane classification based on determining that the one or more areas of the AV map comprise a lane marking of the one or more lanes.

17. The autonomous vehicle of claim 15 , wherein the lane classification of the one or more lanes comprises an indication of a lane marking of the one or more lanes.

18. The autonomous vehicle of claim 17 , wherein the lane marking comprises at least one of:

one or more lines,

one or more letters,

one or more symbols, or

any combination thereof.

19. The autonomous vehicle of claim 15 , wherein the AV map data is determined based on:

providing the image data associated with the image of the road to a convolutional neural network;

processing the image data associated with the image of the road by the convolutional neural network; and

receiving the lane classification of the one or more lanes of the road from the convolutional neural network based on processing the image data associated with the image of the road.

20. The autonomous vehicle of claim 15 , wherein the AV map data is determined based on:

determining one or more lane markings associated with the one or more lanes of the road based on the image data associated with the image of the road; and

determining the lane classification of the one or more lanes based on the one or more lane markings associated with the one or more lanes.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 2, 2024
From: UATC, LLC
To: AURORA OPERATIONS, INC.
Reel/Frame 066973/0513 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 1, 2020
From: LIANG, JUSTIN JIN-WEI; URTASUN SOTIL, RAQUEL
To: UATC, LLC
Reel/Frame 052281/0634 →