Granted Patent B2

US 10,528,824 · App. 15/837,276 · Granted Jan 7, 2020

Inventor: Guangyu J. Zou (Warren, MI)

Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC

G06K9/00798B60W30/12G05D1/0221G06K9/6217G06K9/6289G06N3/08G08G1/167

Abstract

Claims (25)

1. A method of controlling a vehicle, comprising:

receiving, via at least one processor, image data from each of plural cameras mounted on the vehicle;

assembling, via the at least one processor, the image data from each of the plural cameras to form assembled image data;

classifying and localizing lane features using an artificial neural network based on the assembled image data, to produce classified and localized lane data;

performing, via the at least one processor, a data fusion process based on the classified and localized lane data defining line segments, thereby producing fused lane feature data;

defining, via the at least one processor, anchor points along each of the line segments;

data clustering, via the at least one processor, the anchor points to produce clustered anchor points;

curve fitting, via the at least one processor, the clustered anchor points to produce the classified and localized lane curve data; and

controlling, via the at least one processor, the vehicle based, in part, on the fused lane feature data.

2. The method of claim 1 , performing, via the at least one processor, an image space to real space transformation based on the classified and localized lane data to provide real space lane feature data.

3. The method of claim 2 , wherein the transforming occurs before the fusing.

4. The method of claim 1 , wherein a plurality of the cameras has a different, but overlapping field of view and the fusion comprises consolidating overlapping fields of view.

5. The method of claim 1 , wherein the classified and localized lane data defines line segments.

6. The method of claim 1 , comprising performing, via the at least one processor, an image space to real space transformation based on the classified and localized lane curve data to provide real space lane feature data and performing the data fusion process based on the real space lane feature data, thereby producing the fused lane feature data.

7. A system for localizing and classifying line-based image features, wherein the system comprises:

at least one processor configured to:

receive image data from each of plural cameras; and

assemble the image data from each of the plural cameras to form assembled image data; and

an artificial neural network configured to:

classify and localize line-based image features based on the assembled image data to produce classified and localized line data defining line segments;

wherein the at least one processor is configured to define anchor points along each of the line segments, to data cluster the anchor points and to curve fit the clustered anchor points to produce the classified and localized lane curve data;

wherein the at least one processor is configured to perform a data fusion process based on the classified and localized line data, thereby producing fused line-based feature data.

8. The system of claim 7 , wherein the artificial neural network is configured to classify and localize line based lane features.

9. The system of claim 7 , wherein the system is a vehicle system and the processor is configured to control a vehicle based, in part, on the fused lane feature data.

10. The system of claim 7 , wherein the at least one processor is configured to perform an image space to real space transformation based on the classified and localized lane curve data to provide real space lane feature data and to perform the data fusion process based on the real space lane feature data, thereby producing the fused lane feature data.

Assignments (1)

Continuity (1)

Cited By (16)

US 12,198,396 US 12,216,610 US 12,223,428 US 12,236,689 US 12,307,350 US 12,346,816 US 12,367,405 US 12,455,739 US 12,462,575 US 12,522,243 US 12,536,131 US 12,554,467 US 12,591,240 US 12,618,976 US 12,623,691 US 12,709,294