IP Library › Granted Patent US 10,678,255
Granted Patent B2
US 10,678,255 · App. 15/896,104 · Granted Jun 9, 2020

Systems, methods and apparatuses are provided for enhanced surface condition detection based on image scene and ambient light analysis

Inventors: Qingrong Zhao (Madison Heights, MI); Mohsen Khalili (Sterling Heights, MI); Jinsong Wang (Troy, MI); Bakhtiar B. Litkouhi (Washington, MI)
Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
G05D1/0246G05D1/0238G06T7/40G06T7/62G06T7/73G06T7/90
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,678,255
App. No.
15/896,104
Granted
Jun 9, 2020
Kind
B2
Abstract

Systems, Methods and Apparatuses are provided for detecting surface conditions, which includes: an image scene captured by a camera wherein the image scene includes: a set of a plurality of regions of interest (ROIs); and a processor configured to receive the image scene to: extract at least a first and a second ROI from the set of the plurality of ROIs of the image scene; associate the first ROI with an above-horizon region and associate the second ROI with a surface region; analyze the first ROI and the second ROI in parallel for a condition related to an ambient lighting in the first ROI and for an effect related to the ambient lighting in the second ROI; and extract from the first ROI features of the condition of the ambient lighting and extract from the second ROI features of the effect of the ambient lighting on a surface region.

Claims (54)

1. A surface detection system for detecting surface conditions comprising:

an image scene captured by a camera wherein the image scene comprises: a set of a plurality of regions of interest (ROIs); and a processor configured to receive the image scene to:

extract at least a first and a second ROI from the set of the plurality of ROIs of the image scene;

associate the first ROI with an above-horizon region and associate the second ROI with a surface region;

analyze the first ROI and the second ROI in parallel for a condition related to an ambient lighting in the first ROI and for an effect related to the ambient lighting in the second ROI; and

extract from the first ROI features of the condition of the ambient lighting and extract from the second ROI features of the effect of the ambient lighting on a surface region.

2. The surface detection system of claim 1 , further comprising:

analyzing the second ROI to extract the features of the effects of the ambient lighting on the surface region in common with the features of the first ROI of the conditions of ambient lighting based on a plurality of image attributes wherein a set of extracted features from the first and second ROIs comprise: a common set of features for both ROIs.

3. The surface detection system of claim 2 , wherein the image attributes comprise: a set of one or more of a texture, a brightness and a color information.

4. The surface detection system of claim 3 , further comprising:

generating the common set of features for both the ROIs based on combining the plurality of image attributes of the second ROI with that of the first ROI.

5. The surface detection system of claim 4 , further comprising:

classifying by a classifier each of the common set of features to a particular surface type.

6. The surface detection system of claim 1 , further comprising:

an autonomous, semi-autonomous, or non-autonomous vehicle system, or non-vehicular applications.

7. The surface detection system of claim 1 , wherein

the processor is configured to:

determine, by thresholds related to values of pixels of the first ROI, features of the condition of the ambient lighting and, by thresholds related to values of pixels of the second ROI, features of the effect of the ambient lighting on the surface region.

8. A method for detecting a surface condition, the method comprising:

capturing an image scene by a camera wherein the image scene comprises: a set of a plurality of regions of interest (ROIs);

extracting, by a processor associated with the vehicle, at least a first and a second ROI from the set of the plurality of ROIs of the image scene;

associating, by the processor, the first ROI with an above-horizon region and associating the second ROI with a surface region;

analyzing, by the processor, the first ROI and the second ROI in parallel for a condition related to an ambient lighting in the first ROI and for an effect related to the ambient lighting in the second ROI; and

extracting, by the processor from the first ROI, features of the condition of the ambient lighting and extracting, from the second ROI, features of the effect of the ambient lighting on a surface region.

9. The method of claim 8 , further comprising:

an autonomous, semi-autonomous, non-autonomous driving method, or non-driving use.

10. The method of claim 8 , further comprising:

taking an action by the processor based on a feature of the surface condition or the effect on a surface related to vehicle control.

11. The method of claim 8 , further comprising:

taking an action by the processor based on the surface condition or the effect on a surface of sending a notification for notifying of a particular surface condition or effect on the surface.

12. The method of claim, 8 , further comprising:

analyzing, by the processor, the second ROI to extract the features of the effects of the ambient lighting on the surface region in common with the features of the first ROI of the conditions of ambient lighting based on a plurality of image attributes wherein a set of extracted features from the first and second ROIs comprise: a common set of features for both ROIs.

13. The method of claim 12 , wherein the image attributes comprise: a set of one or more of a texture, a brightness and a color information.

14. The method of claim 13 , further comprising:

generating, by the processor, the common set of features for both the ROIs based on combining the plurality of image attributes of the second ROI with that of the first ROI.

15. A surface detection apparatus, comprising:

a camera for generating images wherein the images comprise: regions of interest (ROI);

a processor, configured to receive the images to:

extract at least a first and a second ROI from the set of the plurality of ROIs of the image scene;

associate the first ROI with an above-horizon region and associate the second ROI with a surface region;

analyze the first ROI and the second ROI in parallel for a condition related to an ambient lighting in the first ROI and for an effect related to the ambient lighting in the second ROI; and

extract from the first ROI features of the condition of the ambient lighting and extract from the second ROI features of the effect of the ambient lighting on a surface region.

16. The apparatus of claim 15 , further comprising:

an autonomous, semi-autonomous or conventional driving apparatus.

17. The apparatus of claim 15 , further comprising:

a cloud connected apparatus to send messages of surface conditions to third party networks.

18. The apparatus of claim 15 , further comprising:

the processor is configured to:

determine, by thresholds related to values of pixels of the first ROI, features of the condition of the ambient lighting and, by thresholds related to values of pixels of the second ROI, features of the effect of the ambient lighting on the surface region.

19. The apparatus of claim 15 , further comprising:

the processor is configured to:

analyze the second ROI to extract the features of the effects of the ambient lighting on the surface region in common with the features of the first ROI of the conditions of ambient lighting based on a plurality of image attributes wherein a set of extracted features from the first and second ROIs comprise: a common set of features for both ROIs;

generate the common set of features for both the ROIs based on combining the plurality of image attributes of the second ROI with that of the first ROI; and

classify, by a classifier, each of the common set of features to a particular surface type.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 14, 2018
From: ZHAO, QINGRONG; KHALILI, MOHSEN; WANG, JINSONG; LITKOUHI, BAKHTIAR B.
To: GM GLOBAL TECHNOLOGY OPERATIONS LLC
Reel/Frame 044919/0981 →
Continuity (1)
Related Publication 20190250630A1 · Aug 15, 2019