IP Library Granted Patent US 12,049,172
Granted Patent B2
US 12,049,172 · App. 17/504,648 · Granted Jul 30, 2024

Camera mirror system display for commercial vehicles including system for identifying road markings

Inventors: Liang Ma (Rochester, MI); Yifan Men (Northville, MI); Troy Otis Cooprider (White Lake, MI)
Assignee: Stoneridge, Inc.
B60R1/12G06T7/13G06T7/73G06V10/22G06V10/443G06V10/56G06V20/588H04N23/695B60R2001/1253G06T2207/10024G06T2207/20061G06T2207/30256G06T2207/30264
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,049,172
App. No.
17/504,648
Granted
Jul 30, 2024
Kind
B2
Abstract

A process for identifying a road feature in an image includes receiving an image at a controller, identifying a region of interest within the image and converting the region of interest from red-green-blue (RGB) to a single color using the controller. A set edges is detected within the region of interest, and at least one line within the set of edges is identified using the controller. The at least one line is compared with a set of known and expected road marking features, and the set of at least one first line in the at least one line is identified as corresponding to a road feature in response to the at least the first line matching the set of known and expected road marking features.

Claims (35)

1. A process for identifying a road feature in an image comprising:

receiving an image at a controller;

identifying a region of interest within the image and converting the region of interest from red-green-blue (RGB) to a single color using the controller;

detecting a set of edges within the region of interest, and identifying at least one line within the set of edges using the controller;

comparing the at least one line with a set of known and expected road marking features, and identifying a set of at least one first line in the at least one line as corresponding to a road feature in response to the at least the first line matching at least one of the set of known and expected road marking features; and

identifying a set of at least one second line in the at least one line as corresponding to a trailer feature in response to every line in the set of at least one second line not matching the set of known and expected road marking features.

2. The process of claim 1 , wherein identifying the set of known and expected road marking features includes an expected width of at least one of parking lines and lane lines.

3. The process of claim 2 , further comprising identifying two lines in the set of at least one first line as corresponding to a lane line in response to the two lines being approximately parallel and being consistently spaced apart by between 4.5 inches (11.43 cm) and 6.5 inches (16.51 cm).

4. The process of claim 3 , wherein the line spacing is an average of the shortest distances between the two approximately parallel lines.

5. The process of claim 1 , wherein identifying the set of lines within the set of edges includes feature extraction via Hough transform.

6. The process of claim 5 , wherein identifying the set of lines within the set of edges further includes filtering background noise from an output of the feature extraction.

7. The process of claim 1 , wherein the set of known and expected road marking features includes known and expected features of parking lines, lane lines, and curbs.

8. The process of claim 1 , further comprising digitally removing edges not corresponding to lines in the set of at least one second line to create a filtered edge image, and providing the filtered edge image to a trailer feature detection module of a camera minor system (CMS).

9. The process of claim 8 , further comprising identifying a trailer bottom line using the trailer feature detection module and determining a trailer angle at least partially using the position of the trailer bottom line within the region of interest.

10. The process of claim 9 , further comprising panning a CMS view based at least partially on the determined trailer angle such that a trailer end is maintained with the CMS view.

11. The process of claim 1 , wherein identifying at least one line within the set of edges using the controller is performed without using polynomial data fitting.

12. The process of claim 1 , wherein converting the region of interest from red-green-blue (RGB) to the single color using the controller comprises one of converting the region of interest to gray scale and extracting a green channel from the region of interest.

13. The process of claim 1 , further comprising distinguishing between at least two corresponding road features by identifying a color of the at least the first line and comparing the identified color to expected colors of the corresponding road features.

14. A camera mirror system for a vehicle comprising:

at least one camera defining a field of view including a view of a ground;

a camera-mirror-system (CMS) controller including a processor and a memory storing instructions for causing the controller to perform an image-based detection process for identifying a road feature in the image, the image based detection process including:

receiving an image from the at least one camera at the controller;

identifying a region of interest within the image and converting the region of interest from red-green-blue (RGB) to a single color using the controller;

detecting a set of edges within the region of interest, and identifying at least one line within the set of edges using the controller;

comparing the at least one line with a set of known and expected road marking features, and identifying a set of at least one first line in the at least one line as corresponding to a road feature in response to the at least the first line matching at least one of the set of known and expected road marking features, wherein comparing the at least one line with a set of known and expected road marking features includes identifying a set of at least one second line in the at least one line as corresponding to a trailer feature in response to every line in the set of at least one second line not matching the set of known and expected road marking features.

15. The camera mirror system of claim 14 , wherein the at least one camera defines a class II and a class IV view.

16. The camera mirror system of claim 14 , wherein the controller further includes a trailer feature detection module and wherein the trailer feature detection module is configured to determine a trailer angle at least partially based on a position of the at least one second line in the image.

17. The camera mirror system of claim 16 , wherein the controller is further configured to pan a class II view presented to a vehicle operator based at least in part on the determined trailer angle such that the class II view includes at least a portion of a trailer end.

18. The camera mirror system of claim 14 , wherein converting the region of interest from red-green-blue (RGB) to the single color using the controller comprises one of converting the region of interest to gray scale and extracting a green channel from the region of interest.

19. A process for identifying features in an image comprising:

receiving an image at a controller;

identifying a region of interest within the image and converting the region of interest from red-green-blue (RGB) to a single color using the controller;

detecting a set of edges within the region of interest, and identifying at least one line within the set of edges using the controller;

comparing the at least one line with a set of road marking features, and identifying a set of at least one first line in the at least one line as corresponding to a road feature in response to the at least the first line matching at least one of the set of road marking features; and

identifying a set of at least one second line in the at least one line as corresponding to a trailer feature in response to every line in the set of at least one second line not matching the set of road marking features.

Assignments (3)
SECURITY INTEREST Recorded Nov 2, 2023
From: STONERIDGE, INC.; STONERIDGE ELECTRONICS, INC.; STONERIDGE CONTROL DEVICES, INC.; STONERIDGE FLEET SOLUTIONS, INC.; SRI HOLDINGS US LLC; SRI DELAWARE HOLDINGS LLC
To: PNC BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 065447/0664 →
SECURITY INTEREST Recorded Dec 27, 2022
From: STONERIDGE, INC.; STONERIDGE CONTROL DEVICES, INC.
To: PNC BANK, NATIONAL ASSOCIATION, AS ADMINISTRATIVE AGENT AND COLLATERAL AGENT
Reel/Frame 062214/0167 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 19, 2021
From: MA, LIANG; MEN, YIFAN; COOPRIDER, TROY OTIS
To: STONERIDGE, INC.
Reel/Frame 058297/0077 →