IP Library Granted Patent US 11,023,747
Granted Patent B2
US 11,023,747 · App. 16/293,328 · Granted Jun 1, 2021

Method, apparatus, and system for detecting degraded ground paint in an image

Inventors: Nicholas Pojman (San Francisco, CA); Anish Mittal (San Francisco, CA); David Lawlor (Chicago, IL); Zhanwei Chen (Richmond, CA)
Assignee: HERE Global B.V.
G06K9/00798G06T5/005G06T7/0002G06T7/10G06T2207/10024G06T2207/30256
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,023,747
App. No.
16/293,328
Granted
Jun 1, 2021
Kind
B2
Abstract

An approach is provided for detecting degraded ground paint in an image. The approach, for example, involves performing semantic segmentation on the image to determine one or more pixels of the image that are classified in a ground paint category. The approach also involves generating a binary image that contains the one or more pixels of the image that are classified in the ground paint category. The approach further involves generating a hole-filled binary image by filling in the binary image to generate one or more curvilinear structures from the one or more pixels. The approach further involves determining a difference between the image and the hole-filled binary image to identify one or more degraded ground paint pixels of the image and providing the one or more degraded ground paint pixels as an output.

Claims (47)

1. A computer-implemented method for detecting degraded ground paint in an image comprising:

performing semantic segmentation on the image to determine one or more pixels of the image that are classified in a ground paint category;

generating a binary image that contains the one or more pixels of the image that are classified in the ground paint category;

generating a hole-filled binary image by filling in the binary image to generate one or more curvilinear structures from the one or more pixels;

determining a difference between the image and the hole-filled binary image to identify one or more degraded ground paint pixels of the image; and

providing the one or more degraded ground paint pixels as an output.

2. The method of claim 1 , further comprising:

replacing a color value of the one or more degraded ground paint pixels in the image with a ground paint color value to create an output image.

3. The method of claim 2 , further comprising:

determining the ground paint color value from a respective color value of the one or more pixels of the image that are classified in the ground paint category.

4. The method of claim 3 , wherein the one or more pixels of the image from which the ground value is determined are within a proximity threshold of the one or more degraded ground paint pixels.

5. The method of claim 2 , wherein the ground paint color value is an aggregate value based on the respective color value of the one or more pixels of the image that are classified in the ground paint category.

6. The method of claim 2 , further comprising:

providing the output image for feature labeling.

7. The method of claim 6 , wherein the feature labeling is used from determining a ground control point.

8. The method of claim 1 , further comprising:

identifying the image as a faded ground paint image based on the one or more degraded ground paint pixels.

9. The method of claim 8 , wherein the faded ground paint image is excluded from feature labeling.

10. An apparatus for detecting degraded ground paint in an image comprising:

at least one processor; and

at least one memory including computer program code for one or more programs,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,

perform semantic segmentation on the image to determine one or more pixels of the image that are classified in a ground paint category;

generate a binary image that contains the one or more pixels of the image that are classified in the ground paint category;

generate a hole-filled binary image by filling in the binary image to generate one or more curvilinear structures from the one or more pixels;

determine a difference between the image and the hole-filled binary image to identify one or more degraded ground paint pixels of the image; and

provide the one or more degraded ground paint pixels as an output.

11. The apparatus of claim 10 , wherein the apparatus is further caused to:

replace a color value of the one or more degraded ground paint pixels in the image with a ground paint color value to create an output image.

12. The apparatus of claim 11 , wherein the apparatus is further caused to:

determine the ground paint color value from a respective color value of the one or more pixels of the image that are classified in the ground paint category.

13. The apparatus of claim 12 , wherein the one or more pixels of the image from which the ground value is determined are within a proximity threshold of the one or more degraded ground paint pixels.

14. The apparatus of claim 11 , wherein the ground paint color value is an aggregate value based on the respective color value of the one or more pixels of the image that are classified in the ground paint category.

15. The apparatus of claim 11 , wherein the apparatus is further caused to:

provide the output image for feature labeling.

16. A non-transitory computer-readable storage medium for detecting degraded ground paint in an image, carrying one or more sequences of one or more instructions which, when executed by one or more processors, cause an apparatus to perform:

performing semantic segmentation on the image to determine one or more pixels of the image that are classified in a ground paint category;

generating a binary image that contains the one or more pixels of the image that are classified in the ground paint category;

generating a hole-filled binary image by filling in the binary image to generate one or more curvilinear structures from the one or more pixels;

determining a difference between the image and the hole-filled binary image to identify one or more degraded ground paint pixels of the image; and

providing the one or more degraded ground paint pixels as an output.

17. The non-transitory computer-readable storage medium of claim 16 , wherein the apparatus is caused to further perform:

replacing a color value of the one or more degraded ground paint pixels in the image with a ground paint color value to create an output image.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the apparatus is caused to further perform:

determining the ground paint color value from a respective color value of the one or more pixels of the image that are classified in the ground paint category.

19. The non-transitory computer-readable storage medium of claim 18 , wherein the one or more pixels of the image from which the ground value is determined are within a proximity threshold of the one or more degraded ground paint pixels.

20. The non-transitory computer-readable storage medium of claim 17 , wherein the ground paint color value is an aggregate value based on the respective color value of the one or more pixels of the image that are classified in the ground paint category.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 6, 2019
From: POJMAN, NICHOLAS; MITTAL, ANISH; LAWLOR, DAVID; CHEN, ZHANWEI
To: HERE GLOBAL B.V.
Reel/Frame 048521/0046 →
Continuity (1)
Related Publication 20200285862A1 · Sep 10, 2020
Cited By (1)
US 12,437,375