Granted Patent B2

US 12,450,917 · App. 17/820,212 · Granted Oct 21, 2025

Inventor: Chuan Xiao (Shanghai, CN)

Assignee: HARMAN INTERNATIONAL INDUSTRIES, INCORPORATED

G06V20/586G06V10/751G06V10/82

Abstract

Claims (77)

1. A method for an automatic parking system, comprising:

obtaining a parking-line segmentation image through a segmentation neural network;

preprocessing the parking-line segmentation image to obtain at least one parking-line skeleton image; and

calculating straight-line equations and straight-line endpoints for parking lines in the parking-line segmentation image based on the at least one parking-line skeleton image;

wherein the preprocessing of the parking-line segmentation image to obtain the at least one parking-line skeleton image comprises:

extracting pixels belonging to the parking lines of the parking-line segmentation image, displaying the extracted pixels with a first color, and displaying remaining pixels of the parking-line segmentation image not belonging to the parking lines with a second color, to generate an extracted parking-line segmentation image;

dilating the extracted parking-line segmentation image to generate a dilated parking-line segmentation image;

determining one or more connection regions of the parking-line segmentation image based on the dilated parking-line segmentation image, wherein each connection region of the one or more connection regions corresponds to a region in the dilated parking-line segmentation image, and a connection region of the one or more connection regions comprises a first set of pixels belonging to the parking lines that are connected to each other and separated from a second set of pixels belonging to the parking lines included in different connection regions of the one or more connection regions;

dividing the dilated parking-line segmentation image according to the one or more connection regions to generate one or more respective sub-images; and

performing a skeleton extraction on a sub-image of the one or more respective sub-images to generate a respective parking-line skeleton image corresponding to the sub-image.

2. The method according to claim 1 , wherein the calculating of straight-line equations and straight-line endpoints for parking lines in the parking-line segmentation image based on the respective parking-line skeleton image comprises:

performing, for the parking-line skeleton image, the following operations:

a first operation dividing the parking-line skeleton image into multiple patches;

a second operation generating multiple straight-line equations based on the multiple patches, wherein each straight-line equation is formed by fitting the extracted pixels with the first color in each patch;

a third operation selecting an optimal straight-line equation from the multiple straight-line equations, wherein the optimal straight-line equation represents a straight line that matches the largest quantity of the extracted pixels with the first color in the parking-line skeleton image;

a fourth operation calculating corresponding endpoints of the optimal straight-line equation in the parking-line skeleton image; and

a fifth operation deleting the extracted pixels with the first color corresponding to the optimal straight-line equation from the parking-line skeleton image, and

performing the first operation, the second operation, the third operation, the fourth operation, and the fifth operation in a loop until at least one of: a number of repetitions of the first operation, the second operation, the third operation, the fourth operation, and the fifth operation reaches a straight-line quantity threshold; and a quantity of the extracted pixels with the first color remaining in the parking-line skeleton image is less than a pixel quantity threshold.

3. The method according to claim 2 , wherein the generating of multiple straight-line equations based on the multiple patches comprises:

performing, for each patch, the following operations in parallel:

getting the patch;

obtaining coordinates of the extracted pixels with the first color in the patch; and

fitting a straight-line equation based on the obtained coordinates of the extracted pixels with the first color.

4. The method according to claim 3 , further comprising:

performing, for each patch, the following operations in parallel:

determining whether a ratio of a first quantity of the extracted pixels with the first color matched by the straight-line equation to a second quantity of all the extracted pixels with the first color in the patch is greater than a ratio threshold, wherein a pixel with the first color in the patch whose distance from a first straight line represented by the straight-line equation is less than a distance threshold is considered to be matched by the straight-line equation;

in response to the ratio being greater than the ratio threshold, transferring the straight-line equation to a coordinate system of the parking-line skeleton image to represent a second straight line in the parking-line skeleton image, and taking the first quantity of the extracted pixels with the first color matched by the straight-line equation in the parking-line skeleton image as a value of a comparison parameter; and

setting the value of the comparison parameter to 0 in response to the ratio being equal to or less than the ratio threshold.

5. The method according to claim 4 , further comprising:

comparing multiple comparison parameter values calculated for the multiple patches; and

selecting the straight-line equation corresponding to a largest comparison parameter value among the multiple comparison parameter values as the optimal straight-line equation.

6. The method according to claim 1 , further comprising:

filtering the at least one parking-line skeleton image to generate a filtered parking-line skeleton image.

7. An automatic parking system, comprising:

one or more processors; and

a memory having executable instructions that, when executed, cause the one or more processors to:

obtain a parking-line segmentation image through a segmentation neural network;

extract pixels belonging to parking lines from the parking-line segmentation image, display one or more pixels belonging to the parking lines with a first color, and display one or more remaining pixels not belonging to the parking lines with a second color, to generate an extracted parking-line segmentation image;

dilate the extracted parking-line segmentation image to generate a dilated parking-line segmentation image;

determine one or more connection regions of the parking-line segmentation image based on the dilated parking-line segmentation image, wherein each connection region of the one or more connection regions corresponds to a region in the dilated parking-line segmentation image, and a connection region of the one or more connection regions comprises a first set of pixels of the parking lines that are connected to each other and separated from a second set of pixels of the parking lines included in different connection regions of the one or more connection regions,

divide the dilated parking-line segmentation image according to the one or more connection regions to generate one or more respective sub-images;

perform a skeleton extraction on a sub-image of the one or more respective sub-images to generate a respective parking-line skeleton image corresponding to the sub-image; and

calculate straight-line equations and straight-line endpoints for the parking lines in the parking-line segmentation image based on the respective parking-line skeleton image.

8. The automatic parking system according to claim 7 , wherein to calculate straight-line equations and straight-line endpoints for parking lines in the parking-line segmentation image based on the respective parking-line skeleton image comprises:

perform, for the parking-line skeleton image, the following operations:

a first operation to divide the parking-line skeleton image into multiple patches;

a second operation to generate multiple straight-line equations based on the multiple patches, wherein each straight-line equation is formed by fitting the extracted pixels with the first color in each patch;

a third operation to select an optimal straight-line equation from the multiple straight-line equations, wherein the optimal straight-line equation represents a straight line that matches the largest quantity of extracted pixels with the first color in the parking-line skeleton image;

a fourth operation to calculate corresponding endpoints of the optimal straight-line equation in the parking-line skeleton image; and

a fifth operation to delete the extracted pixels with the first color corresponding to the optimal straight-line equation from the parking-line skeleton image, and

perform the first operation, the second operation, the third operation, the fourth operation, and the fifth operation in a loop until at least one of: a number of repetitions of the first operation, the second operation, the third operation, the fourth operation, and the fifth operation reaches a straight-line quantity threshold; and a quantity of the extracted pixels with the first color remaining in the parking-line skeleton image is less than a pixel quantity threshold.

9. The automatic parking system according to claim 8 , wherein to generate multiple straight-line equations based on the multiple patches comprises:

perform, for each patch, the following operations in parallel:

get the patch;

obtain coordinates of the extracted pixels with the first color in the patch; and

fit a straight-line equation based on the obtained coordinates of the extracted pixels with the first color.

10. The automatic parking system according to claim 9 , wherein the executable instructions further cause the one or more processors to:

perform, for each patch, the following operations in parallel:

determine whether a ratio of a first quantity of the extracted pixels with the first color matched by the straight-line equation to a second quantity of all the extracted pixels with the first color in the patch is greater than a ratio threshold, wherein a pixel with the first color in the patch whose distance from a first straight line represented by the straight-line equation is less than a distance threshold is considered to be matched by the straight-line equation;

in response to the ratio being greater than the ratio threshold, transfer the straight-line equation to a coordinate system of the parking-line skeleton image to represent a second straight line in the parking-line skeleton image, and take the first quantity of the extracted pixels with the first color matched by the straight-line equation in the parking-line skeleton image as a value of a comparison parameter; and

set the value of the comparison parameter to 0 in response to the ratio being equal to or less than the ratio threshold.

11. The automatic parking system according to claim 10 , wherein the executable instructions further cause the one or more processors to:

compare multiple comparison parameter values calculated for the multiple patches; and

select the straight-line equation corresponding to a largest comparison parameter value among the multiple comparison parameter values as the optimal straight-line equation.

12. The automatic parking system according to claim 7 , wherein the executable instructions further cause the one or more processors to:

filter the respective parking-line skeleton image to generate a filtered parking-line skeleton image.

13. A method for an automatic parking system, comprising:

obtaining a parking-line segmentation image through a segmentation neural network;

determining a connection region of the parking-line segmentation image, the connection region comprising a first set of pixels belonging to a first set of parking lines of the parking-line segmentation image that are connected to each other and separated from a second set of pixels belonging to a second set of parking lines included in different connection regions of the parking-line segmentation image;

performing a skeleton extraction on a portion of the parking-line segmentation image corresponding to the connection region, to generate a respective parking-line skeleton image; and

calculating straight-line equations and straight-line endpoints for the first set of parking lines based on the parking-line skeleton image;

wherein determining the connection region and performing the skeleton extraction on the portion of the parking-line segmentation image corresponding to the connection region, to generate the respective parking-line skeleton image further comprises:

extracting pixels belonging to the parking lines of the parking-line segmentation image, displaying the extracted pixels with a first color, and displaying remaining pixels of the parking-line segmentation image not belonging to the parking lines with a second color, to generate an extracted parking-line segmentation image;

dilating the extracted parking-line segmentation image to generate a dilated parking-line segmentation image;

determining one or more connection regions of the parking-line segmentation image based on the dilated parking-line segmentation image, wherein each connection region of the one or more connection regions corresponds to a region in the dilated parking-line segmentation image;

dividing the dilated parking-line segmentation image according to the one or more connection regions to generate one or more respective sub-images; and

performing the skeleton extraction on a sub-image of the one or more respective sub-images.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (9)