Granted Patent B2

US 12,374,126 · App. 17/950,644 · Granted Jul 29, 2025

Inventor: Yuan Shen (Shenzhen, CN)

Assignee: Tencent Technology (Shenzhen) Company Limited

G06V20/58G06T7/11G06T7/149G06T7/50G06V10/44G06V10/764G06T2207/10028G06T2207/30261

Abstract

Claims (88)

1. An obstacle detection method, the method comprising:

acquiring a road scene image of a road where a target vehicle is located;

performing obstacle recognition on the road scene image to obtain region information and depth-of-field information corresponding to obstacles in the road scene image;

determining target obstacles in an occlusion relationship based on the region information corresponding to each of the obstacles, and determining a relative depth-of-field relationship between the target obstacles based on the depth-of-field information of the target obstacles;

acquiring a ranging result of each of the obstacles collected by a ranging apparatus corresponding to the target vehicle; and

determining, based on the relative depth-of-field relationship between the target obstacles and the ranging result of each of the obstacles, an obstacle detection result corresponding to the road where the target vehicle is located;

wherein the determining target obstacles in the occlusion relationship comprises:

acquiring a candidate region corresponding to each of the obstacles based on the region information corresponding to each of the obstacles during the obstacle recognition;

calculating an overlapping ratio between the candidate regions corresponding to each of the obstacles; and

determining that the target obstacles in an occlusion relationship are the obstacles corresponding to the candidate regions with the overlapping ratio greater than an overlapping ratio threshold.

2. The method according to claim 1 , wherein the acquiring the road scene image comprises:

receiving, when the target vehicle is in an automated driving state, the road scene image transmitted by an image capture apparatus corresponding to the target vehicle, the road scene image being an image of the road where the target vehicle is located captured by the image capture apparatus.

3. The method according to claim 1 , wherein the performing obstacle recognition on the road scene image comprises:

inputting the road scene image to an obstacle recognition model, the obstacle recognition model comprising a feature extraction network, an instance segmentation network, and a depth-of-field prediction network;

performing, using the feature extraction network, feature extraction on the road scene image to obtain a plurality of feature maps corresponding to different scales;

performing, using the instance segmentation network, instance segmentation on the plurality of feature maps of different scales, to obtain the region information corresponding to each of the obstacles in the road scene image; and

performing, using the depth-of-field prediction network, depth-of-field prediction on each of the obstacles based on the region information corresponding to each of the obstacles to obtain the depth-of-field information corresponding to each of the obstacles.

4. The method according to claim 3 , wherein the feature extraction network comprises a feature extraction backbone network and a feature pyramid network (FPN), and the performing feature extraction on the road scene image comprises:

inputting the road scene image to the feature extraction backbone network for processing by convolutional network layers of different scales in the feature extraction backbone network, to obtain a plurality of backbone feature maps corresponding to different scales; and

inputting the plurality of backbone feature maps to the FPN for processing by the FPN to obtain a plurality of feature maps corresponding to different scales.

5. The method according to claim 3 , wherein the instance segmentation network comprises a region proposal network (RPN), a target classification network, and a target segmentation network, and the performing instance segmentation on the plurality of feature maps of different scales comprises:

inputting the plurality of feature maps of different scales to the RPN for processing by a convolutional network layer of a preset scale in the RPN to obtain candidate regions corresponding to the feature maps;

predicting, using the target classification network, an obstacle class corresponding to each of the candidate regions; and

performing instance segmentation on the candidate regions based on the predicted obstacle classes of the candidate regions to obtain region information corresponding to each of the obstacles.

6. The method according to claim 1 , wherein the determining the relative depth-of-field relationship between the target obstacles comprises:

sorting the target obstacles by distance based on the depth-of-field information of the target obstacles to obtain a corresponding depth-of-field sorting result; and

determining the relative depth-of-field relationship between the target obstacles based on the depth-of-field sorting result.

7. The method according to claim 1 , wherein the ranging apparatus comprises at least one of a camera or a radar, and the acquiring the ranging result of each of the obstacles comprises:

acquiring ranging data collected by the ranging apparatus corresponding to the target vehicle;

preprocessing the ranging data in a manner matching the ranging apparatus to obtain preprocessed ranging data; and

performing distance estimation on the preprocessed ranging data using a distance prediction model matching the ranging apparatus to obtain the ranging result of each of the obstacles.

8. The method according to claim 1 , wherein the determining the obstacle detection result comprises:

determining a hazard level of each of the obstacles based on the ranging result of each of the obstacles and the relative depth-of-field relationship between the target obstacles; and

determining the obstacle detection result is at least one of the relative depth-of-field relationship between the target obstacles, the ranging result of each of the obstacles, or the hazard level of each of the obstacles.

9. The method according to claim 8 , wherein the determining a hazard level of each of the obstacles comprises:

determining an original hazard level corresponding to each of the obstacles based on the ranging result of each of the obstacles;

correcting the original hazard levels of the target obstacles based on the relative depth-of-field relationship between the target obstacles to obtain corrected hazard levels; and

determining, based on the original hazard levels and the corrected hazard levels of the target obstacles, a hazard level corresponding to each of the obstacles.

10. A computer device comprising at least one processor and at least one memory, the at least one memory storing computer-readable instructions, the computer-readable instructions, when executed by the at least one processor, cause the at least one processor to:

acquire a road scene image of a road where a target vehicle is located;

perform obstacle recognition on the road scene image to obtain region information and depth-of-field information corresponding to obstacles in the road scene image;

determine target obstacles in an occlusion relationship based on the region information corresponding to each of the obstacles, and determine a relative depth-of-field relationship between the target obstacles based on the depth-of-field information of the target obstacles;

acquire a candidate region corresponding to each of the obstacles based on the region information corresponding to each of the obstacles during the obstacle recognition;

calculate an overlapping ratio between the candidate regions corresponding to each of the obstacles;

determine that the target obstacles in an occlusion relationship are the obstacles corresponding to the candidate regions with the overlapping ratio greater than an overlapping ratio threshold;

acquire a ranging result of each of the obstacles collected by a ranging apparatus corresponding to the target vehicle; and

determine, based on the relative depth-of-field relationship between the target obstacles and the ranging result of each of the obstacles, an obstacle detection result corresponding to the road where the target vehicle is located.

11. The computer device according to claim 10 , further configured to:

receive, when the target vehicle is in an automated driving state, the road scene image transmitted by an image capture apparatus corresponding to the target vehicle, the road scene image being an image of the road where the target vehicle is located captured by the image capture apparatus.

12. The computer device according to claim 10 , further configured to:

input the road scene image to an obstacle recognition model, the obstacle recognition model comprising a feature extraction network, an instance segmentation network, and a depth-of-field prediction network;

perform feature extraction on the road scene image using the feature extraction network to obtain a plurality of feature maps corresponding to different scales;

perform instance segmentation on the plurality of feature maps of different scales using the instance segmentation network to obtain the region information corresponding to each of the obstacles in the road scene image; and

perform depth-of-field prediction on each of the obstacles based on the region information corresponding to each of the obstacles by using the depth-of-field prediction network to obtain the depth-of-field information corresponding to each of the obstacles.

13. The computer device according to claim 12 , wherein the feature extraction network comprises a feature extraction backbone network and a feature pyramid network (FPN), and the computer device is further configured to:

input the road scene image to the feature extraction backbone network for processing by convolutional network layers of different scales in the feature extraction backbone network to obtain a plurality of backbone feature maps corresponding to different scales; and

input the plurality of backbone feature maps to the FPN for processing by the FPN to obtain a plurality of feature maps corresponding to different scales.

14. The computer device according to claim 12 , wherein the instance segmentation network comprises a region proposal network (RPN), a target classification network, and a target segmentation network, and computer device is further configured to:

input the plurality of feature maps of different scales to the RPN for processing by a convolutional network layer of a preset scale in the RPN to obtain candidate regions corresponding to the feature maps;

predict, using the target classification network, an obstacle class corresponding to each of the candidate regions; and

perform instance segmentation on the candidate regions based on the predicted obstacle classes of the candidate regions to obtain region information corresponding to each of the obstacles.

15. The computer device according to claim 10 , further configured to:

sort the target obstacles by distance based on the depth-of-field information of the target obstacles to obtain a corresponding depth-of-field sorting result; and

determine the relative depth-of-field relationship between the target obstacles based on the depth-of-field sorting result.

16. The computer device according to claim 10 , wherein the ranging apparatus comprises at least one of a camera or a radar, and the computer device is further configured to:

acquire ranging data collected by the ranging apparatus corresponding to the target vehicle;

preprocess the ranging data in a manner matching the ranging apparatus to obtain preprocessed ranging data; and

perform distance estimation on the preprocessed ranging data using a distance prediction model matching the ranging apparatus to obtain the ranging result of each of the obstacles.

17. The computer device according to claim 10 , further configured to:

determine a hazard level of each of the obstacles based on the ranging result of each of the obstacles and the depth-of-field relationship between the obstacles; and

determine the obstacle detection result is at least one of the relative depth-of-field relationship between the target obstacles, the ranging result of each of the obstacles, or the hazard level of each of the obstacles.

18. The computer device according to claim 17 , further configured to:

determine an original hazard level corresponding to each of the obstacles based on the ranging result of each of the obstacles;

correct the original hazard levels of the target obstacles based on the relative depth-of-field relationship between the target obstacles to obtain corrected hazard levels; and

determine, based on the original hazard levels and the corrected hazard levels of the target obstacles, a hazard level corresponding to each of the obstacles.

19. A non-transitory computer readable storage medium storing at least one instruction, at least one program, a code set, or an instruction set, the at least one instruction, the at least one program, the code set, or the instruction set being loaded and executed by at least one processor, causing the at least one processor to perform an obstacle detection method comprising:

acquiring a road scene image of a road where a target vehicle is located;

performing obstacle recognition on the road scene image to obtain region information and depth-of-field information corresponding to obstacles in the road scene image;

determining target obstacles in an occlusion relationship based on the region information corresponding to each of the obstacles, and determining a relative depth-of-field relationship between the target obstacles based on the depth-of-field information of the target obstacles;

acquiring a ranging result of each of the obstacles collected by a ranging apparatus corresponding to the target vehicle; and

determining, based on the relative depth-of-field relationship between the target obstacles and the ranging result of each of the obstacles, an obstacle detection result corresponding to the road where the target vehicle is located;

wherein the determining target obstacles in the occlusion relationship comprises:

acquiring a candidate region corresponding to each of the obstacles based on the region information corresponding to each of the obstacles during the obstacle recognition;

calculating an overlapping ratio between the candidate regions corresponding to each of the obstacles; and

determining that the target obstacles in an occlusion relationship are the obstacles corresponding to the candidate regions with the overlapping ratio greater than an overlapping ratio threshold.

20. The non-transitory computer readable storage medium of claim 19 , the obstacle detection method further comprising:

sorting the target obstacles by distance based on the depth-of-field information of the target obstacles to obtain a corresponding depth-of-field sorting result; and

determining the relative depth-of-field relationship between the target obstacles based on the depth-of-field sorting result.

Assignments (1)

Priority Claims (1)

Continuity (2)

References Cited (75)