Obstacle detection method and device, apparatus, and storage medium

1. An obstacle detection method, comprising:

acquiring position information of a two-dimensional detection frame and position information of a three-dimensional detection frame of an obstacle in an image;

converting the position information of the three-dimensional detection frame of the obstacle into position information of a two-dimensional projection frame of the obstacle; and

optimizing the position information of the three-dimensional detection frame of the obstacle by using the position information of the two-dimensional detection frame, the position information of the three-dimensional detection frame and the position information of the two-dimensional projection frame of the obstacle in the image,

wherein the optimizing the position information of the three-dimensional detection frame of the obstacle by using the position information of the two-dimensional detection frame, the position information of the three-dimensional detection frame and the position information of the two-dimensional projection frame of the obstacle in the image comprises:

setting at least one constraint item for the position information of the three-dimensional detection frame of the obstacle by using the position information of the two-dimensional detection frame, the position information of the three-dimensional detection frame and the position information of the two-dimensional projection frame of the obstacle in the image;

constructing a loss function by using the constraint item; and

optimizing the position information of the three-dimensional detection frame of the obstacle by using the loss function,

wherein the constraint item comprises a first constraint item, which is set according to coordinates of a central point of the two-dimensional detection frame and coordinates of a central point of the two-dimensional projection frame of the obstacle.

2. The obstacle detection method according to claim 1 , wherein the constraint item further comprises a second constraint item, which is set according to coordinates of left and right sides of the two-dimensional detection frame and coordinates of left and right sides of the two-dimensional projection frame of the obstacle.

3. The obstacle detection method according to claim 1 , wherein the constraint item further comprises a third constraint item, which is set according to coordinates of projection points of the three-dimensional detection frame of the obstacle.

4. The obstacle detection method according to claim 1 , wherein the optimizing the position information of the three-dimensional detection frame of the obstacle by using the loss function comprises:

calculating initial loss, by using an initial prediction result of the three-dimensional detection frame of the obstacle and the loss function, as a reference loss;

setting an initial solution space range according to a prediction error;

performing random sampling m times within the initial solution space range, to obtain a set of position information of three-dimensional detection frames associated with sample points, wherein m is an integer greater than 0;

calculating loss associated with the sample points according to the loss function; and

comparing the loss associated with the sample points and the reference loss, to obtain a sample point with minimum loss, and taking position information of a three-dimensional detection frame associated with the sample point with the minimum loss as an alternative result, and taking loss associated with the sample point with the minimum loss as an updated reference loss.

5. The obstacle detection method according to claim 4 , wherein the optimizing the position information of the three-dimensional detection frame of the obstacle by using the loss function further comprises performing following operations repeatedly for n−1 times, wherein n represents a number of times resampling is performed:

determining statistically a distribution range of sample points with loss less than the reference loss from m sample points resampled last time, to obtain an updated solution space range;

performing random sampling m times within the updated solution space range, to obtain a set of position information of three-dimensional detection frames of the obstacle associated with sample points, wherein m is an integer greater than 0;

calculating loss associated with the sample points according to the loss function; and

comparing the loss associated with the sample points and the reference loss, to obtain a sample point with minimum loss, and taking position information of a three-dimensional detection frame associated with the sample point with the minimum loss as an alternative result, and taking loss associated with the sample point with the minimum loss as an updated reference loss.

6. The obstacle detection method according to claim 1 , wherein the optimizing the position information of the three-dimensional detection frame of the obstacle by using the loss function further comprises performing following operations repeatedly for n−1 times, wherein n represents a number of times resampling is performed:

determining statistically a distribution range of sample points with loss less than the reference loss from m sample points resampled last time, to obtain an updated solution space range;

performing random sampling m times within the updated solution space range, to obtain a set of position information of three-dimensional detection frames of the obstacle associated with sample points, wherein m is an integer greater than 0;

calculating loss associated with the sample points according to the loss function; and

comparing the loss associated with the sample points and the reference loss, to obtain a sample point with minimum loss, and taking position information of a three-dimensional detection frame associated with the sample point with the minimum loss as an alternative result, and taking loss associated with the sample point with the minimum loss as an updated reference loss.

7. An obstacle detection device, comprising:

at least one processor; and

a memory communicatively connected to the at least one processor, wherein

the memory stores instructions executable by the at least one processor, the instructions are executed by the at least one processor to enable the at least one processor to:

acquire position information of a two-dimensional detection frame and position information of a three-dimensional detection frame of an obstacle in an image;

convert the position information of the three-dimensional detection frame of the obstacle into position information of a two-dimensional projection frame of the obstacle; and

optimize the position information of the three-dimensional detection frame of the obstacle by using the position information of the two-dimensional detection frame, the position information of the three-dimensional detection frame and the position information of the two-dimensional projection frame of the obstacle in the image,

wherein the instructions are executed by the at least one processor to further enable the at least one processor to:

set at least one constraint item for the position information of the three-dimensional detection frame of the obstacle by using the position information of the two-dimensional detection frame, the position information of the three-dimensional detection frame and the position information of the two-dimensional projection frame of the obstacle in the image;

construct a loss function by using the constraint item; and

optimize the position information of the three-dimensional detection frame of the obstacle by using the loss function,

wherein the constraint item comprises a first constraint item, which is set according to coordinates of a central point of the two-dimensional detection frame and coordinates of a central point of the two-dimensional projection frame of the obstacle.

8. The obstacle detection device according to claim 7 , wherein the constraint item further comprises a second constraint item, which is set according to coordinates of left and right sides of the two-dimensional detection frame and coordinates of left and right sides of the two-dimensional projection frame of the obstacle.

9. The obstacle detection device according to claim 7 , wherein the constraint item further comprises a third constraint item, which is set according to coordinates of projection points of the three-dimensional detection frame of the obstacle.

10. The obstacle detection device according to claim 7 , wherein the instructions are executed by the at least one processor to further enable the at least one processor to:

calculate initial loss, by using an initial prediction result of the three-dimensional detection frame of the obstacle and the loss function, as a reference loss;

set an initial solution space range according to a prediction error;

perform random sampling m times within the initial solution space range, to obtain a set of position information of three-dimensional detection frames associated with sample points, wherein m is an integer greater than 0;

calculate loss associated with the sample points according to the loss function; and

compare the loss associated with the sample points and the reference loss, to obtain a sample point with minimum loss, and take position information of a three-dimensional detection frame associated with the sample point with the minimum loss as an alternative result, and take loss associated with the sample point with the minimum loss as an updated reference loss.

11. The obstacle detection device according to claim 10 , wherein the instructions are executed by the at least one processor to further enable the at least one processor to perform following operations repeatedly for n−1 times, wherein n represents a number of times resampling is performed:

determining statistically a distribution range of sample points with loss less than the reference loss from m sample points resampled last time, to obtain an updated solution space range;

performing random sampling m times within the updated solution space range, to obtain a set of position information of three-dimensional detection frames of the obstacle associated with sample points, wherein m is an integer greater than 0;

calculating loss associated with the sample points according to the loss function; and

comparing the loss associated with the sample points and the reference loss, to obtain a sample point with minimum loss, and taking position information of a three-dimensional detection frame associated with the sample point with the minimum loss as an alternative result, and taking loss associated with the sample point with the minimum loss as an updated reference loss.

12. The obstacle detection device according to claim 7 , wherein the instructions are executed by the at least one processor to further enable the at least one processor to perform following operations repeatedly for n−1 times, wherein n represents a number of times resampling is performed:

determining statistically a distribution range of sample points with loss less than the reference loss from m sample points resampled last time, to obtain an updated solution space range;

performing random sampling m times within the updated solution space range, to obtain a set of position information of three-dimensional detection frames of the obstacle associated with sample points, wherein m is an integer greater than 0;

calculating loss associated with the sample points according to the loss function; and

comparing the loss associated with the sample points and the reference loss, to obtain a sample point with minimum loss, and taking position information of a three-dimensional detection frame associated with the sample point with the minimum loss as an alternative result, and taking loss associated with the sample point with the minimum loss as an updated reference loss.

13. A non-transitory computer readable storage medium for storing computer instructions, wherein the computer instructions, when executed by a computer, cause the computer to

acquire position information of a two-dimensional detection frame and position information of a three-dimensional detection frame of an obstacle in an image;

convert the position information of the three-dimensional detection frame of the obstacle into position information of a two-dimensional projection frame of the obstacle; and

optimize the position information of the three-dimensional detection frame of the obstacle by using the position information of the two-dimensional detection frame, the position information of the three-dimensional detection frame and the position information of the two-dimensional projection frame of the obstacle in the image,

wherein the computer instructions, when executed by a computer, further cause the computer to:

set at least one constraint item for the position information of the three-dimensional detection frame of the obstacle by using the position information of the two-dimensional detection frame, the position information of the three-dimensional detection frame and the position information of the two-dimensional projection frame of the obstacle in the image;

construct a loss function by using the constraint item; and

optimize the position information of the three-dimensional detection frame of the obstacle by using the loss function,

wherein the constraint item comprises a first constraint item, which is set according to coordinates of a central point of the two-dimensional detection frame and coordinates of a central point of the two-dimensional projection frame of the obstacle.

14. The non-transitory computer-readable storage medium according to claim 13 , wherein the constraint item further comprises a second constraint item, which is set according to coordinates of left and right sides of the two-dimensional detection frame and coordinates of left and right sides of the two-dimensional projection frame of the obstacle.

15. The non-transitory computer-readable storage medium according to claim 13 , wherein the constraint item further comprises a third constraint item, which is set according to coordinates of projection points of the three-dimensional detection frame of the obstacle.

16. The non-transitory computer-readable storage medium according to claim 13 , wherein the computer instructions, when executed by a computer, further cause the computer to:

calculate initial loss, by using an initial prediction result of the three-dimensional detection frame of the obstacle and the loss function, as a reference loss;

set an initial solution space range according to a prediction error;

perform random sampling m times within the initial solution space range, to obtain a set of position information of three-dimensional detection frames associated with sample points, wherein m is an integer greater than 0;

calculate loss associated with the sample points according to the loss function; and

compare the loss associated with the sample points and the reference loss, to obtain a sample point with minimum loss, and take position information of a three-dimensional detection frame associated with the sample point with the minimum loss as an alternative result, and take loss associated with the sample point with the minimum loss as an updated reference loss.

17. The non-transitory computer-readable storage medium according to claim 16 , wherein the computer instructions, when executed by a computer, further cause the computer to perform following operations repeatedly for n−1 times, wherein n represents a number of times resampling is performed:

determining statistically a distribution range of sample points with loss less than the reference loss from m sample points resampled last time, to obtain an updated solution space range;

performing random sampling m times within the updated solution space range, to obtain a set of position information of three-dimensional detection frames of the obstacle associated with sample points, wherein m is an integer greater than 0;

calculating loss associated with the sample points according to the loss function; and

comparing the loss associated with the sample points and the reference loss, to obtain a sample point with minimum loss, and taking position information of a three-dimensional detection frame associated with the sample point with the minimum loss as an alternative result, and taking loss associated with the sample point with the minimum loss as an updated reference loss.

18. The non-transitory computer-readable storage medium according to claim 13 , wherein the computer instructions, when executed by a computer, further cause the computer to perform following operations repeatedly for n−1 times, wherein n represents a number of times resampling is performed:

determining statistically a distribution range of sample points with loss less than the reference loss from m sample points resampled last time, to obtain an updated solution space range;

performing random sampling m times within the updated solution space range, to obtain a set of position information of three-dimensional detection frames of the obstacle associated with sample points, wherein m is an integer greater than 0;

calculating loss associated with the sample points according to the loss function; and

comparing the loss associated with the sample points and the reference loss, to obtain a sample point with minimum loss, and taking position information of a three-dimensional detection frame associated with the sample point with the minimum loss as an alternative result, and taking loss associated with the sample point with the minimum loss as an updated reference loss.