Granted Patent B2

US 12,254,683 · App. 17/439,302 · Granted Mar 18, 2025

Inventors: Anton Bruch (Berlin, DE); Fabian Diegmann (Ingolstadt, DE)

Assignee: CARIAD SE

G06V10/82G06V10/764G06V20/588G06V20/58

Abstract

Claims (52)

1. A method for determining a source of danger on a roadway in a detection area in front of or behind a vehicle, with the aid of a camera of the vehicle, in which with the aid of the camera an image of the detection area is captured, the method comprising:

generating image data corresponding to the image;

determining a first image area of the image with the aid of the image data using a neural network, the first image area corresponding to substantially only the roadway in the detection area specifically designed for vehicle traffic; and

determining a second image area of the image with the aid of the first image area using the neural network or a further neural network, the second image area corresponding to the source of danger on the roadway in the detection area.

2. The method of claim 1 , further comprising, with the aid of the first image area, using an output information of an intermediate layer of the neural network, determining the second image area, the neural network comprising at least an encoder network and a decoder network.

3. The method of claim 1 , further comprising:

with the aid of the image data, using the neural network, for each pixel of the image, determining one first feature vector; and

with the aid of these first feature vectors, determining the first image area.

4. The method of claim 1 , further comprising:

determining second features vectors using the neural network for the first image area;

determining a mean value of the second feature vectors assigned to the pixels of the first image area; and

using this mean value and the second feature vectors, determining the second image area.

5. The method of claim 1 , further comprising:

with the aid of the image data using the neural network, for each pixel of the image, determining one first feature vector;

with the aid of these first feature vectors, determining the first image;

with the aid of the neural network, determining second feature vectors for the first image area;

assigning a mean value of the second feature vectors to the pixels of the first image area;

for each of the second feature vectors, determining the difference between the mean value of all second feature vectors forming the first image area and the respective second feature vector; and

when the value of this difference is above, below, or on a predetermined threshold value, assigning a predetermined subarea of the image assigned to the respective second feature vector to the second image area.

6. The method of claim 1 , further comprising, with the aid of the first image area, using an output information of an encoder network of the neural network, determining the second image area.

7. The method of claim 1 , further comprising, with the aid of the image data using the neural network pixels of the image assigned to the first area are determined.

8. The method of claim 1 , wherein the neural network is a convolutional neural network.

9. The method of claim 1 , further comprising training the neural network with the aid of images of real traffic situations that have been captured with the aid of the camera of the vehicle or a camera of a further vehicle.

10. The method of claim 1 , further comprising, with the aid of the image data, determining an enveloping body surrounding the source of danger.

11. The method of claim 1 , further comprising classifying the source of danger with the aid of image data using an image recognition method performed in a further neural network.

12. A device for determining a source of danger on a roadway in a detection area in front of or behind a vehicle, the device comprising:

a camera configured to capture an image of the detection area; and

an image processing and evaluation unit configured to generate image data corresponding to the image to:

determine a first image area of the image with the aid of the image data using a neural network, the first image area corresponding to substantially only the roadway in the detection area specifically designed for vehicle traffic; and

determine a second image area of the image with the aid of the first image area using the neural network or a further neural network, the second image area corresponding to the source of danger on the roadway in the detection area.

13. A computer-readable medium, wherein the computer-readable medium does not constitute a transitory data signal, having contents configured to cause a computing system to determine a source of danger on a roadway in a detection area in front of or behind a vehicle, with the aid of a camera of the vehicle, in which with the aid of the camera an image of the detection area is captured, wherein, to determine the source of danger, the computing system is caused to:

generate image data corresponding to the image;

determine a first image area of the image with the aid of the image data using a neural network, the first image area corresponding to substantially only the roadway in the detection area specifically designed for vehicle traffic; and

determine a second image area of the image with the aid of the first image area using the neural network or a further neural network, the second image area corresponding to the source of danger on the roadway in the detection area.

14. The computer-readable medium of claim 13 , wherein the computing system is further caused to, with the aid of the first image area, use an output information of an intermediate layer of the neural network to determine the second image area, the neural network comprising at least an encoder network and a decoder network.

15. The computer-readable medium of claim 13 , wherein the computing system is further caused to:

with the aid of the image data, using the neural network, for each pixel of the image, to determine one first feature vector; and

with the aid of these first feature vectors, determine the first image area.

16. The computer-readable medium of claim 13 , wherein the computing system is further caused to:

determine second features vectors using the neural network for the first image area;

determine a mean value of the second feature vectors assigned to the pixels of the first image area; and

use this mean value and the second feature vectors, determining the second image area.

17. The computer-readable medium of claim 13 , wherein the computing system is further caused to:

with the aid of the image data using the neural network, for each pixel of the image, determine one first feature vector;

with the aid of these first feature vectors, determine the first image;

with the aid of the neural network, determine second feature vectors for the first image area;

assign a mean value of the second feature vectors to the pixels of the first image area;

for each of the second feature vectors, determine the difference between the mean value of all second feature vectors forming the first image area and the respective second feature vector; and

when the value of this difference is above, below, or on a predetermined threshold value, assign a predetermined subarea of the image assigned to the respective second feature vector to the second image area.

18. The computer-readable medium of claim 13 , wherein the computing system is further caused to, with the aid of the first image area, use an output information of an encoder network of the neural network to determine the second image area.

19. The computer-readable medium of claim 13 , wherein the computing system is further caused to, with the aid of the image data, use the neural network to determine pixels of the image assigned to the first area.

20. The computer-readable medium of claim 13 , wherein the computing system is further caused to classify the source of danger with the aid of image data using an image recognition method performed in a further neural network.

Assignments (3)

Priority Claims (1)

Continuity (1)

References Cited (25)