Granted Patent B2

US 12,403,936 · App. 18/043,449 · Granted Sep 2, 2025

Inventors: Nikhil Kapoor (Wolfsburg, DE); Jan David Schneider (Wolfsburg, DE); Serin Varghese (Braunschweig, DE)

Assignee: VOLKSWAGEN AKTIENGESELLSCHAFT

B60W60/0015B60W50/0098B60W50/14B60W60/0053G06F18/2131G06F21/56B60W2420/403B60W2420/408G06N3/08

Abstract

Claims (69)

1. A method for controlling a driver assistance system during operation of a vehicle, wherein the driver assistance system comprises at least one sensor for observing an environment of the vehicle and an electronic control unit using a neural network for analyzing sensor data of the at least one sensor and providing perception tasks for the vehicle based on the analyzed sensor data, wherein the sensor data comprises one or more of optical, acoustical, and electromagnetic data, the method comprising:

providing a data set of the sensor data by the at least one sensor in a spatial domain;

transforming the data set of the sensor data by the electronic control unit using frequency analysis into a frequency spectrum in a frequency domain; and

analyzing the frequency spectrum of the data set in order to detect an adversarially attacked data set; wherein

wherein in response to an adversarially attacked data set being detected from analyzing the frequency spectrum, at least the following is conducted: selectively activating a security mechanism against an adversarial attack;

wherein the security mechanism comprises one or more of:

activating a pre-processing technique to remove adversarial attack from the data set;

using a denoising filter for cleaning the adversarially attacked data set;

using a hardened neural network for analyzing the adversarially attacked data set;

using multiple other neural networks with different architectures in an ensemble and performing a consolidation of their output for analyzing the adversarially attacked data set;

applying rule-based technique for analyzing the adversarially attacked data set, especially using external, human-readable devices; and

requesting a user of the vehicle for analyzing the adversarially attacked data set; and

wherein the neural network or a special detecting neural network used for detecting the adversarially attacked data set are trained in the frequency domain on frequency spectrums of clean data sets and on frequency spectrums of prepared adversarially attacked data sets in order to learn differences between the frequency spectrums of the clean data sets and the frequency spectrums of the adversarially attacked data sets in the frequency domain.

2. The method of claim 1 , wherein providing the data set of the sensor data by the sensor in the spatial domain, transforming the data set of the sensor data by the electronic control unit using frequency analysis into the frequency spectrum in the frequency domain, and analyzing the frequency spectrum of the data set in order to detect the adversarially attacked data set are executed before analyzing the sensor data for providing perception tasks for the vehicle.

3. The method of claim 1 , wherein the sensor data comprise camera, lidar, and/or radar data.

4. The method of claim 1 , wherein the frequency analysis is provided by one or more of a discrete Fourier transformation (DFT), a discrete cosine transformation, and Wavelet transformation.

5. The method of claim 1 , wherein for detecting the adversarially attacked data set, the neural network is used, which is also used for analyzing the sensor data of the sensor and for providing perception tasks for the vehicle.

6. The method of claim 1 , wherein for detecting the adversarially attacked data set, a denoising filter is used for denoising the frequency spectrum of the data set in the frequency domain; wherein the denoising filter is built in the frequency domain using frequency spectrums of clean data sets and noise spectrums of prepared adversarially attacked data sets.

7. The method of claim 1 , wherein for detecting the adversarially attacked data set, the frequency spectrum of the data set are compared with a denoised frequency spectrum; wherein a threshold is applied by comparing the frequency spectrum of the data set with the denoised frequency spectrum; wherein the attacked data sets will be detected in response to a difference between the frequency spectrum of the data set and the denoised frequency spectrum is greater than the applied threshold.

8. The method of claim 1 , wherein when an adversarially attacked data set is detected, at least the following conducted: initiating an emergency maneuver.

9. The method of claim 8 , wherein the emergency maneuver comprises one or more of: reducing vehicle speed, initiating an emergency stop, providing an alarm inside and/or outside of the vehicle, activating a manual driving mode, and deactivate the driver assistance system.

10. An electronic control unit, comprising:

a memory, in which a program code is stored; and

a processor, wherein when executing the program code by the processor, the processor is configured to at least:

provide a data set of sensor data in a spatial domain, wherein the sensor data comprises one or more of optical, acoustical, and electromagnetic data;

transform the data set of the sensor data using frequency analysis into a frequency spectrum in a frequency domain; and

analyze the frequency spectrum of the data set in order to detect an adversarially attacked data set; wherein

wherein in response to an adversarially attacked data set being detected from analyzing the frequency spectrum, at least the following is conducted: selectively activating a security mechanism against an adversarial attack;

wherein the security mechanism comprises one or more of:

activating a pre-processing technique to remove adversarial attack from the data set;

using a denoising filter for cleaning the adversarially attacked data set;

using a hardened neural network for analyzing the adversarially attacked data set;

using multiple other neural networks with different architectures in an ensemble and performing a consolidation of their output for analyzing the adversarially attacked data set;

applying rule-based technique for analyzing the adversarially attacked data set, especially using external, human-readable devices; and

requesting a user of the vehicle for analyzing the adversarially attacked data set; and

wherein the neural network and/or a special detecting neural network used for detecting the adversarially attacked data set are trained in the frequency domain on frequency spectrums of clean data sets and on frequency spectrums of prepared adversarially attacked data sets in order to learn differences between the frequency spectrums of the clean data sets and the frequency spectrums of the adversarially attacked data sets in the frequency domain.

11. A vehicle comprising an electronic control unit, the electronic control unit comprising:

a memory, in which a program code is stored; and

a processor, wherein when executing the program code by the processor, the processor is configured to at least:

provide a data set of sensor data in a spatial domain, wherein the sensor data comprises one or more of optical, acoustical, and electromagnetic data;

transform the data set of the sensor data using frequency analysis into a frequency spectrum in a frequency domain; and

analyze the frequency spectrum of the data set in order to detect an adversarially attacked data set; wherein

wherein in response to an adversarially attacked data set being detected from analyzing the frequency spectrum, at least the following is conducted: selectively activating a security mechanism against an adversarial attack;

wherein the security mechanism comprises one or more of:

activating a pre-processing technique to remove adversarial attack from the data set;

using a denoising filter for cleaning the adversarially attacked data set;

using a hardened neural network for analyzing the adversarially attacked data set;

using multiple other neural networks with different architectures in an ensemble and performing a consolidation of their output for analyzing the adversarially attacked data set;

applying rule-based technique for analyzing the adversarially attacked data set, especially using external, human-readable devices; and

requesting a user of the vehicle for analyzing the adversarially attacked data set; and

wherein the neural network or a special detecting neural network used for detecting the adversarially attacked data set are trained in the frequency domain on frequency spectrums of clean data sets and on frequency spectrums of prepared adversarially attacked data sets in order to learn differences between the frequency spectrums of the clean data sets and the frequency spectrums of the adversarially attacked data sets in the frequency domain.

12. A non-transitory storage medium comprising instructions for a processor, which instructions, when executed by the processor, cause the processor to:

provide a data set of the sensor data by the sensor in a spatial domain, wherein the sensor data comprises one or more of optical, acoustical, and electromagnetic data;

transform the data set of the sensor data by the electronic control unit using frequency analysis into a frequency spectrum in a frequency domain; and

analyze the frequency spectrum of the data set in order to detect an adversarially attacked data set; wherein

wherein in response to an adversarially attacked data set being detected from analyzing the frequency spectrum, at least the following is conducted: selectively activating a security mechanism against an adversarial attack;

wherein the security mechanism comprises one or more of:

activating a pre-processing technique to remove adversarial attack from the data set;

using a denoising filter for cleaning the adversarially attacked data set;

using a hardened neural network for analyzing the adversarially attacked data set;

using multiple other neural networks with different architectures in an ensemble and performing a consolidation of their output for analyzing the adversarially attacked data set;

applying rule-based technique for analyzing the adversarially attacked data set, especially using external, human-readable devices; and

requesting a user of the vehicle for analyzing the adversarially attacked data set; and

wherein the neural network or a special detecting neural network used for detecting the adversarially attacked data set are trained in the frequency domain on frequency spectrums of clean data sets and on frequency spectrums of prepared adversarially attacked data sets in order to learn differences between the frequency spectrums of the clean data sets and the frequency spectrums of the adversarially attacked data sets in the frequency domain.

13. The method of claim 1 , wherein providing the data set of the sensor data by the sensor in the spatial domain, transforming the data set of the sensor data by the electronic control unit using frequency analysis into the frequency spectrum in the frequency domain, and analyzing the frequency spectrum of the data set in order to detect the adversarially attacked data set are executed periodically.

14. The method of claim 1 , wherein providing the data set of the sensor data by the sensor in the spatial domain, transforming the data set of the sensor data by the electronic control unit using frequency analysis into the frequency spectrum in the frequency domain, and analyzing the frequency spectrum of the data set in order to detect the adversarially attacked data set are executed for each data set of the sensor data during operation of the vehicle.

15. The method of claim 1 , wherein the data set is provided as a picture of the environment of the vehicle.

16. The method of claim 1 , wherein the electronic control unit uses a deep neural network as the neural network for analyzing the sensor data of the sensor and for providing perception tasks for the vehicle based on the analyzed sensor data.

17. The method of claim 1 , wherein for detecting the adversarially attacked data set, a special detecting neural network is used.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (25)