Methods, systems and computer programs for processing image data for generating a filter
A method, system and computer program for processing image data, to a vehicle comprising such a system, and to a method, system and computer program for generating a filter. Image data processing may include obtaining the image data, and applying a filter on the image data to generate filtered image data, the filter being configured to suppress adversarial perturbations within the image data. The filtered image data is processed using a machine-learning model.
1 . A method for processing image data, comprising:
obtaining image data from a camera sensor of a vehicle;
transforming the image data into a frequency domain;
applying a filter to the transformed image data in the frequency domain to generate filtered image data, the filter comprising a Wiener filter configured to suppress adversarial perturbations within the image data occurring according to one or more regular patterns in the frequency domain to improve robustness of a machine-learning model against adversarial perturbations;
transforming the filtered image data into the spatial domain; and
processing the transformed filtered image data using a machine-learning model to perform object detection or image segmentation for a driving assistance feature of the vehicle, wherein the filtered image data is input to the machine-learning model after the filtered image data is transformed into the spatial domain.
2 . The method according to claim 1 , wherein applying the filter comprises multiplying the filter with the image data in the frequency domain, and suppressing a subset of frequencies that are indicative of adversarial perturbations.
3 . The method according to claim 2 , wherein the subset of frequencies that are indicative of adversarial perturbations, and suppressed by the filter, occur according to one or more regular patterns in the frequency domain.
4 . The method according to claim 1 , wherein the filter is based on a plurality of types of adversarial perturbations.
5 . The method according to claim 1 , further comprising triggering an alert in the vehicle's autonomous or semi-autonomous driving assistance system in response to a detected presence of adversarial perturbations to improve vehicle safety.
6 . The method according to claim 1 , further comprising detecting a presence of adversarial perturbations based on a comparison between the image data and the filtered image data.
7 . The method according to claim 1 , wherein the machine-learning model comprises a deep neural network.
8 . A system for processing image data, comprising:
an interface for obtaining image data from a camera sensor of a vehicle;
a memory, operatively coupled to the interface; and
a processing apparatus, operatively coupled to the memory, wherein the processing apparatus is configured to
transform the image data into a frequency domain;
apply a filter to the transformed image data in the frequency domain to generate filtered image data, the filter comprising a Wiener filter configured to suppress adversarial perturbations within the image data; transform the filtered image data into the spatial domain; and process the transformed filtered image data using a machine-learning model, wherein the filtered image data is input to the machine-learning model after the filter image data is transformed into the spatial domain.
9 . The system according to claim 8 , wherein the processing apparatus is configured to apply the filter by multiplying the filter with the image data in the frequency domain, and suppressing a subset of frequencies that are indicative of adversarial perturbations.
10 . The system according to claim 9 , wherein the subset of frequencies that are indicative of adversarial perturbations, and suppressed by the filter, occur according to one or more regular patterns in the frequency domain.
11 . The system according to claim 8 , wherein the filter is based on a plurality of types of adversarial perturbations.
12 . The system according to claim 8 , further comprising triggering an alert in the vehicle's autonomous or semi-autonomous driving assistance system in response to a detected presence of adversarial perturbations to improve vehicle safety.
13 . The system according to claim 8 , wherein the processing apparatus is configured to detect a presence of adversarial perturbations based on a comparison between the image data and the filtered image data.
14 . The system according to claim 8 , wherein the machine-learning model comprises a deep neural network.
15 . A non-transitory computer-readable medium storing instructions that, when executed by one or more processors of a vehicle system, cause the one or more processors to:
obtain image data from a camera sensor of the vehicle;
transform the image data into a frequency domain;
apply a filter to the transformed image data in the frequency domain to generate filtered image data, the filter comprising a Wiener filter configured to suppress adversarial perturbations within the image data occurring according to one or more regular patterns in the frequency domain to improve robustness of a machine-learning model against adversarial perturbations;
transform the filtered image data into the spatial domain; and
process the transformed filtered image data using the machine-learning model to perform object detection or image segmentation for a driving-assistance feature of the vehicle,
wherein the filtered image data is input to the machine-learning model after the filtered image data is transformed into the spatial domain.
16 . The computer-readable medium according to claim 15 , wherein the instructions further cause the one or more processors to select, from a plurality of predefined parameter sets, a parameter set for the Wiener filter based on at least one of a vehicle speed, ambient illumination, or sensor-noise level.
17 . The computer-readable medium according to claim 16 , wherein the instructions further cause the one or more processors to monitor a confidence level of an output of the machine-learning model and to select a different parameter set for the Wiener filter in response to the confidence level falling below a threshold.
18 . The computer-readable medium according to claim 15 , wherein applying the filter comprises weighting suppression of frequencies associated with the adversarial perturbations according to respective amplitudes of the frequencies in the transformed image data.
19 . The computer-readable medium according to claim 15 , wherein the image data comprises at least one of camera, radar, or lidar sensor data of the vehicle, and the filter is applied to the sensor data transformed into a frequency domain corresponding to the respective sensor type.
20 . The computer-readable medium according to claim 15 , wherein the instructions further cause the one or more processors to trigger a diagnostic alert in the vehicle's driving-assistance controller when a difference between the image data and the filtered image data exceeds a threshold.