IP Library › Granted Patent US 12,651,168
Granted Patent B2
US 12,651,168 · App. 17/613,158 · Granted Jun 9, 2026

Method for operating a deep neural network

Inventors: Peter Schlicht (Wolfsburg, DE); Nico Maurice Schmidt (Berlin, DE)
Assignee: VOLKSWAGEN AKTIENGESELLSCHAFT
G06N3/084G06N3/04
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,651,168
App. No.
17/613,158
Granted
Jun 9, 2026
Kind
B2
Abstract

The invention relates to a method for operating a deep neural network, wherein the deep neural network is operated with multiple layers between an input layer and an output layer, and wherein, in addition, at least one classic filter is used in the deep neural network between the input and the output layer. The invention also relates to a device for data processing and to a computer-readable storage medium.

Claims (28)

1 . A method for operating a deep neural network, comprising:

operating the deep neural network with multiple layers between an input layer and an output layer, wherein at least one classic filter is used in the deep neural network between the input layer and the output layer in parallel to at least one filter channel;

conducting a training phase of the deep neural network, wherein filter parameters of the at least one classic filter are selected on the basis of empirical values and fixed during the training phase of the deep neural network, and wherein filter parameters of the at least one filter channel are randomly initialized;

feeding output data from the at least one classic filter to a backpropagation network of the deep neural network in addition feature maps extracted by the at least one filter channel; and

conducting an inference phase of the deep neural network for the operation of a driver assistance system of a vehicle, wherein the filter parameters of the at least one classic filter remain fixed during the inference phase of the deep neural network.

2 . The method of claim 1 , comprising: conducting a training phase of the deep neural network, wherein at least one part of filter parameters of the at least one classic filter is changed during the training phase of the deep neural network.

3 . The method of claim 2 , wherein at least one part of the filter parameters of the at least one classic filter is adapted with a lower learning rate than the remaining deep neural network.

4 . The method of claim 1 , wherein the at least one classic filter is operated directly after the input layer and/or in the vicinity of the input layer of the deep neural network.

5 . The method of claim 1 , wherein an output of the at least one classic filter is fed to multiple layers of the deep neural network.

6 . The method of claim 1 , wherein the deep neural network is a convolutional neural network.

7 . The method of claim 2 , wherein the at least one classic filter is operated directly after the input layer and/or in the vicinity of the input layer of the deep neural network.

8 . The method of claim 3 , wherein the at least one classic filter is operated directly after the input layer and/or in the vicinity of the input layer of the deep neural network.

9 . The method of claim 2 , wherein an output of the at least one classic filter is fed to multiple layers of the deep neural network.

10 . The method of claim 3 , wherein an output of the at least one classic filter is fed to multiple layers of the deep neural network.

11 . The method of claim 4 , wherein an output of the at least one classic filter is fed to multiple layers of the deep neural network.

12 . The method of claim 2 , wherein the deep neural network is a convolutional neural network.

13 . The method of claim 3 , wherein the deep neural network is a convolutional neural network.

14 . The method of claim 4 , wherein the deep neural network is a convolutional neural network.

15 . A device for operating a deep neural network, the device having a processor configured to:

operate the deep neural network with multiple layers between an input layer and an output layer, wherein at least one classic filter is used in the deep neural network between the input layer and the output layer in parallel to at least one filter channel;

conduct a training phase of the deep neural network, wherein filter parameters of the at least one classic filter are selected on the basis of empirical values and fixed during the training phase of the deep neural network, and wherein filter parameters of the at least one filter channel are randomly initialized;

feed output data from the at least one classic filter to a backpropagation network of the deep neural network in addition feature maps extracted by the at least one filter channel; and

conduct an inference phase of the deep neural network for the operation of a driver assistance system of a vehicle, wherein the filter parameters of the at least one classic filter remain fixed during the inference phase of the deep neural network.

16 . A non-transitory computer-readable storage medium comprising commands which, when run by a computer, prompt the computer to:

operate a deep neural network with multiple layers between an input layer and an output layer, wherein at least one classic filter is used in the deep neural network between the input layer and the output layer in parallel to at least one filter channel;

conduct a training phase of the deep neural network, wherein filter parameters of the at least one classic filter are selected on the basis of empirical values and fixed during the training phase of the deep neural network, and wherein filter parameters of the at least one filter channel are randomly initialized;

feed output data from the at least one classic filter to a backpropagation network of the deep neural network in addition feature maps extracted by the at least one filter channel; and

conduct an inference phase of the deep neural network for the operation of a driver assistance system of a vehicle, wherein the filter parameters of the at least one classic filter remain fixed during the inference phase of the deep neural network.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 11, 2022
From: SCHMIDT, NICO MAURICE, DR.; SCHLICHT, PETER, DR.
To: VOLKSWAGEN AKTIENGESELLSCHAFT
Reel/Frame 060790/0057 →
Priority Claims (1)
DE 10 2019 207 580.0 · May 23, 2019 · national
Continuity (1)
Related Publication 20220222537A1 · Jul 14, 2022
References Cited (79)
US 5825646A · Keeler · 1998 [cited by examiner]
US 8142998B2 · Liew et al. · 2012 [cited by applicant]
US 10671082B2 · Huang et al. · 2020 [cited by applicant]
US 11423300B1 · Ritter et al. · 2022 [cited by applicant]
US 20040205037A1 · Mrziglod et al. · 2004 [cited by applicant]
US 20140095420A1 · Chun · 2014 [cited by examiner]
US 20180075594A1 · Brauer · 2018 [cited by applicant]
US 20180096232A1 · Danielsson et al. · 2018 [cited by applicant]
US 20180144203A1 · Moosaei et al. · 2018 [cited by applicant]
US 20180157933A1 · Brauer et al. · 2018 [cited by applicant]
US 20180173954A1 · Zink et al. · 2018 [cited by applicant]
US 20180308012A1 · Mummadi et al. · 2018 [cited by applicant]
US 20180336425A1 · Kadav · 2018 [cited by examiner]
US 20190066133A1 · Cotton · 2019 [cited by examiner]
US 20190130224A1 · Tiecke et al. · 2019 [cited by applicant]
US 20200134382A1 · Zhuravlev · 2020 [cited by examiner]
US 20200293892A1 · Zhou · 2020 [cited by applicant]
US 20200311798A1 · Forsyth et al. · 2020 [cited by applicant]
CN 105891215A · 2016 [cited by applicant]
CN 107895359A · 2018 [cited by applicant]
CN 108229276A · 2018 [cited by applicant]
CN 108615071A · 2018 [cited by applicant]
CN 109753973A · 2019 [cited by applicant]
DE 19649618A1 · 1998 [cited by applicant]
DE 102018200724A1 · 2018 [cited by applicant]
DE 102019207573A1 · 2020 [cited by applicant]
DE 102019207575A1 · 2020 [cited by applicant]
DE 102019207580A1 · 2020 [cited by applicant]
WO 2018085697A1 · 2018 [cited by applicant]
WO 2020233961A1 · 2020 [cited by applicant]
WO 2020233991A1 · 2020 [cited by applicant]
WO 2020233992A1 · 2020 [cited by applicant]
Gurkin, Metin N. et al., “Optimal Selection of Neural Network Architecture for CAD Using Simulated Annealing,” Proceedings of the 22nd Annual International Conference of the IEEE Engineering in Medicine and Biology Soci… [cited by applicant]
Calderón, Andrés et al., “Handwritten Digit Recognition Using Convolutional Neural Networks and Gabor Filters,” Proceedings of the International Congress on Computational Intellegence CIIC, 9 pages. [cited by applicant]
Kwolek, Bogdan, “Face Detecction Using Convolutional Neural Networks and Gabor Filters,” Artificial Neural Networks: Biological Inspirations—ICANN 2005 Lecture Notes in Computer Science, pp. 551-556. [cited by applicant]
Wu, Yihui et al., “Traffic Sign Detection based on Convolutional Neural Networks,” The IEEE 2013 International Joint Conference on Neural Networks, 7 pages. [cited by applicant]
Goodfellow, Ian J. et al., “Explaining and Harnessing Adversarial Examples,” URL: https://arxiv.org/pdf/1412.6572.pdf, 11 pages. [cited by applicant]
Papernot, Nicolas et al., “Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks,” 2016 IEEE Symposium on Security and Privacy, pp. 582-597. [cited by applicant]
Kinnikar, Ashwini et al., “Face Recognition Using Gabor Filter and Convolutional Neural Network,” Informatics and Analytics, ACM, 4 pages. [cited by applicant]
Li, Xin et al., “Adversarial Examples Detection in Deep Networks with Convulutional Filter Statistics,” arxiv.org, Cornell University Library, 8 pages. [cited by applicant]
Rashid, Tareq, Part 3 Neural Network MNIST Data with Rotations, Make Your Own Neural Network, URLs: https://github.com/makeyourownneuralnetwork/makeyourownneuralnetwork/blob/master/part3_neural_network_mnist_data_with_r… [cited by applicant]
Sarwar, Syed Shakib et al., “Gabor Filter Assisted Energy Efficient Fast Learning Convulutional Neural Networks,” IEEE/ACM Symposium on Low Power Electronics and Design, 6 pages. [cited by applicant]
Fujieda, Shin et al., “Wavelet Convolutional Neural Networks,” arXiv Preprint, URL: https://arxiv.org/abs/1805.08620v1, 10 pages. [cited by applicant]
Zantedeschi, Valentina et al., “Efficient Defenses Against Adversarial Attacks,” Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp. 39-49. [cited by applicant]
Gao, Ji et al., “Deepcloak: Masking Deep Neural Network Models for Robustness Against Adversarial Samples,” Workshop Track—ICLR 2017, URL: https://arxiv.org/pdf/1702.06763.pdf, 8 pages. [cited by applicant]
Molchanov, Pavlo et al., “Pruning Convolutional Neural Networks for Resource Efficient Inference,” URL: https://arxiv.org/pdf/1611.06440.pdf, 17 pages. [cited by applicant]
Li, Jiakun et al., “Using Gabor Filter in 3D Convolutional Neural Networks for Human Action Recognition,” 36th Chinese Control Conference, Technical Committee on Control Theory, pp. 11139-11144. [cited by applicant]
Chen, Yushi et al., “Hyperspectral Images Classification with Gabor Filtering and Convolutional Neural Network,” IEEE Geoscience and Remote Sensing Letters, vol. 14, No. 12, pp. 2355-2359. [cited by applicant]
Hosseini, Sepidehsadat et al., “Age and Gender Classification Using Wide Convolutional Neural Network and Gabor Filter,” 2018 International Workshop on Advanced Image Technology, 3 pages. [cited by applicant]
Luo, Bo et al., “Towards Imperceptible and Robust Adversarial Example Attacks Against Neural Networks,” arxiv.org, Cornell University Library, 8 pages. [cited by applicant]
Özbulak, Gökhan et al., “Initialization of Convolutional Neural Networks by Gabor Filters,” 26th Signal Processing and Communications Applications Conference, 6 pages. [cited by applicant]
Tian, Yuchi et al., “DeepTest: Automated Testing of Deep-Neural-Network-Driven Autonomous Cars,” Proceedings of the 40th International Conference on Software Engineering, pp. 303-314. [cited by applicant]
Liao, Fangzhou et al., “Defense Against Adversarial Attacks Using High-Level Representation Guided Denoiser,” 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1778-1787. [cited by applicant]
Rakin, Adnan Siraj et al., “Defend Deep Neural Networks Against Adversarial Examples via Fixed and Dynamic Quantized Activation Functions,” arxiv.org, Cornell University Library, XP081113638, 15 pages. [cited by applicant]
Jiang, Chenzhi et al., “Gabor Binary Layer in Convolutional Neural Networks,” 25th IEEE International Conference on Image Processing, pp. 3408-3412. [cited by applicant]
Wang, Siyue et al., “Defending DNN Adversarial Attacks with Pruning and Logits Augmentation,” 2018 IEEE Global Conference on Signal and Information Processing, pp. 1144-1148. [cited by applicant]
Ma, Shiqing et al., “NIC: Detecting Adversarial Samples with Neural Network Invariant Checking,” Proceedings of the 26th Network and Distributed System Security Symposium, 15 pages. [cited by applicant]
Zadeh, Milad Mohammad et al., “Fast Facial Emotion Recognition Using Convolutional Neural Networks and Gabor Filters,” IEEE 5th Conference on Knowledge Based Engineering and Innovation, pp. 577-581. [cited by applicant]
Chen, Hao-Yun et al., “Improving Adversarial Robustness via Guided Complement Entropy,” URL: https://arxiv.org/pdf/1903.09799v1.pdf, 10 pages. [cited by applicant]
Alekseev, Andrey et al., “GaborNet: Gabor Filters with Learnable Parameters in Deep Convolutional Neural Networks,” arxiv.org, Cornell University Library, 10 pages. [cited by applicant]
German Office Action, Application No. 102019207580.0, 8 pages. [cited by applicant]
German Office Action, Application No. 102019207575.4, 8 pages. [cited by applicant]
German Office Action, Application No. 102019207573.8, 9 pages. [cited by applicant]
International Search Report and Written Opinion, Application No. PCT/EP2020/062110, 11 pages. [cited by applicant]
International Search Report and Written Opinion, Application No. PCT/EP2020/062638, 10 pages. [cited by applicant]
International Search Report and Written Opinion, Application No. PCT/EP2020/062636, 10 pages. [cited by applicant]
Szegedy, Christian et al., “Going Deeper with Convolutions,” URL: https://arxiv.org/pdf/1409.4842, 9 pages. [cited by applicant]
Zheng, Stephan et al., “Improving the Robustness of Deep Neural Networks via Stability Training,” URL: https://arxiv.org/pdf/1604.04326, 9 pages. [cited by applicant]
Yim, Junho et al., “A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer Learning,” IEEE Conference on Computer Vision and Pattern Recognition, URL: https://openaccess.thecvf.com/cont… [cited by applicant]
Aguilar, Gustavo et al., “Knowledge Distillation from Internal Representations,” URL: https://arxiv.org/pdf/1910.03723, 8 pages. [cited by applicant]
U.S. Non-Final Office Action, U.S. Appl. No. 17/612,241, 36 pages. [cited by applicant]
Chinese Office Action, Application No. 202080038184.4, 8 pages. [cited by applicant]
U.S. Non-Final Office Action, U.S. Appl. No. 17/612,330, 35 pages. [cited by applicant]
U.S. Final Office Action, U.S. Appl. No. 17/612,330, 24 pages. [cited by applicant]
Sabour, Sara et al., “Adversarial Manipulation of Deep Representations,” arXiv.org, 18 pages. [cited by applicant]
European Office Action, Application No. 20724043.3, 8 pages. [cited by applicant]
U.S. Final Office Action, U.S. Appl. No. 17/612,241, 22 pages. [cited by applicant]
Shi, Hehuan et al., “PCA Pre-Trained Convolutional Neural Network Target Recognition Algorithm,” Journal of Xi'an University of Electronic Science and Technology, vol. 43, No. 3, 6 pages (Chinese w/ English abstract). [cited by applicant]
Chinese Office Action, Application No. 202080038184.4, 9 pages. [cited by applicant]