System and method for deep neural network compression

1. A computing system, comprising:

a data storage configured to store a deep neural network (“DNN”); and

at least one processor communicatively coupled to the data storage,

wherein the at least one processor is configured to execute program instructions to cause the system to perform the steps comprising:

training the DNN, wherein the DNN has at least one convolutional layer with a plurality of filters;

calculating similarity measures between the filters of the at least one layer;

removing one or more of the filters based on the calculated similarity measures;

clustering remaining ones of the filters of the at least one convolutional layer;

applying dimension reduction on the clustered filters of the at least one convolutional layer;

retraining the DNN to generate a compressed DNN;

and storing the retrained DNN in the data storage.

2. The computing system of claim 1 further comprising the step, after the storing step, of transmitting the retrained DNN to a mobile device.

3. The computing system of claim 2 wherein the mobile device is a vehicle and wherein the retrained DNN is transmitted via an over-the-air wireless broadband network to the vehicle.

4. The computing system of claim 1 further comprising the step, after the applying dimension reduction step, of determining whether a data storage size of the DNN has been compressed to a predefined threshold.

5. The computing system of claim 4 in the determining step, wherein the clustering step and the applying dimension reduction step are iteratively performed based on whether the data storage size of the DNN has been compressed to the predefined threshold.

6. The computing system of claim 1 further comprising a step, after the applying dimension reduction step and before the retraining step, of determining whether the DNN meets a predefined level of accuracy for a predefined data set.

7. The computing system of claim 6 in the determining step, wherein the DNN is retrained until the predefined level of accuracy is met.

8. The computing system of claim 7 in the determining step, wherein if a predefined number of consecutive retrains of the DNN is reached, then iteratively apply the clustering step and the applying dimension reduction step.

9. The computing system of claim 1 , wherein the steps of calculating similarity measure and removing one or more of the filters are performed after the training step and before the clustering step.

10. A computer-implemented method for compressing at least one layer of a deep neural network (DNN), comprising:

training the DNN by a processor, wherein the at least one convolutional layer has a plurality of filters;

calculating similarity measures between the filters of the at least one layer;

removing one or more of the filters based on the calculated similarity measures;

clustering, by the processor, remaining ones of the filters of the at least one layer;

applying dimension reduction, by the processor, on the clustered filters of the at least one layer;

retraining, by the processor, the DNN; and

storing the retrained DNN in a storage device.

11. The computer-implemented method of claim 10 further comprising the step, after the storing step, of transmitting the compressed DNN to a mobile device.

12. The computer-implemented method of claim 11 wherein the mobile device is a vehicle and wherein the compressed DNN is transmitted via an over-the-air wireless broadband network to the vehicle.

13. The computer-implemented method of claim 10 further comprising the step, after the applying dimension reduction step, of determining whether a data storage size of the DNN has been compressed to apredefined threshold.

14. The computer-implemented method of claim 13 in the determining step, wherein the clustering step and the applying dimension reduction step are iteratively performed based on whether the data storage size of the DNN has been compressed to the predefined threshold.

15. The computer-implemented method of claim 10 further comprising a step, after the applying dimension reduction step and before the retraining step, of determining whether the DNN meets a predefined level of accuracy for a predefined data set.

16. The computer-implemented method of claim 15 in the determining step, wherein the DNN is retrained until the predefined level of accuracy is met.

17. The computer-implemented method of claim 16 in the determining step, wherein if a predefined number of consecutive retrains of the DNN is reached, then iteratively apply the clustering step and the applying dimension reduction step.

18. The computer-implemented method of claim 10 , wherein the steps of calculating similarity measure and removing one or more of the filters are performed after the training step and before the clustering step.

19. A non-transitory computer readable medium encoded with instructions that when executed by at least one processor causes the processor to carry out the following operations:

training a convolutional neural network (“CNN”), wherein the CNN has at least one layer of filters;

calculating similarity measures between the filters of the at least one layer;

removing one or more of the filters based on the calculated similarity measures;

clustering remaining ones of the filters of the at least one layer;

applying dimension reduction on the clustered filters of the at least one layer;

determining whether a data storage size of the DNN has been compressed to a predefined threshold, wherein the clustering step and the applying dimension reduction step are iteratively performed based on whether the data storage size of the DNN has been compressed to the predefined threshold;

determining whether the DNN meets a predefined level of accuracy for a predefined data set; and

retraining the CNN a predefined number of times until the predefined level accuracy is met.

20. The non-transitory computer readable medium of claim 19 further comprising the operation, after the retraining the CNN operation, of, transmitting the retrained DNN to a vehicle via an over-the-air wireless broadband network.