Deep neural network color space optimization

1. A method comprising:

transmitting a plurality of probe images from an Internet of Things (IoT) device at an edge network to a server hosting a target deep neural network (DNN), wherein the plurality of images are injected with a limited amount of noise to probe sensitivities of the target DNN to red, green, and blue colors;

receiving a feedback comprising a plurality of discrete cosine transform (DCT) coefficients from the server hosting the target DNN, wherein the plurality of DCT coefficients are unique to the target DNN;

computing a plurality of color conversion weights based on the feedback received from the server hosting the target DNN;

converting a set of real-time images from RGB color space to YUV color space using the plurality of color conversion weights unique to the target DNN;

compressing the converted set of real-time images using a quantization table specific to the target DNN by the IoT device at the edge network; and

transmitting the compressed set of real-time images to the server hosting the target DNN for DNN inferences.

2. The method of claim 1 , further comprising:

estimating a gradient of loss with respect to each color sensitivity by performing backward propagation of color sensitivity gradients corresponding to R, G, and B channels.

3. The method of claim 1 , wherein the plurality of color conversion weights comprise a first weight W R , a second weight W G , and a third weight W B corresponding to R, G, and B channels in RGB color space respectively.

4. The method of claim 3 , wherein the first weight W R is calculated as (1) a second ratio divided by (2) a sum of a first ratio, the second ratio, and 1, wherein the first ratio comprises a ratio of blue gradient over green gradient, and wherein the second ratio comprises a median ratio of red gradient over green gradient.

5. The method of claim 3 , wherein the second weight W G is calculated as 1 divided by a sum of a first ratio, a second ratio, and 1, wherein the first ratio comprises a ratio of blue gradient over green gradient, and wherein the second ratio comprises a median ratio of red gradient over green gradient.

6. The method of claim 3 , wherein the third weight W B is calculated as (1) a first ratio divided by (2) a sum of the first ratio, a second ratio, and 1, wherein the first ratio comprises a ratio of blue gradient over green gradient, and wherein the second ratio comprises a median ratio of red gradient over green gradient.

7. The method of claim 4 , wherein the first ratio and the second ratio are both median ratio across all RGB pixels of the plurality of images.

8. The method of claim 1 , wherein computing a plurality of color conversion weights based on the feedback received from the server hosting the target DNN is performed via an offline process without adding overhead to real-time image compression process.

9. The method of claim 1 , further comprising:

applying DCT on a live YUV image converted from a live RGB image using the plurality of color conversion weights by the IoT device to generate a plurality of spatial spectrum;

using a quantization table unique to the target DNN to generate quantized spectrum; and

performing entropy coding on the quantized spectrum to produce compressed image that are color sensitive to the target DNN.

10. A network device comprising at least:

a memory;

a processor executing instructions in the memory to:

transmit a plurality of probe images from an Internet of Things (IoT) device at an edge network to a server hosting a target deep neural network (DNN), wherein the plurality of images are injected with a limited amount of noise to probe sensitivities of the target DNN to red, green, and blue colors;

receive a feedback comprising a plurality of discrete cosine transform (DCT) coefficients from the server hosting the target DNN, wherein the plurality of DCT coefficients are unique to the target DNN;

compute a plurality of color conversion weights based on the feedback received from the server hosting the target DNN;

convert a set of real-time images from RGB color space to YUV color space using the plurality of color conversion weights unique to the target DNN;

compress the converted set of real-time images using a quantization table specific to the target DNN by the IoT device at the edge network; and

transmit the compressed set of real-time images to the server hosting the target DNN for DNN inferences.

11. The network device of claim 10 , wherein the processor further executes the instructions in the memory to:

estimate a gradient of loss with respect to each color sensitivity by performing backward propagation of color sensitivity gradients corresponding to R, G, and B channels.

12. The network device of claim 10 , wherein the plurality of color conversion weights comprise a first weight W R , a second weight W G , and a third weight W B corresponding to R, G, and B channels in RGB color space respectively.

13. The network device of claim 12 , wherein the first weight W R is calculated as (1) a second ratio divided by (2) a sum of a first ratio, the second ratio, and 1, wherein the first ratio comprises a ratio of blue gradient over green gradient, and wherein the second ratio comprises a median ratio of red gradient over green gradient.

14. The network device of claim 12 , wherein the second weight W G is calculated as 1 divided by a sum of a first ratio, a second ratio, and 1, wherein the first ratio comprises a ratio of blue gradient over green gradient, and wherein the second ratio comprises a median ratio of red gradient over green gradient.

15. The network device of claim 12 , wherein the third weight W B is calculated as (1) a first ratio divided by (2) a sum of the first ratio, a second ratio, and 1, wherein the first ratio comprises a ratio of blue gradient over green gradient, and wherein the second ratio comprises a median ratio of red gradient over green gradient.

16. The network device of claim 13 , wherein the first ratio and the second ratio are both median ratio across all RGB pixels of the plurality of images.

17. The network device of claim 10 , wherein computing a plurality of color conversion weights based on the feedback received from the server hosting the target DNN is performed via an offline process without adding overhead to real-time image compression process.

18. The network device of claim 10 , wherein the processor further executes the instructions in the memory to:

apply DCT on a live YUV image converted from a live RGB image using the plurality of color conversion weights by the IoT device to generate a plurality of spatial spectrum;

use a quantization table unique to the target DNN to generate quantized spectrum; and

perform entropy coding on the quantized spectrum to produce compressed image that are color sensitive to the target DNN.

19. A non-transitory machine-readable storage medium encoded with instructions executable by a processor of a computing device, the non-transitory machine-readable storage medium comprising instructions to:

transmit a plurality of probe images from an Internet of Things (IoT) device at an edge network to a server hosting a target deep neural network (DNN), wherein the plurality of images are injected with a limited amount of noise to probe sensitivities of the target DNN to red, green, and blue colors;

receive a feedback comprising a plurality of discrete cosine transform (DCT) coefficients from the server hosting the target DNN, wherein the plurality of DCT coefficients are unique to the target DNN;

compute a plurality of color conversion weights based on the feedback received from the server hosting the target DNN;

convert a set of real-time images from RGB color space to YUV color space using the plurality of color conversion weights unique to the target DNN;

compress the converted set of real-time images using a quantization table specific to the target DNN by the IoT device at the edge network; and

transmit the compressed set of real-time images to the server hosting the target DNN for DNN inferences.

20. The non-transitory machine-readable storage medium of claim 19 , further comprising instructions to:

apply DCT on a live image by the IoT device to generate a plurality of DCT coefficient;

use the quantization table to generate quantized coefficients; and

perform entropy coding on the quantized coefficient to produce compressed image that are sensitive to the target DNN.