Performance optimization of pre-processor using video proxy codec

1 . A method comprising:

receiving source content;

pre-processing, by a pre-processor, the source content to output pre-processed source content, wherein:

the pre-processor includes a first parameter that is trained based on a differentiable proxy codec,

a calculated adjustment to a second parameter of the differentiable proxy codec is determined, and

a gradient based on a loss between the calculated adjustment of the second parameter of the differentiable proxy codec and the second parameter is used to train the first parameter of the pre-processor;

encoding the pre-processed source content into compressed pre-processed source content; and

outputting the compressed pre-processed source content.

2 . The method of claim 1 , further comprising:

causing the compressed pre-processed source content to be sent to a client device.

3 . The method of claim 1 , wherein:

a target codec is used to encode the pre-processed source content, and

the second parameter of the differentiable proxy codec is trained to output compressed content that minimizes a first loss when compared to source content that is encoded by the target codec.

4 . The method of claim 3 , wherein training the second parameter of the differentiable proxy codec comprises:

determining a second loss between a first bitrate from encoding the source content by the differentiable proxy codec and a second bitrate from encoding the source content from the target codec;

adjusting the second parameter of the differentiable proxy codec based on the second loss.

5 . The method of claim 4 , wherein:

the second parameter of the differentiable proxy codec is adjusted to minimize the first loss or the second loss.

6 . The method of claim 1 , wherein:

the second parameter of the differentiable proxy codec is frozen during training; and

the calculated adjustment of the second parameter of the differentiable proxy codec is calculated and used to train the first parameter of the pre-processor.

7 . The method of claim 6 , wherein:

the gradient is calculated based on the loss between the calculated adjustment of the second parameter and the second parameter that is frozen, and

the gradient is used to determine a change to the first parameter of the pre-processor.

8 . The method of claim 7 , wherein:

the gradient is back propagated to the pre-processor to determine the change to the first parameter of the pre-processor.

9 . The method of claim 1 , wherein training the pre-processor comprises:

training the second parameter of the differentiable proxy codec to encode source content based on a first loss between first compressed source content that is output by the differentiable proxy codec and second compressed source content that is output by a target codec;

pre-processing, by the pre-processor, a source image to output a pre-processed source image, the pre-processing being based on the first parameter;

encoding, by the differentiable proxy codec, the pre-processed source image into a compressed pre-processed source image based on the first parameter;

determining a second loss between the source image and the compressed pre-processed source image;

determining the calculated adjustment to the second parameter of the differentiable proxy codec based on the second loss; and

using the calculated adjustment to adjust the first parameter of the pre-processor based on the second loss.

10 . The method of claim 9 , wherein training the second parameter of the differentiable proxy codec comprises:

processing, by the differentiable proxy codec, the source content to generate the first compressed source content;

processing, by the target codec, the source content to generate the second compressed source content; and

comparing the first compressed source content to the second compressed source content to determine the first loss.

11 . The method of claim 10 , wherein training the second parameter of the differentiable proxy codec comprises:

adjusting the second parameter based on minimizing the first loss.

12 . The method of claim 11 , wherein training the second parameter of the differentiable proxy codec comprises:

receiving information from processing the source content to generate the first compressed source content;

processing the information to generate a first bitrate for the first compressed source content; and

comparing the first bitrate to a second bitrate for the second compressed source content to determine a third loss.

13 . The method of claim 12 , wherein training the second parameter of the differentiable proxy codec comprises:

adjusting the second parameter based on minimizing the third loss.

14 . The method of claim 13 , wherein the information is based on a latent representation from encoding the source content.

15 . The method of claim 9 , wherein the differentiable proxy codec includes a neural network that encodes the source content based on the first parameter.

16 . The method of claim 1 , wherein:

a target codec is used to encode the pre-processed source content and not the differentiable proxy codec.

17 . The method of claim 1 , wherein the second parameter is not adjusted using the calculated adjustment to the second parameter when training the first parameter.

18 . A non-transitory computer-readable storage medium having stored thereon computer executable instructions, which when executed by a computing device, cause the computing device to be operable for:

receiving source content;

pre-processing, by a pre-processor, the source content to output pre-processed source content, wherein:

the pre-processor includes a first parameter that is trained based on a differentiable proxy codec,

a calculated adjustment to a second parameter of the differentiable proxy codec is determined, and

a gradient based on a loss between the calculated adjustment of the second parameter of the differentiable proxy codec and the second parameter is used to train the first parameter of the pre-processor;

encoding the pre-processed source content into compressed pre-processed source content; and

outputting the compressed pre-processed source content.

19 . An apparatus comprising:

one or more computer processors; and

a computer-readable storage medium comprising instructions for controlling the one or more computer processors to be operable for:

receiving source content;

pre-processing, by a pre-processor, the source content to output pre-processed source content, wherein:

the pre-processor includes a first parameter that is trained based on a differentiable proxy codec,

a calculated adjustment to a second parameter of the differentiable proxy codec is determined, and

a gradient based on a loss between the calculated adjustment of the second parameter of the differentiable proxy codec and the second parameter is used to train the first parameter of the pre-processor;

encoding the pre-processed source content into compressed pre-processed source content; and

outputting the compressed pre-processed source content.