Method and device for selecting transform matrices for down-sampling DCT image using learning with forgetting algorithm
View Patent ↗Down-sampling of an image may be performed in the DCT domain. A multiple layered network is used to select transform matrices for down-sampling a DCT image of size M×N to a DCT image of size I×J. A spatial domain down-sampling method is selected and applied to the DCT image to produce a down-sampled DCT reference image. A learning with forgetting algorithm is used to apply a decay to the elements of the transform matrix and select a transform matrices which solve an optimization problem. The optimization problem is a function of the visual quality of images obtained using the transform matrices and the computational complexity associated with using the transform matrices. The visual quality is a measure of the difference between the down-sampled DCT image obtained using the transform matrices and the visual quality of the DCT reference image obtained using a spatial domain down-sampling method.
1. A method of selecting transform matrices for down-sampling a discrete cosine transform (DCT) image directly in the DCT domain, the method comprising:
initializing the transform matrices; and
adjusting the transform matrices by applying a learning-with-forgetting algorithm to the transform matrices to obtain transform matrices which reduce a learning objective function, wherein the learning objective function is a function of the error between a down-sampled DCT image obtained using the transform matrices and a down-sampled DCT reference image obtained using another down-sampling method.
2. The method of claim 1 , wherein the learning objective function is also a function of the computational complexity associated with using the transform matrices to down-sample the DCT image.
3. The method of claim 1 , wherein adjusting the transform matrices comprises:
computing a learning amount for the transform matrices; and
adjusting the transform matrices using the learning amount.
4. The method of claim 3 , wherein computing a learning amount and adjusting the transform matrices are repeated until a change in the learning objective function is less than a predetermined amount.
5. The method of claim 1 , wherein the transform matrices comprise a row transform matrix for down-sampling the number of rows in the DCT image and a column transform matrix for down-sampling the number of columns in the DCT image and wherein the learning objective function is
J f =∥D IM ·C MN ·W NJ −V IJ ∥ 2 λ·r g ,
where f f is the learning objective function; D IM is the row transform matrix; W NJ is the column transform matrix; C MN is the DCT image; D IM ·C MN ·W NJ is the down-sampled DCT image obtained using the transform matrices; V IJ is a down-sampled DCT reference image obtained using another down-sampling method; r g is a computational complexity associated with using the transform matrices D IM ,W NJ to down-sample an image C MN ; and λ is a quality and complexity trade-off parameter used to balance a trade-off between visual quality of down-sampled DCT images and the computational complexity associated with using the transform matrices to down-sample the DCT image.
6. The method of claim 5 , wherein the computational complexity is determined from the absolute values of all elements in the transform matrices.
7. The method of claim 1 further comprising:
applying a selective learning-with-forgetting algorithm to the transform matrices, the selective learning-with-forgetting algorithm selectively applying a decay to elements of the transform matrices having weights less than a predetermined selective learning-with-forgetting threshold parameter.
8. The method of claim 7 , wherein the selective learning-with-forgetting algorithm is repeatedly applied to the transform matrices until the change in the learning objective function is less than a second predetermined threshold.
9. The method of claim 8 , wherein the learning objective function used in the selective learning-with-forgetting algorithm is
J f =∥D IM ·C MN ·W NJ −V IJ ∥ 2 +λ·r q ,
where J f is the learning objective function; D IM is the row transform matrix; W NJ is the column transform matrix; C MN is the DCT image; D IM ·C MN ·W NJ is the down-sampled DCT image obtained using the transform matrices; V IJ is the down-sampled DCT reference image obtained by down-sampling the DCT image C MN using another down-sampling method; r q is a measure of a selective learning-with-forgetting computational complexity associated with using the transform matrices D IM ,W NJ to down-sample the DCT image C MN ; and λ is a quality and complexity trade-off parameter which is used to obtain a desired trade-off between visual quality of the down-sampled DCT image obtained using the transform matrices and the computational complexity associated with using the transform matrices to down-sample the DCT image.
10. The method of claim 9 , wherein the selective learning-with-forgetting computational complexity is a function of the absolute values of all elements in the transform matrices that are below the selective learning-with-forgetting threshold parameter, and wherein applying a selective learning-with-forgetting algorithm further comprises:
determining the selective learning-with-forgetting computational complexity of the transform matrices.
11. The method of claim 10 , wherein the selective learning-with-forgetting computational complexity is also a function of the number of non-zero elements in a binary representation of elements of the transform matrices.
12. An electronic device comprising:
a memory for storing transform matrices for down-sampling a discrete cosine transform (DCT) image directly in the DCT domain; and
a processor connected to the memory for obtaining the transform matrices, the processor being configured to:
initialize the transform matrices; and
adjust the transform matrices by applying a learning-with-forgetting algorithm to the transform matrices to obtain transform matrices which reduce a learning objective function,
wherein, the learning objective function is a function of the error between a down-sampled DCT image obtained using the transform matrices and a down-sampled DCT reference image obtained using another down-sampled method.
13. The electronic device of claim 12 , wherein the learning objective function is also a function of the computational complexity associated with using the transform matrices to down-sample the DCT image.
14. The electronic device of claim 12 , wherein adjusting the transform matrices comprises computing a learning amount for the transform matrices and adjusting the transform matrices using the learning amount.
15. The electronic device of claim 14 , wherein the processor is configured to repeatedly compute a learning amount and adjust the transform matrices until a change in the learning objective function is less than a predetermined amount.
16. The electronic device of claim 12 , wherein the learning objective function is
J f =∥D IM ·C MN ·W NJ −V IJ ∥ 2 +λ·r g ,
where J f is the learning objective function; D IM is the row transform matrix; W NJ is the column transform matrix; C MN is the DCT image; D IM ·C MN ·W NJ is the down-sampled DCT image obtained using the transform matrices; V IJ is a down-sampled DCT reference image obtained using another down-sampling method; r g is a computational complexity associated with using the transform matrices D IM ,W NJ to down-sample an image C MN ; and λ is a quality and complexity trade-off parameter used to balance a trade-off between visual quality of down-sampled DCT images and the computational complexity associated with using the transform matrices to down-sample the DCT image.
17. The electronic device of claim 12 wherein the processor is further configured to apply a selective learning-with-forgetting algorithm to the transform matrices, the selective learning-with-forgetting algorithm selectively applying a decay to elements of the transform matrices having weights less than a predetermined selective learning-with-forgetting threshold parameter and further minimize the learning objective function.
18. The electronic device of claim 12 , wherein the processor is configured to obtain transform matrices when there is low processor usage.
19. The electronic device of claim 12 , wherein the electronic device is a server which may be accessed by one or more wireless devices.
20. The electronic device of claim 12 , wherein the DCT image is a frame of a DCT video.