Apparatus and method for image processing, and system for training neural network

1. A system for training a generative adversarial network, comprising

a generative adversarial network processor, comprising a generative network microprocessor which is configured to be trained by a discriminative network microprocessor, and the discriminative network microprocessor which is coupled to the generative network,

wherein a plurality of input terminals are coupled to a plurality of apparatuses, and each of the apparatuses is configured to generate a plurality of correlation images and includes:

a feature extracting unit configured to receive a training image and extracting at least one or more of feature from the training image to generate a first feature image based on the training image;

a normalizer configured to normalize the first feature image and generate a second feature image; and

a shift correlating unit configured to perform a plurality of translational shifts on the second feature image to generate a plurality of shifted images, correlate each of the plurality of shifted images with the second feature image to generate the plurality of correlation images; and

wherein the discriminative network microprocessor comprises:

the plurality of input terminals coupled to the plurality of apparatuses;

a plurality of analysis modules, each of the plurality of analysis modules being coupled to one of the plurality of input terminals;

a plurality of pooling modules connected in cascade, each stage of the cascade comprising a pooling module coupled to one of the plurality of analysis modules and to a pooling module in a previous stage of the cascade; and

a discriminator network coupled to the pooling module in a last stage of the cascade.

2. The system according to claim 1 ,

wherein the shift correlating unit is configured to perform the plurality of translational shifts on the second feature image by shifting a number of leftmost or rightmost columns of pixels in a pixel block of the second feature image to be the rightmost and leftmost column, respectively, of the pixel block; and shifting b number of bottommost or topmost rows of pixels in the pixel block of the second feature image to be the topmost or bottommost row, respectively, of the pixel block,

wherein 0≤a<Y, 0≤b<X, each of a and b being an integer, Y being a total number of columns of pixels in the pixel block of the second feature image, and X being a total number of rows of pixels in the pixel block of the second feature image, and

wherein a and b may be the same or different.

3. The system according to claim 1 ,

wherein the shift correlating unit is configured to correlate each of the plurality of shifted images with the second feature image by multiplying a pixel value of each pixel in a pixel block of each of the plurality of shifted images with a pixel value of a positionally corresponding pixel in the pixel block of the second feature image.

4. The system according to claim 1 ,

wherein the first feature image is a luminance feature image, and

wherein the feature extracting unit comprises:

a luminance detector configured to extract luminance information from the training image to generate the luminance feature image.

5. The system according to claim 4 ,

wherein to generate the luminance feature image, the luminance detector is configured to determine a luminance value of a pixel at a given position in the luminance feature image according to the following formula (1):

I=0.299R+0.587G+0.114B  (1)

wherein:

I is the luminance value,

R is a red component value of a positionally corresponding pixel in the training image,

G is a green component value of the positionally corresponding pixel in the training image, and

B is a blue component value of the positionally corresponding pixel in the training image.

6. The system according to claim 4 ,

wherein the normalizer is configured to normalize the luminance feature image according to the following formula (2):

{

N

=

I

-

μ

σ

-

1

σ

2

=

Blur

⁢

(

I

2

)

-

μ

μ

=

Blur

⁢

(

I

)

wherein:

N is the first feature image,

I represents the luminance value of a pixel at a given position in the luminance feature image,

Blur(I) is an image obtained by applying a Gaussian filter to the luminance feature image, and

Blur(I 2 ) is an image obtained by squaring every pixel value in the luminance feature image, and then applying the Gaussian filter to the image.

7. The system according to claim 1 ,

wherein the second feature image comprises a pixel block having a first size,

wherein each of the plurality of shifted images and each of the plurality of correlation images comprises a pixel block having the first size, and

wherein in each of the plurality of shifted images, a pixel having a non-zero pixel value has a corresponding pixel with the same non-zero pixel value in the second feature image.

8. A method of training a generative adversarial network, the method comprising:

generating a first feature image based on a training image;

normalizing the first feature image and generating a second feature image;

performing a plurality of translational shifts on the second feature image to generate a plurality of shifted images;

correlating each of the plurality of shifted images with the second feature image to generate a plurality of correlation images; and

transmitting the plurality of correlation images to a system for training the generative adversarial network;

wherein the system includes a generative adversarial network processor, comprising a generative network microprocessor which is configured to be trained by a discriminative network microprocessor, and the discriminative network microprocessor which is coupled to the generative network;

wherein the discriminative network microprocessor comprises:

a plurality of input terminals for receiving the plurality of correlation images;

a plurality of analysis modules, each of the plurality of analysis modules being coupled to one of the plurality of input terminals;

a plurality of pooling modules connected in cascade, each stage of the cascade comprising a pooling module coupled to one of the plurality of analysis modules and to a pooling module in a previous stage of the cascade; and

a discriminator network coupled to the pooling module in a last stage of the cascade.

9. The method according to claim 8 ,

wherein the correlating of each of the plurality of shifted images with the second feature image comprises multiplying a pixel value of each pixel in a pixel block of each of the plurality of shifted images with a pixel value of a positionally corresponding pixel in the pixel block of the second feature image.

10. The method according to claim 8 ,

wherein the performing of the plurality of translational shifts comprises:

shifting a number of leftmost or rightmost columns of pixels in a pixel block of the second feature image to be the rightmost and leftmost column, respectively, of the pixel block; and

shifting b number of bottommost or topmost rows of pixels in the pixel block of the second feature image to be the topmost or bottommost row, respectively, of the pixel block,

wherein 0≤a<Y, 0≤b<X, each of a and b being an integer, Y being a total number of columns of pixels in the pixel block of the second feature image, and X being a total number of rows of pixels in the pixel block of the second feature image, and

wherein a and b may be the same or different.

11. The method according to claim 10 ,

wherein at least one of a and b changes at least once during the performing of the plurality of translational shifts.

12. The method according to claim 8 ,

wherein the performing of the plurality of translational shifts comprises:

deleting a number of leftmost or rightmost columns of pixels in a pixel block of the second feature image, and adding a number of columns of pixels at the rightmost or leftmost position, respectively, of the pixel block; and

deleting b number of bottommost or topmost rows of pixels in the pixel block of the second feature image, and adding b number of rows at the topmost or bottommost position, respectively, of the pixel block,

wherein 0≤a<Y, 0≤b <X, each of a and b being an integer, Y being a total number of columns of pixels in the pixel block of the second feature image, and X being a total number of rows of pixels in the pixel block of the second feature image, and

wherein each of the added pixels has a pixel value of 0.

13. The method according to claim 12 ,

wherein at least one of a and b changes at least once during the performing of the plurality of translational shifts.

14. The method according to claim 8 , further comprising performing X*Y translational shifts, Y being a total number of columns of pixels in a pixel block of the second feature image, and X being a total number of rows of pixels in the pixel block of the second feature image.

15. The method according to claim 8 , further comprising, before the generating of the first feature image, receiving the training image,

wherein the generating of the first feature image comprises generating a luminance feature image based on luminance information of the training image.

16. The method according to claim 15 , further comprising determining a luminance value of a pixel at a given position in the luminance feature image according to the following formula (1):

I=0.299R+0.587G+0.114B  (1)

wherein:

I is the luminance value,

R is a red component value of a positionally corresponding pixel in the training image,

G is a green component value of the positionally corresponding pixel in the training image, and

B is a blue component value of the positionally corresponding pixel in the training image.

17. The method according to claim 15 , further comprising normalizing the luminance feature image according to the following formula (2):

{

N

=

I

-

μ

σ

-

1

σ

2

=

Blur

⁢

(

I

2

)

-

μ

μ

=

Blur

⁢

(

I

)

(

2

)

wherein:

N is the first feature image,

I represents the luminance feature image, Blur(I) is an image obtained by applying a Gaussian filter to the luminance feature image, and

Blur(I 2 ) is an image obtained by squaring every pixel value in the luminance feature image, and then applying the Gaussian filter to the image.

18. The method according to claim 8 ,

wherein the first feature image comprises a pixel block having a first size,

wherein each of the plurality of shifted images and each of the plurality of correlation images comprises a pixel block having the first size, and

wherein in each of the plurality of shifted images, a pixel having a non-zero pixel value has a corresponding pixel with the same non-zero pixel value in the first feature image.

19. A non-transitory computer-readable medium storing instructions that cause a computer to execute the method according to claim 8 .