Granted Patent B2

US 12,375,699 · App. 17/882,293 · Granted Jul 29, 2025

Inventors: Quockhanh Dinh (Suwon-si, KR); Kwangpyo Choi (Suwon-si, KR)

Assignee: SAMSUNG ELECTRONICS CO., LTD.

H04N19/436G06N3/045G06N3/08H04N19/124H04N19/13H04N19/136H04N19/186H04N19/30H04N19/50

Abstract

Claims (60)

1. A method of decoding an image based on cross-channel prediction using artificial intelligence (AI), the method comprising:

obtaining feature data for cross-channel prediction from a bitstream;

obtaining feature data of a luma image in a current image and feature data of a chroma image in the current image, from the bitstream;

reconstructing the luma image by applying the feature data of the luma image to a neural-network-based luma decoder;

obtaining cross-channel prediction information by applying the feature data for cross-channel prediction to a neural-network-based cross-channel decoder;

obtaining a predicted image of the chroma image by performing cross-channel prediction based on the reconstructed luma image and the cross-channel prediction information;

obtaining a residual image of the chroma image by applying the feature data of the chroma image to a neural-network-based chroma residual decoder; and

reconstructing the chroma image based on the predicted image and the residual image,

wherein the cross-channel prediction information includes a scale parameter value and a bias parameter value,

wherein the scale parameter value is a value on which a multiplication operation is performed for each element with a sample value of the reconstructed luma image,

wherein the bias parameter value is a value obtained by performing an addition operation for each element on a result obtained by performing a multiplication operation for each element on the scale parameter value and the sample value of the reconstructed luma image,

wherein the feature data of the luma image is generated from a neural-network-based luma encoder,

wherein the feature data of the chroma image is generated from a neural-network-based chroma encoder,

wherein the feature data for cross-channel prediction is generated from a neural-network-based cross-channel encoder, and

wherein the neural-network-based luma encoder, the neural-network-based chroma encoder, and the neural-network-based cross-channel encoder are different neural networks from each other.

2. The method of claim 1 , wherein at least one of the feature data for cross-channel prediction, the feature data of the luma image, or the feature data of the chroma image is obtained through entropy-decoding and inverse-quantization on the bitstream.

3. The method of claim 1 , wherein the neural-network-based cross-channel decoder is trained based on:

first loss information corresponding to a difference between a current training chroma image and a current reconstructed training chroma image corresponding to the current training chroma image; and

second loss information corresponding to entropy of the feature data for cross-channel prediction of the current training chroma image.

4. The method of claim 1 , further comprising performing down-sampling on the reconstructed luma image when a chroma sub-sampling format of the current image is not YUV (YCbCr) 4:4:4,

wherein the obtaining of the predicted image of the chroma image comprises obtaining the predicted image of the chroma image by performing cross-channel prediction based on the down-sampled luma image and the cross-channel prediction information.

5. The method of claim 1 , further comprising generating multi-channel luma image data by performing space-to-depth transformation on the reconstructed luma image when a chroma sub-sampling format of the current image is not YCbCr 4:4:4,

wherein the obtaining of the predicted image of the chroma image comprises obtaining the predicted image of the chroma image by performing cross-channel prediction based on the multi-channel luma image data and the cross-channel prediction information.

6. The method of claim 1 , wherein the luma image comprises an image of a Y component, and the chroma image comprises an image of at least one of a Cb component or a Cr component.

7. The method of claim 1 , wherein the obtaining of the cross-channel prediction information by applying the feature data for cross-channel prediction to the neural-network-based cross-channel decoder comprises obtaining the cross-channel prediction information by applying the feature data for cross-channel prediction and the feature data of the luma image to the neural-network-based cross-channel decoder.

8. The method of claim 1 , wherein the obtaining of the residual image of the chroma image by applying the feature data of the chroma image to the neural-network-based chroma residual decoder comprises obtaining the residual image of the chroma image by further applying at least one of the feature data of the luma image or the feature data for cross-channel prediction to the neural-network-based chroma residual decoder.

9. A non-transitory computer-readable recording medium having recorded thereon a program for executing the method of claim 1 on a computer.

10. An apparatus for decoding an image based on cross-channel prediction using artificial intelligence (AI), the apparatus comprising:

an obtainer configured to:

obtain feature data for cross-channel prediction from a bitstream, and

obtain feature data of a luma image in a current image and feature data of a chroma image in the current image, from the bitstream; and

an image decoder configured to:

reconstruct the luma image by applying the feature data of the luma image to a neural-network-based luma decoder,

obtain cross-channel prediction information by applying the feature data for cross-channel prediction to a neural-network-based cross-channel decoder and obtain a predicted image of the chroma image by performing cross-channel prediction based on the reconstructed luma image and the cross-channel prediction information,

obtain a residual image of the chroma image by applying the feature data of the chroma image to a neural-network-based chroma residual decoder, and

reconstruct the chroma image based on the predicted image of the chroma image and the residual image of the chroma image,

wherein the cross-channel prediction information includes a scale parameter value and a bias parameter value,

wherein the scale parameter value is a value on which a multiplication operation is performed for each element with a sample value of the reconstructed luma image,

wherein the bias parameter value is a value obtained by performing an addition operation for each element on a result obtained by performing a multiplication operation for each element on the scale parameter value and the sample value of the reconstructed luma image,

wherein the feature data of the luma image is generated from a neural-network-based luma encoder,

wherein the feature data of the chroma image is generated from a neural-network-based chroma encoder,

wherein the feature data for cross-channel prediction is generated from a neural-network-based cross-channel encoder, and

wherein the neural-network-based luma encoder, the neural-network-based chroma encoder, and the neural-network-based cross-channel encoder are different neural networks from each other.

11. A method of encoding an image based on cross-channel prediction using artificial intelligence (AI), the method comprising:

obtaining feature data of a luma image in a current image by applying an original luma image in a current original image to a neural-network-based luma encoder and reconstructing the luma image by applying the feature data of the luma image to a neural-network-based luma decoder;

obtaining feature data for cross-channel prediction by applying the reconstructed luma image and an original chroma image in the current original image to a neural-network-based cross-channel encoder;

obtaining cross-channel prediction information by applying the obtained feature data for cross-channel prediction to a neural-network-based cross-channel decoder;

obtaining a predicted image of the chroma image by performing cross-channel prediction based on the reconstructed luma image and the cross-channel prediction information;

obtaining feature data of the chroma image by applying a residual image of the chroma image obtained based on the original chroma image and the predicted image of the chroma image to a neural-network-based chroma residual encoder; and

generating a bitstream comprising the feature data of the luma image, the feature data of the chroma image, and the feature data for cross-channel prediction,

wherein the cross-channel prediction information includes a scale parameter value and a bias parameter value,

wherein the scale parameter value is a value on which a multiplication operation is performed for each element with a sample value of the reconstructed luma image,

wherein the bias parameter value is a value obtained by performing an addition operation for each element on a result obtained by performing a multiplication operation for each element on the scale parameter value and the sample value of the reconstructed luma image, and

wherein the neural-network-based luma encoder, the neural-network-based chroma residual encoder, and the neural-network-based cross-channel encoder are different neural networks from each other.

12. The method of claim 11 , wherein at least one of the feature data for cross-channel prediction, the feature data of the luma image, or the feature data of the chroma image is quantized and entropy-encoded.

13. The method of claim 11 , wherein the neural-network-based cross-channel encoder is trained based on:

first loss information corresponding to a difference between a current training chroma image and a current reconstructed training chroma image corresponding to the current training chroma image; and

second loss information corresponding to entropy of the feature data for cross-channel prediction of the current training chroma image.

14. The method of claim 11 , further comprising performing down-sampling on the reconstructed luma image when a chroma sub-sampling format of the current image is not YCbCr 4:4:4,

wherein the obtaining of the predicted image of the chroma image comprises obtaining the predicted image of the chroma image by performing cross-channel prediction based on the down-sampled luma image and the cross-channel prediction information.

Assignments (1)

Priority Claims (2)

Continuity (2)

References Cited (24)