Granted Patent B2

US 12,536,704 · App. 18/079,174 · Granted Jan 27, 2026

Inventors: Hui Yuan (Dongguan, CN); Congrui Fu (Dongguan, CN); Ming Li (Dongguan, CN)

Assignee: GUANGDONG OPPO MOBILE TELECOMMUNICATIONS CORP., LTD.

G06T9/00G06T3/4053G06T5/50G06V10/7715G06T2200/21G06T2207/10021G06T2207/10052

Abstract

Claims (62)

1 . A method for light field picture processing, applied to a light field picture decoder and comprising:

parsing a bitstream to obtain an initial Sub Aperture Image (SAI);

inputting the initial SAI into a super-resolution reconstruction net, and outputting a reconstructed SAI, wherein the spatial resolution and the angular resolution of the reconstructed SAI are both greater than the spatial resolution and the angular resolution of the initial SAI; and

inputting the reconstructed SAI into a Quality Enhancement Net (QENet), and outputting a target SAI;

wherein the parsing a bitstream to obtain an initial SAI comprises:

parsing the bitstream to obtain a picture pseudo-sequence and a preset arrangement order, wherein the preset arrangement order is any one of a rotation order, a raster scanning order, and a zigzag-shaped canning order; and

generating the initial SAI based on the preset arrangement order and the picture pseudo-sequence.

2 . The method of claim 1 , wherein the inputting the initial SAI into a super-resolution reconstruction net, and outputting a reconstructed SAI comprises:

performing extraction processing based on the initial SAI to obtain an initial Epipolar Plane Image (EPI) set;

performing up-sampling processing and feature extraction on the initial EPI set to obtain a target EPI set, wherein a resolution of a picture in the target EPI set is greater than a resolution of a picture in the initial EPI set; and

performing fusion processing on the target EPI set to obtain the reconstructed SAI.

3 . The method of claim 2 , wherein the performing extraction processing based on the initial SAI to obtain an initial EPI set comprises:

performing sort processing on the initial SAI to obtain a stereo picture set; and

performing extraction processing on the stereo picture set according to at least one direction to obtain at least one initial EPI set, wherein one direction corresponds to one initial EPI set.

4 . The method of claim 3 , wherein the performing fusion processing on the target EPI set to obtain the reconstructed SAI comprises:

performing weighted average fusion on at least one target EPI set corresponding to at least one EPI set to obtain the reconstructed SAI.

5 . The method of claim 2 , wherein the performing up-sampling processing and feature extraction on the initial EPI set to obtain a target EPI set comprises:

parsing the bitstream to obtain a sampling parameter;

performing up-sampling processing on the EPI set according to the sampling parameter to obtain a sampled EPI set;

using one or more convolution layers to perform feature extraction on the sampled EPI set to obtain a feature picture corresponding to the initial EPI set; and

constructing the target EPI set based on the sampled EPI set and the feature picture.

6 . The method of claim 1 , further comprising:

determining a first network parameter corresponding to the super-resolution reconstruction net; and

constructing the super-resolution reconstruction net based on the first network parameter.

7 . The method of claim 6 , wherein the determining a first network parameter corresponding to the super-resolution reconstruction net comprises:

acquiring first training data, wherein the first training data comprises a low-resolution picture and a corresponding high-resolution picture; and

performing model training through the first training data to determine the first network parameter.

8 . The method of claim 6 , wherein the determining a first network parameter corresponding to the super-resolution reconstruction net comprises:

parsing the bitstream to obtain the first network parameter.

9 . The method of claim 1 , further comprising:

determining a second network parameter corresponding to the QENet; and

constructing the QENet based on the second network parameter.

10 . The method of claim 9 , wherein the determining a second network parameter corresponding to the QENet comprises:

acquiring second training data, wherein the second training data comprises a low-quality picture and a corresponding high-quality picture; and

performing model training through the second training data to determine the second network parameter.

11 . The method of claim 9 , wherein the determining a second network parameter corresponding to the QENet comprises:

parsing the bitstream to obtain the second network parameter.

12 . A method for light field picture processing, applied to a light field picture encoder and comprising:

obtaining a lenslet image through collection by a light field camera, and generating a Sub Aperture Image (SAI) according to the lenslet image;

performing down-sampling processing on the SAI to obtain an initial SAI;

generating a picture pseudo-sequence corresponding to the SAI based on a preset arrangement order and the initial SAI, wherein the preset arrangement order is any one of a rotation order, a raster scanning order, and a zigzag-shaped canning order; and

performing encoding processing based on the picture pseudo-sequence to generate a bitstream.

13 . The method of claim 12 , wherein after the generating a picture pseudo-sequence corresponding to the SAI based on a preset arrangement order and the initial SAI, the method further comprises:

signaling a sort parameter in the bitstream, wherein the sort parameter is used for indicating the preset arrangement order.

14 . The method of claim 12 , wherein the performing down-sampling processing on the SAI to obtain an initial SAI comprises:

respectively performing, according to a sampling parameter, down-sampling processing on a spatial resolution and an angular resolution of the SAI to complete the construction of the initial SAI.

15 . The method of claim 14 , wherein after the performing down-sampling processing on the SAI to obtain an initial SAI, the method further comprises:

signaling the sampling parameter in the bitstream.

16 . A light field picture decoder, comprising a first processor, and a first memory that stores instructions executable by the first processor, wherein, when the instructions are executed by the first processor, the first processor is configured to:

parse a bitstream to obtain an initial Sub Aperture Image (SAI);

input the initial SAI into a super-resolution reconstruction net, and outputting a reconstructed SAI, wherein the spatial resolution and the angular resolution of the reconstructed SAI are both greater than the spatial resolution and the angular resolution of the initial SAI; and

input the reconstructed SAI into a Quality Enhancement Net (QENet), and output a target SAI;

wherein first processor is further configured to:

parse the bitstream to obtain a picture pseudo-sequence and a preset arrangement order, wherein the preset arrangement order is any one of a rotation order, a raster scanning order, and a zigzag-shaped canning order; and

generate the initial SAI based on the preset arrangement order and the picture pseudo-sequence.

17 . A light field picture encoder, comprising a second processor, and a second memory that stores instructions executable by the second processor, wherein, when the instructions are executed by the second processor, the second processor is configured to:

obtain a lenslet image through collection by a light field camera, and generating a Sub Aperture Image (SAI) according to the lenslet image;

perform down-sampling processing on the SAI to obtain an initial SAI;

generate a picture pseudo-sequence corresponding to the SAI based on a preset arrangement order and the initial SAI, wherein the preset arrangement order is any one of a rotation order, a raster scanning order, and a zigzag-shaped canning order; and

perform encoding processing based on the picture pseudo-sequence to generate a bitstream.

18 . The light field picture encoder of claim 17 , wherein the second processor is further configured to:

signal a sort parameter in the bitstream, wherein the sort parameter is used for indicating the preset arrangement order.

Assignments (1)

Continuity (2)

References Cited (23)