IP Library Granted Patent US 11,095,919
Granted Patent B2
US 11,095,919 · App. 16/194,586 · Granted Aug 17, 2021

Encoder, decoder, encoding method, and decoding method

Inventors: Chong Soon Lim (Singapore, SG); Han Boon Teo (Singapore, SG); Takahiro Nishi (Nara, JP); Tadamasa Toma (Osaka, JP); Ru Ling Liao (Singapore, SG); Sughosh Pavan Shashidhar (Singapore, SG); Hai Wei Sun (Singapore, SG)
Assignee: PANASONIC INTELLECTUAL PROPERTY CORPORATION OF AMERICA
H04N19/597G06T5/006H04N19/159H04N19/176H04N19/46H04N19/85
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,095,919
App. No.
16/194,586
Granted
Aug 17, 2021
Kind
B2
Abstract

An encoder capable of properly handling an image to be encoded or decoded includes processing circuitry and memory connected to the processing circuitry. Using the memory, the processing circuitry: obtains parameters including at least one of (i) one or more parameters related to a first process for correcting distortion in an image captured with a wide angle lens and (ii) one or more parameters related to a second process for stitching a plurality of images; generates an encoded image by encoding a current image to be processed that is based on the image or the plurality of images; and writes the parameters into a bitstream including the encoded image.

Claims (72)

1. An encoder, comprising:

processing circuitry and memory connected to the processing circuitry,

wherein, using the memory, the processing circuitry:

obtains parameters including at least one of (i) one or more parameters related to a first process for correcting distortion in an image captured with a wide angle lens and (ii) one or more parameters related to a second process for stitching a plurality of images;

generates an encoded image by encoding a current image to be processed that is based on the image or the plurality of images; and

writes the parameters into a bitstream including the encoded image, wherein

when writing the parameters, the processing circuitry writes the one or more parameters related to the second process into supplemental enhancement information (SEI) in the bitstream, and

when encoding the current image, when the current image is obtained via the second process, the processing circuitry skips an encoding process on a block by block basis, based on the one or more parameters related to the second process.

2. The encoder according to claim 1 , wherein

when writing the parameters, the processing circuitry writes the parameters into supplemental enhancement information (SEI) in the bitstream,

when encoding the current image, the processing circuitry adapts, on a block by block basis, an encoding process based on the parameters, to encode each block included in the current image, and

the encoding process includes at least one of an inter prediction process and a picture reconstruction process.

3. The encoder according to claim 1 , wherein

when writing the parameters, the processing circuitry writes, as the one or more parameters related to the second process, at least one of a position and a camera angle for each of a plurality of cameras, into supplemental enhancement information (SEI) in the bitstream, and

when encoding the current image, the processing circuitry:

encodes an image from among the plurality of images as the current image; and

stitches the current image with a second image among the plurality of images using the parameters written in the SEI.

4. The encoder according to claim 2 , wherein

the encoding process includes a prediction process, the prediction process being one of the inter prediction process and an intra prediction process, and

the prediction process includes a wrapping process of arranging or rearranging a plurality of pixels included in an image.

5. The encoder according to claim 2 , wherein

the encoding process includes the inter prediction process, and

the inter prediction process includes an image padding process performed on a curved, diagonal, or cornered image boundary using the parameters written in the SEI.

6. The encoder according to claim 2 , wherein

the encoding process includes the inter prediction process and the picture reconstruction process, and

the inter prediction process and the picture reconstruction process each include a process for rewriting a pixel value to a predetermined value based on the parameters written in the SEI.

7. The encoder according to claim 3 , wherein

when encoding the current image, the processing circuitry:

reconstructs the encoded image to generate a reconstructed image; and

stores an image obtained by stitching the reconstructed image and with the second image into the memory as a reference frame to be used in an inter prediction process.

8. A decoder, comprising:

processing circuitry and memory connected to the processing circuitry,

wherein, using the memory, the processing circuitry:

obtains a bitstream including an encoded image;

parses, from the bitstream, parameters including at least one of (i) one or more parameters related to a first process for correcting distortion in an image captured with a wide angle lens and (ii) one or more parameters related to a second process for stitching a plurality of images; and

decodes the encoded image, wherein

when parsing the parameters, the processing circuitry parses the one or more parameters related to the second process from supplemental enhancement information (SEI) in the bitstream, and

when decoding the encoded image, when the encoded image is generated by encoding an image obtained via the second process, the processing circuitry skips a decoding process on a block by block basis, based on the one or more parameters related to the second process.

9. The decoder according to claim 8 , wherein

when parsing the parameters, the processing circuitry parses the parameters from supplemental enhancement information (SEI) in the bitstream,

when decoding the encoded image, the processing circuitry adapts, on a block by block basis, a decoding process based on the parameters, to decode each block included in the encoded image, and

the decoding process includes at least one of an inter prediction process and a picture reconstruction process.

10. The decoder according to claim 8 , wherein

when parsing the parameters, the processing circuitry parses, as the one or more parameters related to the second process, at least one of a position and a camera angle for each of a plurality of cameras, from supplemental enhancement information (SEI) in the bitstream, and

when decoding the encoded image, the processing circuitry:

decodes an image encoded from among the plurality of images as the encoded image; and

stitches the encoded image with a second image among the plurality of images using the parameters parsed from the SEI.

11. The decoder according to claim 9 , wherein

the decoding process includes a prediction process, the prediction process being one of the inter prediction process and an intra prediction process, and

the prediction process includes a wrapping process of arranging or rearranging a plurality of pixels included in an image.

12. The decoder according to claim 9 , wherein

the decoding process includes the inter prediction process, and

the inter prediction process includes an image padding process performed on a curved, diagonal, or cornered image boundary using the parameters parsed from the SEI.

13. The decoder according to claim 9 , wherein

the decoding process includes the inter prediction process and the picture reconstruction process, and

the inter prediction process and the picture reconstruction process each include a process for rewriting a pixel value to a predetermined value based on the parameters parsed from the SEI.

14. The decoder according to claim 10 , wherein

when decoding the encoded image, the processing circuitry:

decodes the encoded image to generate a decoded image; and

stores an image obtained by stitching the decoded image with the second image into the memory as a reference frame to be used in an inter prediction process.

15. An encoding method, comprising:

obtaining parameters including at least one of (i) one or more parameters related to a first process for correcting distortion in an image captured with a wide angle lens and (ii) one or more parameters related to a second process for stitching a plurality of images;

generating an encoded image by encoding a current image to be processed that is based on the image or the plurality of images; and

writing the parameters into a bitstream including the encoded image, wherein

when writing the parameters, the one or more parameters related to the second process are written into supplemental enhancement information (SEI) in the bitstream, and

when encoding the current image, when the current image is obtained via the second process, an encoding process is skipped on a block by block basis, based on the one or more parameters related to the second process.

16. A decoding method, comprising:

obtaining a bitstream including an encoded image;

parsing, from the bitstream, parameters including at least one of (i) one or more parameters related to a first process for correcting distortion in an image captured with a wide angle lens and (ii) one or more parameters related to a second process for stitching a plurality of images; and

decoding the encoded image, wherein

when parsing the parameters, the one or more parameters related to the second process are parsed from supplemental enhancement information (SEI) in the bitstream, and

when decoding the encoded image, when the encoded image is generated by encoding an image obtained via the second process, a decoding process is skipped on a block by block basis, based on the one or more parameters related to the second process.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 25, 2019
From: LIM, CHONG SOON; TEO, HAN BOON; NISHI, TAKAHIRO; TOMA, TADAMASA; LIAO, RU LING; SHASHIDHAR, SUGHOSH PAVAN; SUN, HAI WEI
To: PANASONIC INTELLECTUAL PROPERTY CORPORATION OF AMERICA
Reel/Frame 048988/0289 →
Continuity (3)
Continuation PCTJP2017019113 · May 23, 2017
Provisional Application 62342517 · May 27, 2016
Related Publication 20190110076A1 · Apr 11, 2019