IP Library › Granted Patent US 11,704,839
Granted Patent B2
US 11,704,839 · App. 17/124,226 · Granted Jul 18, 2023

Multiview video encoding and decoding method

Inventors: Soo Woong Kim (Sejong-si, KR); Jung Won Kang (Daejeon, KR); Gun Bang (Daejeon, KR); Jin Ho Lee (Daejeon, KR); Ha Hyun Lee (Seoul, KR); Sung Chang Lim (Daejeon, KR)
Assignee: Electronics and Telecommunications Research Institute
G06T9/001G06T15/04G06T15/205G06T17/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,704,839
App. No.
17/124,226
Granted
Jul 18, 2023
Kind
B2
Abstract

The present disclosure provides a multi-view image decoding method including: obtaining, from a bitstream, three-dimensional geometry information indicating a three-dimensional space of a multi-view image, view independent component information indicating a view independent component, which is uniformly applied to every view, and view dependent component information indicating a view dependent component, which is differently applied according to views; determining a view dependent component for a texture map of a current view from the view dependent component information; generating the texture map of the current view from a view independent component of the view independent component information and the determined view dependent component for the current view; and reconstructing a current view image according to a three-dimensional space that is constructed according to the texture map of the current view and the three-dimensional geometry information.

Claims (56)

1. A multi-view image decoding method, the method comprising:

obtaining, from a bitstream, three-dimensional geometry information indicating a three-dimensional space of a multi-view image, view independent component information indicating a view independent component, which is uniformly applied to every view, and view dependent component information indicating a view dependent component, which is differently applied according to views;

determining the view dependent component for a texture map of a current view from the view dependent component information;

generating the texture map of the current view from the view independent component of the view independent component information and the determined view dependent component for the current view; and

reconstructing a current view image according to the texture map of the current view and the three-dimensional space that is constructed according to the three-dimensional geometry information,

wherein the three-dimensional space is defined by a triangular mesh structure in the unit of voxels, the triangular mesh structure for each voxel set as one among a plurality of pre-determined triangular mesh structure pattern candidates,

wherein the three-dimensional geometry information is index information for indicating the triangular mesh structure for each voxel from among the plurality of pre-determined triangular mesh structure pattern candidates, and

at least some of the plurality of pre-determined triangular mesh structure pattern candidates are defined at least based on preset UV locations.

2. The multi-view image decoding method of claim 1 ,

wherein the view dependent component information comprises the view dependent component of a texture map for a plurality of captured view images.

3. The multi-view image decoding method of claim 2 ,

wherein the determining of the view dependent component for the current view from the view dependent component information comprises:

determining weights of the plurality of captured view images according to the current view;

calculating a weighted average of view dependent components of the texture map for the plurality of captured view images according to the weights of the plurality of captured view images; and

determining a view dependent component of the texture map for the current view according to the calculated weighted average.

4. The multi-view image decoding method of claim 3 ,

wherein the weights of the plurality of the captured view images according to the current view are determined according to a view angle difference and a view position difference between the current view and each of the plurality of captured view images.

5. The multi-view image decoding method of claim 2 ,

wherein the view independent component is determined from an average texture map of texture maps for the plurality of captured view images, and

wherein the view dependent component of the texture map for the plurality of captured view images is determined according to a difference value between a texture value of the texture map for the plurality of captured view images and a component value of the view independent component.

6. The multi-view image decoding method of claim 1 further comprises

obtaining current view information for the current view,

wherein the current view information comprises position information, rotation information, focus information and center pixel coordinate information of the current view.

7. The multi-view image decoding method of claim 6 ,

wherein the reconstructing of the current view image comprises:

matching a two-dimensional coordinate of the texture map of the current view and a three-dimensional coordinate of the three-dimensional space according to the current view information; and

reconstructing the current view image from the texture map of the current view according to the matched relation between the two-dimensional coordinate and the three-dimensional coordinate.

8. A multi-view image encoding method, the method comprising:

generating, from a plurality of captured view images constituting a multi-view image, three-dimensional geometry information indicating a three-dimensional space of the multi-view image;

generating a texture map of the plurality of captured view images constituting the multi-view image;

obtaining, from the texture map of the plurality of captured view images, view independent component information indicating a view independent component, which is uniformly applied to every view, and view dependent component information indicating a view dependent component, which is differently applied according to views; and

generating a bitstream comprising the three-dimensional geometry information, the view independent component information and the view dependent component information,

wherein the three-dimensional space is defined by a triangular mesh structure in the unit of voxels, the triangular mesh structure for each voxel set as one among a plurality of pre-determined triangular mesh structure pattern candidates,

wherein the three-dimensional geometry information is index information for indicating the triangular mesh structure for each voxel from among the plurality of pre-determined triangular mesh structure pattern candidates, and

wherein at least some of the plurality of pre-determined triangular mesh structure pattern candidates are defined at least based on preset UV locations.

9. The multi-view image encoding method of claim 8 ,

wherein the view dependent component information comprises the view dependent component of a texture map for a plurality of captured view images.

10. The multi-view image encoding method of claim 9 ,

wherein the view independent component is determined from an average texture map of a texture map for the plurality of captured view images, and

wherein a view dependent component of the texture map for the plurality of captured view images is determined according to a difference value between the texture map for the plurality of captured view images and the view independent component.

11. The multi-view image encoding method of claim 8 ,

wherein the generating of the texture map of the plurality of captured view images constituting the multi-view image further comprises obtaining capture view information of the plurality of captured view images, and

wherein the capture view information comprises position information, rotation information, focus information and center pixel coordinate information of the captured view images.

12. The multi-view image encoding method of claim 11 ,

wherein the generating of the texture map of the plurality of captured view images constituting the multi-view image comprises:

matching a two-dimensional coordinate of the texture map of the captured view images and a three-dimensional coordinate of the three-dimensional space according to the capture view information; and

generating the texture map of the plurality of captured view images from the plurality of captured view images according to the matched relation between the two-dimensional coordinate and the three-dimensional coordinate.

13. A non-transitory computer-readable recording medium for storing a bitstream comprising coded data of a multi-view image that is encoded according to a multi-view image encoding method,

wherein the multi-view image encoding method comprises:

generating, from a plurality of captured view images constituting a multi-view image, three-dimensional geometry information indicating a three-dimensional space of the multi-view image;

generating a texture map of the plurality of captured view images constituting the multi-view image;

obtaining, from the texture map of the plurality of captured view images, view independent component information indicating a view independent component, which is uniformly applied to every view, and view dependent component information indicating a view dependent component, which is differently applied according to views; and

generating a bitstream comprising the three-dimensional geometry information, the view independent component information and the view dependent component information,

wherein the three-dimensional space is defined by a triangular mesh structure in the unit of voxels, the triangular mesh structure for each voxel set as one among a plurality of pre-determined triangular mesh structure pattern candidates,

wherein the three-dimensional geometry information is index information for indicating the triangular mesh structure for each voxel from among the plurality of pre-determined triangular mesh structure pattern candidates, and

wherein at least some of the plurality of pre-determined triangular mesh structure pattern candidates are defined at least based on preset UV locations.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 16, 2020
From: KIM, SOO WOONG; KANG, JUNG WON; BANG, GUN; LEE, JIN HO; LEE, HA HYUN; LIM, SUNG CHANG
To: ELECTRONICS AND TELECOMMUNICATIONS RESEARCH INSTITUTE
Reel/Frame 054672/0747 →
Priority Claims (3)
KR 10-2019-0168783 · Dec 17, 2019 · national
KR 10-2019-0179057 · Dec 31, 2019 · national
KR 10-2020-0176820 · Dec 16, 2020 · national
Continuity (1)
Related Publication 20210233282A1 · Jul 29, 2021
Cited By (1)
US 12,614,338