IP Library › Granted Patent US 11,244,422
Granted Patent B2
US 11,244,422 · App. 16/463,980 · Granted Feb 8, 2022

Image processing apparatus and image processing method therefor

Inventors: Il-Hoe Jung (Seoul, KR); In-Su Yu (Seoul, KR); Jin-Ho Lim (Gyeonggi-do, KR); Byung-Po Choi (Gyeonggi-do, KR); Yoon-Joo Kim (Seoul, KR); Byung-Hyun Ahn (Seoul, KR); Jae-Eun Yang (Gyeonggi-do, KR); Dong-Hyun Yeom (Gyeonggi-do, KR)
G06T3/0062G06T3/0018H04N13/111H04N13/167H04N13/178H04N13/194
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,244,422
App. No.
16/463,980
Granted
Feb 8, 2022
Kind
B2
Abstract

An image processing apparatus is disclosed. The image processing apparatus includes a storage unit; a transceiver; and a processor for controlling the storage unit to store an input frame including a plurality of image areas having preset arrangement attributes and metadata including the preset arrangement attributes, control the transceiver to receive viewing angle information, and control the transceiver to transmit the metadata and image data of at least one image region corresponding to the viewing angle information among the plurality of image regions by using at least one of the plurality of transmission channels matched with the plurality of image regions.

Claims (64)

1. An apparatus for processing images, the apparatus comprising:

a storage unit;

a transceiver; and

a processor configured to:

control the storage unit to store a two-dimensional image including a plurality of image areas and metadata including an arrangement attribute that indicates mapping information between each image area of the plurality of the image areas and each face of a plurality of faces of a three-dimensional polyhedron,

control the transceiver to receive viewing angle information from an external electronic device, and

identify at least one image area corresponding to the viewing angle information among the plurality of the image areas,

control the transceiver to transmit, to the external electronic device, the metadata and image data of the identified at least one image area through at least one of a plurality of transmission channels that match the plurality of image areas,

wherein the two-dimensional image is mapped to an omni-directional image of the three-dimensional polyhedron,

wherein the plurality of image areas of the two-dimensional image are mapped to a plurality of image areas of the omni-directional image corresponding to the plurality of faces of the three-dimensional polyhedron, respectively, and

wherein the image data of the identified at least one image area corresponds to a portion of the plurality of image areas of the omni-directional image.

2. The apparatus of claim 1 , wherein the viewing angle information includes indicators indicating the at least one image area corresponding to a field of view of a user among the plurality of image areas.

3. The apparatus of claim 2 , wherein the viewing angle information further includes importance information of the at least one image area corresponding to the field of view of the user among the plurality of image areas, and

wherein, when a number of the at least one image area corresponding to the viewing angle information among the plurality of image areas is plural, the processor is configured to:

set a quality of image data of an image area having a low importance among the image areas corresponding to the viewing angle information to be lower than a preset value, and

control the transceiver to transmit the image data.

4. The apparatus of claim 1 , wherein the viewing angle information includes first information on a size of the viewing angle of the user and second information on a center location of the viewing angle of the user.

5. The apparatus of claim 4 , wherein, when a number of the at least one image area corresponding to the viewing angle information among the plurality of image areas is plural, the processor is configured to:

set a quality of image data of an image area of a location distant from the center location of the viewing angle of the user among the at least one image area corresponding to the viewing angle information to be lower than a preset value, and

control the transceiver to transmit the image data.

6. The apparatus of claim 1 , wherein the metadata further includes at least one piece of unique Identifier (ID) information of content, information on the plurality of transmission channels, type information of a polyhedron corresponding to the plurality of image areas, matching information between the plurality of transmission channels and the plurality of image areas, and quality information of at least one image data among the plurality of image areas corresponding to the viewing angle information.

7. The apparatus of claim 1 , wherein the processor is configured to:

control the transceiver to further transmit audio data of content, and

wherein synchronization of each of the plurality of transmission channels is determined according to synchronization of the audio data.

8. A method of processing images, the method comprising:

controlling a storage unit of an electronic device to store a two-dimensional image comprising a plurality of image areas, and to store metadata comprising an arrangement attribute that indicates mapping information between each of the plurality of the image areas and each of a plurality of faces of a three-dimensional polyhedron;

receiving viewing angle information from an external electronic device;

identifying at least one image area corresponding to the viewing angle information among the plurality of image areas; and

transmitting, to the external electronic device, the metadata and image data of the at least one image area corresponding to the viewing angle information among the plurality of image areas through at least one of a plurality of transmission channels that match the plurality of image areas,

wherein the two-dimensional image is mapped to an omni-directional image of the three-dimensional polyhedron,

wherein the plurality of image areas of the two-dimensional image are mapped to a plurality of image areas of the omni-directional image corresponding to the plurality of faces of the three-dimensional polyhedron, respectively, and

wherein the image data of the identified at least one image area corresponds to a portion of the plurality of image areas of the omni-directional image.

9. The method of claim 8 , wherein the viewing angle information includes indicators indicating at least one image area corresponding to a field of view of a user among the plurality of image areas.

10. The method of claim 9 , wherein the viewing angle information further includes importance information of at least one image area corresponding to the field of view of the user among the plurality of image areas, and

wherein, when a number of the at least one image area corresponding to the viewing angle information among the plurality of image areas is plural, the transmitting of the image data comprises:

setting a quality of image data of an image area having a low importance among the at least one image area corresponding to the viewing angle information to be lower than a preset value; and

transmitting the image data.

11. The method of claim 8 , wherein the viewing angle information includes first information on a size of the viewing angle of the user and second information on a center location of the viewing angle of the user.

12. An apparatus for processing images, the apparatus comprising:

a sensor;

a transceiver; and

a processor configured to:

identify a field of view of a user by using the sensor,

control the transceiver to transmit viewing angle information generated based on the identified field of view of the user to an external electronic device,

control the transceiver to receive, from the external electronic device, metadata including an arrangement attribute included in an image frame and image data of at least one image area corresponding to the viewing angle information among a plurality of image areas through at least one of a plurality of transmission channels that match the plurality of image areas,

map the received image data to at least one face corresponding to the at least one image area among a plurality of faces of a three-dimensional polyhedron, based on the metadata, and

render an output frame corresponding to at least one part of a three-dimensional image that is mapped to the at least one face corresponding to the at least one image area,

wherein the arrangement attribute indicates mapping information between each of the plurality of the image areas and each of the plurality of the faces of a three-dimensional polyhedron,

wherein the plurality of image areas are included in a two-dimensional image,

wherein the two-dimensional image is mapped to an omni-directional image of the three-dimensional polyhedron,

wherein the plurality of image areas of the two-dimensional image are mapped to a plurality of image areas of the omni-directional image corresponding to the plurality of faces of the three-dimensional polyhedron, respectively, and

wherein the image data of the at least one image area corresponds to a portion of the plurality of image areas of the omni-directional image.

13. A method of processing images, the method comprising:

identifying a field of view of a user;

transmitting viewing angle information generated based on the identified field of view of the user to an external electronic device;

receiving, from the external electronic device, metadata including an arrangement attribute included in an image frame;

receiving image data of at least one image area corresponding to the viewing angle information among a plurality of image areas through at least one of a plurality of transmission channels that match the plurality of image areas;

mapping the received image data to at least one face corresponding to the at least one image area among a plurality of faces of a three-dimensional polyhedron, based on the metadata; and

rendering an output frame corresponding to at least one part of a three-dimensional image that is mapped to at least one face corresponding to the at least one image area,

wherein the arrangement attribute indicates mapping information between each of the plurality of the image areas and each of the plurality of the faces of a three-dimensional polyhedron,

wherein the plurality of image areas are included in a two-dimensional image,

wherein the two-dimensional image is mapped to an omni-directional image of the three-dimensional polyhedron,

wherein the plurality of image areas of the two-dimensional image are mapped to a plurality of image areas of the omni-directional image corresponding to the plurality of faces of the three-dimensional polyhedron, respectively, and

wherein the image data of the at least one image area corresponds to a portion of the plurality of image areas of the omni-directional image.

Assignments (1)
CORRECTIVE ASSIGNMENT TO CORRECT THE APPLICATION NUMBER PREVIOUSLY RECORDED AT REEL: 049303 FRAME: 0247. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jun 19, 2019
From: JUNG, IL-HOE; YU, IN-SU; LIM, JIN-HO; CHOI, BYUNG-PO; KIM, YOON-JOO; AHN, BYUNG-HYUN; YANG, JAE-EUN; YEOM, DONG-HYUN
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 050036/0724 →
Priority Claims (1)
KR 10-2016-0158505 · Nov 25, 2016 · national
Continuity (1)
Related Publication 20200388002A1 · Dec 10, 2020