IP Library › Granted Patent US 11,694,303
Granted Patent B2
US 11,694,303 · App. 16/822,654 · Granted Jul 4, 2023

Method and apparatus for providing 360 stitching workflow and parameter

Inventors: Hee Kyung Lee (Daejeon, KR); Gi Mun Um (Seoul, KR); Jeong Il Seo (Daejeon, KR); Seong Yong Lim (Daejeon, KR); Joung Il Yun (Daejeon, KR); Won Sik Cheong (Daejeon, KR)
Assignee: Electronics and Telecommunications Research Institute
G06T3/4038G06F9/3836G06F9/48G06T19/006G06T2210/52
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,694,303
App. No.
16/822,654
Granted
Jul 4, 2023
Kind
B2
Abstract

Disclosed herein is a method of creating an image stitching workflow including acquiring 360-degree virtual reality (VR) image parameters necessary to makes a request for image stitching and create the image stitching workflow, acquiring a list of functions applicable to the image stitching workflow, creating the image stitching workflow based on functions selected from the list of functions, determining the number of media processing entities necessary to perform tasks configuring the image stitching workflow and generating a plurality of media processing entities according to the determined number of media processing entities, and allocating the tasks configuring the image stitching workflow to the plurality of media processing entities.

Claims (57)

1. A method of creating an image stitching workflow, the method comprising:

acquiring 360-degree virtual reality (VR) image workflow descriptors necessary to make a request for image stitching and create the image stitching workflow;

acquiring function descriptors applicable to the image stitching workflow from a function registry;

creating the image stitching workflow based on the function descriptors selected from the function registry;

determining configurations of media processing entities necessary to perform tasks for the image stitching workflow based on the 360-degree VR image workflow descriptors;

configuring a plurality of media processing entities according to the configurations of media processing entities; and

allocating the tasks configuring the image stitching workflow to the plurality of media processing entities,

wherein the function descriptors include configuration parameters, and

wherein the configuration parameters include a feature parameter, a camera parameter, a projection parameter, and a stitching parameter,

wherein the feature parameter includes a feature extraction method parameter indicating a method to extract features,

wherein the camera parameter includes at least one of a focal length parameter, a principal point parameter, a camera skew value parameter, or a camera transformation parameter,

wherein the projection parameter includes a projection type parameter, and

wherein the stitching parameter includes at least one of a seam position parameter or a seam mask parameter.

2. The method according to claim 1 , further comprising:

determining configuration information of a current task configuring the image stitching workflow and transmitting the configuration information of the current task to a task manager; and

acquiring, from the task manager, access information of a next task performed immediately after the current task, when configuration of the current task is confirmed by the task manager.

3. The method according to claim 2 , further comprising, when a content-dependent parameter according to content among the 360-degree VR workforce descriptors is changed, updating configuration information of a task according to the changed content-dependent parameter.

4. The method according to claim 1 , wherein, when it is requested to stop the image stitching workflow, the acquiring of the 360-degree VR image workflow descriptors to the allocating of the tasks configuring the image stitching workflow to the plurality of media processing entities are stopped.

5. The method according to claim 1 , wherein the tasks configuring the image stitching workflow include:

decoding an encoded raw image;

extracting a feature point of the decoded raw image;

extracting a camera parameter based on the feature point;

generating a 360-degree VR image, by projecting the decoded raw image onto a projection object based on the camera parameter;

determining seam information of the 360-degree VR image based on the camera parameter;

postprocessing the 360-degree VR image based on the seam information; and

encoding the postprocessed 360-degree VR image.

6. The method according to claim 1 , wherein the 360-degree VR image workflow descriptors include at least one of a general descriptor parameter, an input descriptor parameter, an output descriptor parameter, a processing descriptor parameter, a requirement descriptor parameter, a client assistance descriptor parameter, a failover descriptor parameter, a monitoring descriptor parameter or a reporting descriptor parameter.

7. An image stitching workflow creation apparatus for creating an image stitching workflow, the image stitching workflow apparatus comprising one or more processors and one or more memory devices,

wherein the one or more processors perform one or more instructions of a program for creating the image stitching workflow stored in the one or more memory devices,

wherein the one or more memory devices stores the program for creating the image stitching workflow, and

wherein the one or more instructions include:

acquiring 360-degree virtual reality (VR) image workflow descriptors necessary to make a request for image stitching and create the image stitching workflow;

acquiring function descriptors applicable to the image stitching workflow from a function registry;

creating the image stitching workflow based on the function descriptors selected from the function registry;

determining configurations of media processing entities necessary to perform tasks for the image stitching workflow based on the 360-degree VR image workflow descriptors;

configuring a plurality of media processing entities according to the configurations of media processing entities; and

allocating the tasks configuring the image stitching workflow to the plurality of media processing entities,

wherein the function descriptors include configuration parameters, and

wherein the configuration parameters include a feature parameter, a camera parameter, a projection parameter, and a stitching parameter,

wherein the feature parameter includes a feature extraction method parameter indicating a method to extract features,

wherein the camera parameter includes at least one of a focal length parameter, a principal point parameter, a camera skew value parameter, or a camera transformation parameter,

wherein the projection parameter includes a projection type parameter, and

wherein the stitching parameter includes at least one of a seam position parameter or a seam mask parameter.

8. The image stitching workflow creation apparatus according to claim 7 , wherein the one or more instructions include:

determining configuration information of a current task configuring the image stitching workflow and transmitting the configuration information of the current task to a task manager; and

acquiring, from the task manager, access information of a next task performed immediately after the current task, when successful configuration of the current task is confirmed by the task manager.

9. The image stitching workflow creation apparatus according to claim 8 , wherein the one or more instructions include, when a content-dependent parameter according to content among the 360-degree VR workflow descriptors is changed, updating configuration information of a task according to the changed content-dependent parameter.

10. The image stitching workflow creation apparatus according to claim 7 , wherein, when it is requested to stop the image stitching workflow, the performing of the one or more instructions is stopped.

11. The image stitching workflow creation apparatus according to claim 7 , wherein the tasks configuring the image stitching workflow include:

decoding an encoded raw image;

extracting a feature point of the decoded raw image;

extracting a camera parameter based on the feature point;

generating a 360-degree VR image, by projecting the decoded raw image onto a projection object based on the camera parameter;

determining seam information of the 360-degree VR image based on the camera parameter;

postprocessing the 360-degree VR image based on the seam information; and

encoding the postprocessed 360-degree VR image.

12. The image stitching workflow creation apparatus according to claim 7 , wherein the 360-degree VR image workflow descriptors include at least one of a general descriptor parameter, an input descriptor parameter, an output descriptor parameter, a processing descriptor parameter, a requirement descriptor parameter, a client assistance descriptor parameter, a failover descriptor parameter, a monitoring descriptor parameter or a reporting descriptor parameter.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 18, 2020
From: LEE, HEE KYUNG; UM, GI MUN; SEO, JEONG IL; LIM, SEONG YONG; YUN, JOUNG IL; CHEONG, WON SIK
To: ELECTRONICS AND TELECOMMUNICATIONS RESEARCH INSTITUTE
Reel/Frame 052154/0380 →
Priority Claims (4)
KR 10-2019-0031415 · Mar 19, 2019 · national
KR 10-2019-0035223 · Mar 27, 2019 · national
KR 10-2019-0080820 · Jul 4, 2019 · national
KR 10-2020-0033089 · Mar 18, 2020 · national
Continuity (1)
Related Publication 20200302575A1 · Sep 24, 2020
Cited By (2)
US 12,321,386 US 12,530,226