Granted Patent B2

US 12,591,948 · App. 17/584,662 · Granted Mar 31, 2026

Inventors: Jingu Heo (Yongin-si, KR); Dong Kyung Nam (Yongin-si, KR); Juyong Park (Seongnam-si, KR); Sung Kwang Cho (Seoul, KR)

Assignee: Samsung Electronics Co., Ltd.

G06T3/4038G06T3/18G06T5/50

Abstract

Claims (59)

1 . A method of displaying an image, comprising:

obtaining initial homography information between a plurality of tele images that covers a field of view (FOV) of a wide image;

determining whether a user input of zooming a partial region of the wide image from a screen on which the wide image is displayed exceeds a maximum zoom level;

in response to the zoom exceeding the maximum zoom level, determining whether a number of the tele images corresponding to the partial region is more than one tele image;

in response to the number of the tele images corresponding to the partial region being more than one tele image, stitching tele images corresponding to the partial region using the initial homography information; and

rendering the stitched tele images and displaying an image obtained by the rendering on the screen,

wherein the stitching comprises aligning the multiple tele images corresponding to the partial region based on the initial homography information and removing an artifact occurring in a boundary region of an overlapping region among the tele images by:

determining a deformation field of the boundary region of the tele images by an optical flow of the boundary region; and

warping pixels in the boundary region based on the deformation field.

2 . The method of claim 1 , wherein the stitching in response to the number of the tele images corresponding to the partial region being more than one tele image comprises:

selecting a boundary line candidate from among a plurality of boundary line candidates preset in the overlapping region of the aligned tele images.

3 . The method of claim 2 , wherein the selecting of the boundary line candidate comprises:

determining the boundary line candidate based on pixel values of pixels of a line corresponding to each of the boundary line candidates.

4 . The method of claim 2 , wherein the selecting of the boundary line candidate comprises:

determining the boundary line candidate based on a result of accumulating differences of pixel values of pixels in a boundary region adjacent to the boundary line candidate among the boundary line candidates.

5 . The method of claim 1 , wherein the stitching in response to the number of the tele images corresponding to the partial region being multiple comprises:

scaling the multiple tele images corresponding to the partial region according to a current scale corresponding to the user input, based on the initial homography information; and

stitching the scaled tele images.

6 . The method of claim 1 , wherein the determining whether the number of the tele images corresponding to the partial region is multiple comprises:

determining whether the number of the tele images corresponding to the partial region is multiple based on tiling information of the tele images corresponding to the partial region.

7 . The method of claim 1 , further comprising:

in response to the number of the tele images corresponding to the partial region not being more than one tele image, rendering a tele image corresponding to the partial region and displaying an image obtained by the rendering.

8 . The method of claim 1 , further comprising:

rendering the wide image according to an environment of a display device and displaying an image obtained by the rendering on the screen.

9 . A method of displaying an image, comprising:

obtaining a wide image, a plurality of tele images that cover a field of view (FOV) of the wide image, and initial homography information between the tele images;

receiving a user input of zooming a partial region of the wide image from a screen of a display on which the wide image is displayed;

in response to a zoom corresponding to the user input not exceeding a maximum zoom level of the wide image, rending a wide image scaled according to a scale corresponding to the user input and displaying an image obtained by the rendering on the screen;

in response to the zoom corresponding to the user input exceeding the maximum zoom level of the wide image, scaling tele images corresponding to the partial region according to the scale corresponding to the user input and aligning the scaled tele images, based on the initial homography information;

in response to the number of the tele images corresponding to the partial region being more than one tele image, stitching tele images corresponding to the partial region using the initial homography information; and

removing an artifact occurring by a boundary line candidate selected from among a plurality of boundary line candidates preset in an overlapping region of the aligned tele images; and

rendering tele image from which the artifact is removed and displaying an image obtained by the rendering on the screen,

wherein the stitching comprises aligning the multiple tele images corresponding to the partial region based on the initial homography information and

wherein the removing of the artifact comprises:

determining a deformation field of the boundary region of the tele images by an optical flow of the boundary region; and

warping pixels in the boundary region based on the deformation field.

10 . A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of claim 1 .

11 . An apparatus for displaying an image, comprising:

a camera module configured to obtain a wide image, a plurality of tele images that covers a field of view (FOV) of the wide image, and initial homography information between the tele images;

a display device configured to display at least one of the wide image and the tele images;

a user interface (UI) configured to receive a user input of zooming a partial region of the wide image from a screen of the display device on which the wide image is displayed; and

a processor configured to:

determine whether a zoom corresponding to the user input exceeds a maximum zoom level;

in response to the zoom exceeding the maximum zoom level, determine whether a number of the tele images corresponding to the partial region is more than one tele image;

in response to the number of the tele images corresponding to the partial region being more than one tele image, stitch tele images corresponding to the partial region using the initial homography information; and

render the stitched tele images and display an image obtained by the rendering,

wherein stitching comprises aligning the multiple tele images corresponding to the partial region based on the initial homography information and removing an artifact occurring in a boundary region of an overlapping region among the tele images by:

determining a deformation field of a boundary region of the tele images by an optical flow of the boundary region, and

warping pixels in the boundary region based on the deformation field.

12 . The apparatus of claim 11 , wherein the processor is configured to:

select a boundary line candidate from among a plurality of boundary line candidates preset in the overlapping region of the aligned tele images.

13 . The apparatus of claim 12 , wherein the processor is configured to:

determine the boundary line candidate based on pixel values of pixels of a line corresponding to each of the boundary line candidates.

14 . The apparatus of claim 12 , wherein the processor is configured to:

determine the boundary line candidate based on a result of accumulating differences of pixel values of pixels in a boundary region adjacent to the boundary line candidate among the boundary line candidates.

15 . The apparatus of claim 11 , wherein the processor is configured to:

scale the multiple tele images corresponding to the partial region according to a current scale corresponding to the user input, based on the initial homography information, and stitch the scaled tele images.

16 . The apparatus of claim 11 , wherein the processor is configured to:

determine whether the number of the tele images corresponding to the partial region is more than one tele image based on tiling information of the tele images corresponding to the partial region.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (31)