IP Library › Granted Patent US 12,579,614
Granted Patent B2
US 12,579,614 · App. 17/978,489 · Granted Mar 17, 2026

Imaging device

Inventors: Bongki Son (Suwon-si, KR); Jeongyong Shin (Suwon-si, KR); Seunghyun Lim (Suwon-si, KR)
Assignee: SAMSUNG ELECTRONICS CO., LTD.
G06T5/50G06T3/4038G06T3/60G06T2207/20221
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,579,614
App. No.
17/978,489
Granted
Mar 17, 2026
Kind
B2
Abstract

An imaging device includes an image sensor configured to generate a plurality of original images, and an image processor configured to generate a merged image based on the plurality of original images. The image processor is configured to generate the merged image by classifying the plurality of original images into a plurality of stitching groups based on a first stitching direction, stitching original images included in a same stitching group to generate a plurality of intermediate images, and stitching the plurality of intermediate images based on a second stitching direction to generate the merged image.

Claims (45)

1 . An imaging device comprising:

an image sensor configured to generate a plurality of original images of a target object; and

an image processor configured to generate a merged image based on the plurality of original images,

wherein the image processor is configured to

classify the plurality of original images into a plurality of stitching groups based on a first stitching direction,

generate a plurality of intermediate images by stitching original images from the plurality of original images included in a same stitching group from among the plurality of stitching groups,

generate the merged image by stitching the plurality of intermediate images based on a second stitching direction, and

stitch the original images included in the same stitching group by blending the original images included in the same stitching group based on a conversion matrix calculated in an overlap region between the original images included in the same stitching group and a seam detected in the overlap region.

2 . The imaging device of claim 1 , wherein one of the first stitching direction and the second stitching direction is a row direction, and another of the first stitching direction and the second stitching direction is a column direction.

3 . The imaging device of claim 2 , further comprising an optical device configured to rotate based on rotation information, the rotation information including a direction and an angle to adjust fields of view of the plurality of original images,

wherein the image processor is configured to determine the first stitching direction and the second stitching direction based on the rotation information.

4 . The imaging device of claim 1 , wherein the image processor is configured to calculate the conversion matrix based on a feature point of the overlap region between the original images included in the same stitching group.

5 . The imaging device of claim 1 , further comprising a memory configured to store the conversion matrix,

wherein the image processor is configured to blend the original images included in the same stitching group based on the conversion matrix read from the memory.

6 . The imaging device of claim 1 , wherein the image processor is configured to correct the original images included in the same stitching group.

7 . The imaging device of claim 1 , wherein the image processor is configured to crop a region in the original images included in the same stitching group.

8 . The imaging device of claim 7 , further comprising a memory configured to store image processing data,

wherein the image processor is configured to optimize the memory by cropping the region in the original images included in the same stitching group, and blending the original images included in the same stitching group.

9 . The imaging device of claim 1 , wherein the image processor is configured to stitch the plurality of intermediate images by blending the plurality of intermediate images based on the conversion matrix.

10 . An imaging device for generating a merged image based on a plurality of original images, the imaging device comprising:

a memory configured to store conversion matrixes for warping the plurality of original images, and a plurality of cropping regions of the plurality of original images;

an image sensor configured to generate the plurality of original images of a target object, warp the plurality of original images by using the conversion matrixes to generate a plurality of warping images, and crop the plurality of warping images by using the plurality of cropping regions to generate a plurality of cropping images;

an image processor configured to receive the plurality of cropping images from the image sensor, and blend the plurality of cropping images to generate the merged image; and

an optical device configured to collect light from the target object and rotate to adjust fields of view of the plurality of original images generated by the image sensor based on rotation information, the rotation information including a direction and an angle,

wherein the conversion matrixes and the plurality of cropping regions are calculated based on the rotation information, and

the image processor is configured to generate the merged image based on a conversion matrix from among the conversion matrixes calculated in an overlap region between original images from among the plurality of original images, and a seam detected in the overlap region.

11 . The imaging device of claim 10 , further comprising a controller configured to

transmit a rotation signal to the optical device, and

transmit a photograph signal, corresponding to the rotation signal, to the image sensor.

12 . The imaging device of claim 11 , wherein the image sensor is configured to photograph the target object to generate the plurality of original images, in response to the photograph signal.

13 . The imaging device of claim 10 , wherein the plurality of cropping regions are calculated to include regions which are not used in the plurality of warping images to generate the merged image.

14 . The imaging device of claim 13 , wherein the image sensor is configured to crop the plurality of cropping regions in the plurality of warping images to generate the plurality of cropping images.

15 . The imaging device of claim 10 , wherein

the image processor is configured to generate the merged image based on the plurality of cropping images received from the image sensor including N cropping images, where N is a natural number greater than or equal to 2.

16 . The imaging device of claim 10 , wherein

the merged image is generated based on the plurality of original images arranged in a first direction and a second direction, and

the plurality of cropping regions are set with the plurality of warping images cropped in one direction selected from among the first direction and the second direction.

17 . An imaging device comprising:

a memory configured to store conversion matrixes for warping original images, and cropping regions of the original images;

an image sensor configured to, in response to a photograph signal, photograph a target object to generate the original images, warp the original images by using the conversion matrixes to generate warping images, and crop the warping images by using the cropping regions to generate cropping images;

an optical device configured to collect light from the target object and rotate to adjust fields of view of the original images generated by the image sensor based on rotation information, the rotation information including a direction and an angle;

a controller configured to transmit a rotation signal to the optical device, and generate the photograph signal corresponding to the rotation signal; and

an application processor configured to receive the cropping images from the image sensor, stitch the cropping images in a first stitching direction and a second stitching direction based on the rotation information and a conversion matrix from among the conversion matrixes, and blend the cropping images to generate a merged image,

wherein the application processor is configured to generate the merged image based on the conversion matrix calculated in an overlap region between the original images included in a same stitching group, and a seam in the overlap region, and

wherein one of the first stitching direction and the second stitching direction is a row direction, and another one of the first stitching direction and the second stitching direction is a column direction.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 15, 2022
From: SON, BONGKI; SHIN, JEONGYONG; LIM, SEUNGHYUN
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 061948/0806 →
Priority Claims (4)
KR 10-2021-0155151 · Nov 11, 2021 · national
KR 10-2021-0155152 · Nov 11, 2021 · national
KR 10-2022-0029468 · Mar 8, 2022 · national
KR 10-2022-0085879 · Jul 12, 2022 · national
Continuity (1)
Related Publication 20230146691A1 · May 11, 2023
References Cited (30)
US 5657402A · Bender · 1997 [cited by examiner]
US 5991444A · Burt · 1999 [cited by examiner]
US 6215914B1 · Nakamura · 2001 [cited by examiner]
US 7327899B2 · Liu et al. · 2008 [cited by applicant]
US 8554014B2 · Levy et al. · 2013 [cited by applicant]
US 8588546B2 · Habuka et al. · 2013 [cited by applicant]
US 8744213B2 · Tatke et al. · 2014 [cited by applicant]
US 9088714B2 · Doepke · 2015 [cited by examiner]
US 9390324B1 · Ariga et al. · 2016 [cited by applicant]
US 9641754B2 · Oh · 2017 [cited by examiner]
US 9652856B2 · Takenaka · 2017 [cited by examiner]
US 10013763B1 · Brailovskiy et al. · 2018 [cited by applicant]
US 10068334B2 · Wang et al. · 2018 [cited by applicant]
US 10257494B2 · Sadi et al. · 2019 [cited by applicant]
US 10313656B2 · Sadi · 2019 [cited by examiner]
US 10585344B1 · Lablans · 2020 [cited by applicant]
US 10728474B2 · Douady et al. · 2020 [cited by applicant]
US 10991073B2 · Cho et al. · 2021 [cited by applicant]
US 11064110B2 · Douady-Pleven et al. · 2021 [cited by applicant]
US 11455709B2 · Wang · 2022 [cited by examiner]
US 20110141225A1 · Stec · 2011 [cited by examiner]
US 20180025478A1 · Lee et al. · 2018 [cited by applicant]
US 20190058811A1 · Douady-Pleven · 2019 [cited by examiner]
US 20200186710A1 · Sheikh et al. · 2020 [cited by applicant]
US 20210287384A1 · Son et al. · 2021 [cited by applicant]
US 20220414825A1 · Hsu · 2022 [cited by examiner]
JP 2002315980A · 2002 [cited by examiner]
JP 4001233B2 · 2007 [cited by applicant]
KR 101885778B1 · 2018 [cited by applicant]
KR 102075454B1 · 2020 [cited by applicant]