IP Library › Granted Patent US 12,731,417
Granted Patent B2
US 12,731,417 · App. 18/118,445 · Granted Sep 8, 2026

Electronic apparatus and controlling method thereof

Inventors: Dongeui Shin (Suwon-si, KR); Minhee Lee (Suwon-si, KR); Sanghyun Kang (Suwon-si, KR); Hyunsuk Kwak (Suwon-si, KR)
Assignee: SAMSUNG ELECTRONICS CO., LTD.
G06V20/64G05D1/0238G05D1/0246G06T3/4038G06V10/25
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,731,417
App. No.
18/118,445
Filed
Mar 7, 2023
Granted
Sep 8, 2026
Kind
B2
Art Unit
2649
USPC
382/103
Abstract

An electronic apparatus is disclosed. The electronic apparatus comprises: a first camera having a first image capturing angle; a second camera having a second image capturing angle larger than the first image capturing angle; and a processor which when a portion of a target object is identified and an obstacle object overlapping the other portion of the target object is identified from at least one of a first captured image acquired from the first camera and a second captured image acquired from the second camera, acquires location information of the identified obstacle object, identifies a region of interest in the second captured image on the basis of the acquired location information of the obstacle object, and identifies the other portion of the target object on the basis of the region of interest in the second captured image.

Claims (66)

1 . An electronic apparatus comprising:

a first camera having a first angle of view;

a second camera having a second angle of view greater than the first angle of view;

memory storing instructions; and

at least one processor configured to execute the instructions,

wherein the instructions, when executed by the at least one processor, cause the electronic apparatus to:

acquire a first captured image through the first camera,

identify a first portion of a target object and an obstacle object overlapping a second portion of the target object in the first captured image,

identify location information and size information of the obstacle object based on the first captured image,

determine an image capturing location of the second camera based on the identified location information and size information of the obstacle object,

control the second camera to acquire a second captured image in the determined image capturing location,

identify a region of interest of the second captured image based on the location information of the obstacle object,

identify the second portion of the target object based on the region of interest of the second captured image,

generate a combined image including an entire image of the target object by combining a region of the first captured image including the target object and the obstacle object with the region of interest of the second captured image, and

identify the target object based on the combined image, and

wherein the region of interest includes a portion of the target object occluded by the obstacle object.

2 . The electronic apparatus of claim 1 , wherein the first captured image comprises an image captured by the first camera in a first image capturing location, and the second captured image comprises an image captured by the second camera in a second image capturing location different from the first image capturing location.

3 . The electronic apparatus of claim 2 , wherein the instructions, when executed by the at least one processor, further cause the electronic apparatus to:

acquire a third captured image through the second camera in a third image capturing location different from the first image capturing location and the second image capturing location,

identify a region of interest of the third captured image based on the location information of the obstacle object, and

identify the second portion of the target object based on the region of interest of the second captured image and the region of interest of the third captured image.

4 . The electronic apparatus of claim 3 ,

wherein the second image capturing location is located in a first direction relative to the first image capturing location, and

the third image capturing location is located in a second direction different from the first direction relative to the first image capturing location.

5 . The electronic apparatus of claim 4 , wherein the instructions, when executed by the at least one processor, further cause the electronic apparatus to:

identify a region of interest of the second captured image based on direction information corresponding to the second image capturing location and the location information of the obstacle object, and

identify a region of interest of the third captured image based on direction information corresponding to the third image capturing location and the location information of the obstacle object.

6 . The electronic apparatus of claim 1 , wherein the instructions, when executed by the at least one processor, further cause the electronic apparatus to:

acquire first information of the target object based on the first portion of the target object, and

acquire second information of the target object based on the second portion of the target object.

7 . The electronic apparatus of claim 1 , further comprising:

a light emitter,

wherein the instructions, when executed by the at least one processor, further cause the electronic apparatus to:

identify a region of interest of the first captured image,

identify a location corresponding to an area adjacent to the region of interest of the first captured image and the region of interest of the second captured image, and

control the light emitter to irradiate a light greater than or equal to a threshold strength on the identified location.

8 . The electronic apparatus of claim 7 ,

wherein the instructions, when executed by the at least one processor, further cause the electronic apparatus to change a light emitting angle of the light emitter to cause the light to be irradiated on the identified location.

9 . A method of controlling an electronic apparatus, the method comprising:

acquiring a first captured image through a first camera having a first angle of view;

identifying a first portion of a target object and an obstacle object overlapping a second portion of the target object in the first captured image;

identifying location information and size information of the obstacle object based on the first captured image;

determining an image capturing location of a second camera based on the location information and size information of the obstacle object, wherein the second camera has a second angle of view greater than the first angle of view;

controlling the second camera to acquire a second captured image in the determined image capturing location;

identifying a region of interest of the second captured image based on the location information of the obstacle object;

identifying the second portion of the target object based on the region of interest of the second captured image;

generating a combined image including an entire image of the target object by combining a region of the first captured image including the target object and the obstacle object with the region of interest of the second captured image; and

identifying the target object based on the combined image,

wherein the region of interest includes a portion of the target object occluded by the obstacle object.

10 . The method of claim 9 ,

wherein the first captured image comprises an image captured by the first camera in a first image capturing location, and the second captured image comprises an image captured by the second camera in a second image capturing location different from the first image capturing location.

11 . The method of claim 10 , further comprising:

acquiring a third captured image captured through the second camera in a third image capturing location different from the first image capturing location and the second image capturing location;

identifying a region of interest of the third captured image based on the identified location information of the identified obstacle object; and

identifying the second portion of the target object based on the region of interest of the second captured image and the region of interest of the third captured image.

12 . The method of claim 11 ,

wherein the second image capturing location is located in a first direction relative to the first image capturing location, and the third image capturing location is located in a second direction different from the first direction relative to the first image capturing location.

13 . An electronic apparatus comprising:

a first camera having a first angle of view;

a second camera having a second angle of view greater than the first angle of view;

memory storing instructions; and

at least one processor configured to execute the instructions,

wherein the instructions, when executed by the at least one processor, cause the electronic apparatus to:

acquire a first captured image through the first camera,

acquire a second captured image through the second camera, and

based on identification of a first portion of a target object in either the first captured image or the second captured image and identification of an obstacle object overlapping a second portion of the target object in either the first captured image or the second captured image: acquire location information of the identified obstacle object, identify a region of interest of the second captured image based on the acquired location information of the identified obstacle object, and identify the second portion of the target object based on the region of interest of the second captured image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 7, 2023
From: SHIN, DONGEUI; LEE, MINHEE; KANG, SANGHYUN; KWAK, HYUNSUK
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 062910/0399 →
Priority Claims (1)
KR 10-2020-0152728 · Nov 16, 2020 · national
Continuity (2)
Continuation PCTKR2021014242 · Oct 14, 2021
Related Publication 20230222818A1 · Jul 13, 2023
References Cited (38)
US 9258453B2 · Kato et al. · 2016 [cited by applicant]
US 9602720B2 · Okuda et al. · 2017 [cited by applicant]
US 9781340B2 · Shinozaki et al. · 2017 [cited by applicant]
US 11087181B2 · Greene · 2021 [cited by applicant]
US 11237388B2 · Oba et al. · 2022 [cited by applicant]
US 20060192660A1 · Watanabe et al. · 2006 [cited by applicant]
US 20140288413A1 · Hwang et al. · 2014 [cited by applicant]
US 20180020145A1 · Kotfis et al. · 2018 [cited by applicant]
US 20180210452A1 · Shin et al. · 2018 [cited by applicant]
US 20190068943A1 · Nguyen · 2019 [cited by applicant]
US 20190179140A1 · Oba et al. · 2019 [cited by applicant]
US 20190259139A1 · Ichihashi et al. · 2019 [cited by applicant]
US 20200195826A1 · Seiffert et al. · 2020 [cited by applicant]
US 20240265707A1 · Peppoloni · 2024 [cited by examiner]
CN 108932275A · 2018 [cited by applicant]
JP 200234055A · 2002 [cited by applicant]
JP 3913186B2 · 2007 [cited by applicant]
JP 2012235198A · 2012 [cited by applicant]
JP 5610867B2 · 2014 [cited by applicant]
KR 1020120060339A · 2012 [cited by applicant]
KR 1020180059099A · 2018 [cited by applicant]
KR 1020180080936A · 2018 [cited by applicant]
KR 1020190030953A · 2019 [cited by applicant]
KR 1020190067578A · 2019 [cited by applicant]
WO 2017091008A1 · 2017 [cited by applicant]
WO 2018037790A1 · 2018 [cited by applicant]
WO 2019098567A1 · 2019 [cited by applicant]
WO WO2020024684A1 · 2020 [cited by examiner]
Khan, S.M. and Shah, M., 2008. Tracking multiple occluding people by localizing on multiple scene planes. IEEE transactions on pattern analysis and machine intelligence, 31(3), pp. 505-519. [cited by examiner]
Jabborov, F. and Cho, J., 2020. Image-based camera localization algorithm for smartphone cameras based on reference objects. Wireless Personal Communications, 114(3), pp. 2511-2527. [cited by examiner]
Communication dated Feb. 3, 2022, issued by the International Searching Authority in counterpart International Application No. PCT/KR2021/014242 (PCT/ISA/210). [cited by applicant]
Communication dated Feb. 3, 2022, issued by the International Searching Authority in counterpart International Application No. PCT/KR2021/014242 (PCT/ISA/237). [cited by applicant]
Zhao Pei et al., “Occluded-Object 3D Reconstruction Using Camera Array Synthetic Aperture Imaging,” Sensors, 2019, 19, 607; doi:10.3390/s19030607, XP 093107926, Total 22 pages. [cited by applicant]
Communication issued on Dec. 12, 2023 by European Patent Office in corresponding European Application No. 21892165.8. [cited by applicant]
Communication dated Mar. 31, 2025, issued by the European Patent Office in counterpart European Application No. 21892165.8. [cited by applicant]
Communication dated Nov. 29, 2025, issued by the Korean Ministry of Intellectual Property in Korean Application No. 10-2020-0152728. [cited by applicant]
Communication dated Nov. 30, 2025, issued by the Chinese National Intellectual Property Administration in Chinese Application No. 202180069624.7. [cited by applicant]
Office Action issued on May 16, 2026 by the Chinese Patent Office in corresponding CN Patent Application No. 202180069624.7. [cited by applicant]