IP Library Granted Patent US 12,306,003
Granted Patent B2
US 12,306,003 · App. 18/408,939 · Granted May 20, 2025

Image processing apparatus, image processing method, computer program and computer readable recording medium

Inventors: Ki Wook Lee (Seongnam-si, KR); Hye Kyung Byun (Seongnam-si, KR); Tae Kyu Han (Seongnam-si, KR); Shin Hyoung Kim (Seongnam-si, KR); Jeong Kyu Kang (Seongnam-si, KR)
Assignee: THINKWARE CORPORATION
G01C21/3602G01C21/005G01C21/32G06F16/23G06T1/0007G06T5/50G06T7/20G06T7/254G01S5/16G06T2207/10004G06T2207/20221G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,306,003
App. No.
18/408,939
Filed
Jan 10, 2024
Granted
May 20, 2025
Kind
B2
Art Unit
3661
USPC
701/450
Abstract

Disclosed is an image processing method. The method includes the steps of receiving an image obtained from a plurality of vehicles positioned on a road, storing the received images according to acquisition information of the received images; determining a reference image and a target image based on images having the same acquisition information among the stored images, performing an image registration using a plurality of feature points extracted from each of the determined reference image and target image, performing a transparency process for each of the reference image and the target image performed with the image registration, extracting static objects from the transparency-processed image, and comparing the extracted static objects with objects on an electronic map pre-stored to updating the electronic map data, when the objects on the electronic map data pre-stored are different from the extracted static objects.

Claims (43)

1. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions which, when executed by an electronic device, cause the electronic device to:

receive, from a plurality of external electronic devices in a plurality of vehicles, information regarding a plurality of images obtained by a plurality of cameras which are related to the plurality of electronic devices and face at least one direction to an area, wherein the plurality of images comprise a plurality of visual objects related to a plurality of traffic objects;

identify, based on the plurality of images, at least one road infrastructure among the plurality of the traffic objects located in the area; and

update, by adding at least one visual object indicating the at least one road infrastructure into the digital map, the digital map with respect to the area.

2. The non-transitory computer readable storage medium of claim 1 , wherein the one or more programs comprise instructions which, when executed by the electronic device, further cause the electronic device to, in response to updating the digital map with respect to the area, transmit the updated digital map to at least one vehicle in the area.

3. The non-transitory computer readable storage medium of claim 1 , wherein the one or more programs comprise instructions which, when executed by the electronic device, further cause the electronic device to:

in response to receiving the information regarding the plurality of images, identify, from among the plurality of images, an image as a reference image and identify remaining images except the image from among the plurality of images as target images; and

identify, based on the reference image, first feature points and identify, based on the remaining images, second feature points.

4. The non-transitory computer readable storage medium of claim 3 , wherein the one or more programs comprise instructions which, when executed by the electronic device, further cause the electronic device to:

obtain, based on the reference image and the remaining images, the registration images by matching the first feature points and the second feature points; and

execute aggregation by combining the registration images.

5. The non-transitory computer readable storage medium of claim 4 , wherein the one or more programs comprise instructions which, when executed by the electronic device, further cause the electronic device to, identify, based on the aggregation, the at least one road infrastructure among the plurality of the traffic objects located in the area.

6. The non-transitory computer readable storage medium of claim 1 , wherein the at least one visual object indicating the at least one road infrastructure is newly represented in the digital map after updating the digital map with respect to the area.

7. The non-transitory computer readable medium of claim 1 , wherein the at least one road infrastructure comprises at least one of a bridge, a building, a road, a sidewalk, a road construction mark, a speed bump, a crosswalk, an intersection, a traffic light, a median strip, a bus stop, or a directional indication.

8. A method for operating an electronic device, the method comprising:

receiving, from a plurality of external electronic devices in a plurality of vehicles, information regarding a plurality of images obtained by a plurality of cameras which are related to the plurality of electronic devices and face at least one direction to an area, wherein the plurality of images comprise a plurality of visual objects related to a plurality of traffic objects,

identifying, based on the plurality of images, at least one road infrastructure among the plurality of the traffic objects located in the area; and

updating, by adding at least one visual object indicating the at least one road infrastructure into a digital map, the digital map, with respect to the area.

9. The method of claim 8 , wherein the method further comprises, in response to updating the digital map with respect to the area, transmitting the updated digital map to at least one vehicle in the area.

10. The method of claim 8 , wherein the method further comprises:

in response to receiving the information regarding the plurality of images, identifying, from among the plurality of images, an image as a reference image and identify remaining images except the image from among the plurality of images as target images; and

identifying, based on the reference image, first feature points and identify, based on the remaining images, second feature points.

11. The method of claim 10 , wherein the method further comprises:

obtaining, based on the reference image and the remaining images, the registration images by matching the first feature points and the second feature points; and

executing aggregation by combining the registration images.

12. The method of claim 11 , wherein the method further comprises identifying, based on the aggregation, the at least one road infrastructure among the plurality of the traffic objects located in the area.

13. The method of claim 8 , wherein the at least one visual object indicating the at least one road infrastructure is newly represented in the digital map after updating the digital map with respect to the area.

14. The method of claim 8 , wherein the at least one road infrastructure comprises at least one of a bridge, a building, a road, a sidewalk, a road construction mark, a speed bump, a crosswalk, an intersection, a traffic light, a median strip, a bus stop, or a directional indication.

15. An electronic device comprising:

a memory configured to store instructions; and

a processor configured to execute the instructions to:

receive, from a plurality of external electronic devices in a plurality of vehicles, information regarding a plurality of images obtained by a plurality of cameras which are related to the plurality of electronic devices and face at least one direction to an area, wherein the plurality of images comprise a plurality of visual objects related to a plurality of traffic objects,

identify, based on the plurality of images, at least one road infrastructure among the plurality of the traffic objects located in the area; and

update, by adding at least one visual object indicating the at least one road infrastructure into the digital map, the digital map with respect to the area.

16. The electronic device of claim 15 , wherein the processor is further configured to, in response to updating the digital map with respect to the area, transmit the updated digital map to at least one vehicle in the area.

17. The electronic device of claim 15 , wherein the processor is further configured to:

in response to receiving the information regarding the plurality of images, identify, from among the plurality of images, an image as a reference image and identify remaining images except the image from among the plurality of images as target images; and

identify, based on the reference image, first feature points and identify, based on the remaining images, second feature points.

18. The electronic device of claim 17 , wherein the processor is further configured to:

obtain, based on the reference image and the remaining images, the registration images by matching the first feature points and the second feature points; and

execute aggregation by combining the registration images.

19. The electronic device of claim 18 , wherein the processor is further configured to identify, based on the aggregation, the at least one road infrastructure among the plurality of the traffic objects located in the area.

20. The electronic device of claim 15 , wherein the at least one visual object indicating the at least one road infrastructure is newly represented in the digital map after updating the digital map with respect to the area.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 10, 2024
From: LEE, KI WOOK; BYUN, HYE KYUNG; KIM, SHIN HYOUNG; HAN, TAE KYU; KANG, JEONG KYU
To: THINKWARE CORPORATION
Reel/Frame 066081/0379 →
Priority Claims (2)
KR 10-2016-0158832 · Nov 26, 2016 · national
KR 10-2017-0148115 · Nov 8, 2017 · national
Continuity (6)
Continuation 17464953 · Sep 2, 2021
Continuation In Part 17308105 · May 5, 2021
Continuation 17011610 · Sep 3, 2020
Continuation 16226759 · Dec 20, 2018
Continuation 15822705 · Nov 27, 2017
Related Publication 20240142254A1 · May 2, 2024
References Cited (31)
US 5555312A · Shima et al. · 1996 [cited by applicant]
US 6047234A · Cherveny et al. · 2000 [cited by applicant]
US 6789015B2 · Tsuji et al. · 2004 [cited by applicant]
US 7089110B2 · Pechatnikov et al. · 2006 [cited by applicant]
US 7356408B2 · Tsuchiya et al. · 2008 [cited by applicant]
US 8605947B2 · Zhang et al. · 2013 [cited by applicant]
US 10068373B2 · Lee et al. · 2018 [cited by applicant]
US 11841240B2 · Lenz · 2023 [cited by examiner]
US 20040143380A1 · Stam et al. · 2004 [cited by applicant]
US 20060195858A1 · Takahashi et al. · 2006 [cited by applicant]
US 20070088497A1 · Jung · 2007 [cited by applicant]
US 20080243383A1 · Lin · 2008 [cited by applicant]
US 20130114893A1 · Alakuijala · 2013 [cited by applicant]
US 20140297185A1 · Lindner · 2014 [cited by applicant]
US 20150284010A1 · Beardsley et al. · 2015 [cited by applicant]
US 20200018821A1 · Matsuo et al. · 2020 [cited by applicant]
US 20210133465A1 · Akamine et al. · 2021 [cited by applicant]
US 20210158779A1 · Singh · 2021 [cited by applicant]
US 20210247201A1 · Hori et al. · 2021 [cited by applicant]
US 20210279481A1 · Son · 2021 [cited by applicant]
US 20230341240A1 · Seitle · 2023 [cited by examiner]
CN 105260988A · 2016 [cited by applicant]
JP 2002243469A · 2002 [cited by applicant]
JP 2008235989A · 2008 [cited by applicant]
WO 2009059766A1 · 2009 [cited by applicant]
Taneja et al., “Image Based Detection of Geometric Changes in Urban Environments”, IEEE International Conference on Computer Vision, 2011, pp. 2336-2343, CN Office Action dated Apr. 30, 2021. (8 pages). [cited by applicant]
Office Action dated Apr. 30, 2021, issued in CN application No. 201711203339.1 with English translation. (26 pages). [cited by applicant]
Emanuele Palazzolo et al., “Change Detection in 3D Model Based on Camera Images”, IEEE Transactions on Consumer Electronics; Jan. 1, 2011. pp 1465-1470. (6 pages). [cited by applicant]
Office Action dated Apr. 21, 2022, issued in KR Application No. 10-2017-0148115 (counterpart to U.S. Appl. No. 17/011,610), with English translation. (6 pages). [cited by applicant]
Notice of Allowance dated Feb. 24, 2022, issued in U.S. Appl. No. 17/011,610. (17 pages). [cited by applicant]
Notice of Allowance dated Nov. 23, 2022, issued in U.S. Appl. No. 17/308,105. (22 pages). [cited by applicant]