IP Library Granted Patent US 12,223,603
Granted Patent B2
US 12,223,603 · App. 17/991,791 · Granted Feb 11, 2025

Method of learning a target object using a virtual viewpoint camera and a method of augmenting a virtual model on a real object implementing the target object using the same

Inventors: Ki Young Kim (Seoul, KR); Thorsten Korpitsch (Schwechat Wien, AT)
Assignee: VIRNECT INC.
G06T19/006G06T15/20G06T19/20G06V10/74G06V20/20G06T2215/16G06T2219/2016
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,223,603
App. No.
17/991,791
Granted
Feb 11, 2025
Kind
B2
Abstract

Provided is a method of learning a target object implemented on a computer-aided design program of an authoring computing device using a virtual viewpoint camera, including displaying a digital model of a target object that is a target for image recognition, setting at least one observation area surrounding the digital model of the target object and having a plurality of viewpoints on the digital model, generating a plurality of pieces of image data obtained by viewing the digital model of the target object at the plurality of viewpoints of the at least one observation area, and generating object recognition library data for recognizing a real object implementing the digital model of the target object based on the generated plurality of pieces of image data.

Claims (19)

1. A method of learning a target object implemented on a computer-aided design program of an authoring computing device using a virtual viewpoint camera, comprising:

displaying a digital model of a target object that is a target for image recognition;

setting at least one observation area surrounding the digital model of the target object and having a plurality of viewpoints on the digital model, wherein the viewpoints include a plurality of active viewpoints and a plurality of inactive viewpoints;

generating a plurality of pieces of image data obtained by viewing the digital model of the target object at the plurality of viewpoints of the at least one observation area; and

generating object recognition library data for recognizing a real object implementing the digital model of the target object based on the generated plurality of pieces of image data,

wherein the observation area has a hemispherical shape, and the viewpoints closest to each other among the plurality of viewpoints are spaced apart at regular intervals,

wherein an observation viewpoint of the digital model is set by converting at least some of the inactive viewpoints into the active viewpoints or converting all inactive viewpoints into the active viewpoints, wherein the all inactive viewpoints are equally adjusted the distances between the plurality of inactive points.

2. The method of claim 1 , wherein the generating the plurality of pieces of image data comprises generating a plurality of pieces of image data obtained by viewing the digital model of the target object at the plurality of viewpoints in a direction of an origin of the hemispherical shape.

3. The method of claim 1 , wherein the setting the observation area comprises setting a plurality of hemispherical observation areas each having the same origin and different radii in order to observe the digital model of the target object by distance.

4. The method of claim 3 , wherein the setting the observation area comprises rotating any one of the plurality of hemispherical observation areas around the origin in order to observe the digital model of the target object at different viewpoints by distance.

5. The method of claim 1 , further comprising changing a position of the digital model of the target object with respect to the observation area.

6. The method of claim 5 , further comprising allowing the digital model to be observed only at the active viewpoints among the plurality of viewpoints of the observation area.

7. The method of claim 2 , further comprising adjusting a magnification of the target object based on distances from the viewpoints to the origin such that the digital model of the target object has the same size as a size of the real object when the distance from a shooting point to the digital model at a viewpoint is the same as the distance from a camera of a user computing device to the real object.

8. The method of claim 1 , further comprising generating augmented content, matching the augmented content to the digital model of the target object, and storing the augmented content.

9. A method of augmenting a virtual model on a real object implementing a trained target object using a virtual viewpoint camera, performed by an augmented reality program of a terminal including a camera, the method comprising:

receiving and storing the generated object recognition library data of claim 1 ;

acquiring a captured image by photographing a surrounding environment;

detecting a real object matching the stored object recognition library data in the acquired captured image; and

displaying augmented content on the detected real object such that the augmented content matches the real object.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 11, 2023
From: VIRNECT INC.
To: VIRNECT CO., LTD.
Reel/Frame 064252/0252 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 28, 2022
From: KIM, KI YOUNG; KORPITSCH, THORSTEN
To: VIRNECT INC.
Reel/Frame 061895/0600 →
Priority Claims (1)
KR 10-2021-0191588 · Dec 29, 2021 · national
Continuity (1)
Related Publication 20230206566A1 · Jun 29, 2023
References Cited (16)
US 9489040B2 · Antonyuk · 2016 [cited by examiner]
US 9530250B2 · Mazula · 2016 [cited by applicant]
US 10762702B1 · Dhua et al. · 2020 [cited by applicant]
US 20130187852A1 · Ebina · 2013 [cited by examiner]
US 20190279424A1 · Clausen et al. · 2019 [cited by applicant]
US 20200371665A1 · Clausen et al. · 2020 [cited by applicant]
US 20210104052A1 · Hayashi · 2021 [cited by examiner]
KR 20150067741A · 2015 [cited by applicant]
KR 102110123B1 · 2020 [cited by applicant]
Wang Yue et al. “A LINE-MOD-based markerless tracking approach for AR applications” The international journal of advanced manufacturing technology, Springer, London, vol. 89, No. 5, pp. 1699-1707, Aug. 1, 2016 (Year: 20… [cited by examiner]
A line-mod-based markerless tracking approach for AR applications (Year: 2016). [cited by examiner]
The extended European search report of EP 22 20 8582, Apr. 5, 2023. [cited by applicant]
Yue Wang et al, A LINE-MOD-based markerless tracking approach for AR applications, The International Journal of Advanced Manufacturing Technology, Aug. 2016, vol. 89, No. 5, pp. 1699-1707, Springer, London, England. [cited by applicant]
Wang Li et al, Fully convolutional network-based registration for augmented assembly systems, Journal of Manufacturing Systems, 2021, vol. 61, pp. 673-684, Springer, New York City, USA. [cited by applicant]
Yue Wang et al, Mechanical assembly assistance using marker-less augmented reality system, Assembly Automation, 2018, vol. 38, No. 1, pp. 77-87, Emerald Publishing Limited, Bradford, United Kingdom. [cited by applicant]
Stefan Hinterstoisser et al, Model Based Training, Detection and Pose Estimation of Texture-Less 3D Objects in Heavily Cluttered Scenes, Computer Vision ACCV, 2012, pp. 548-562, Springer, Berlin, Germany. [cited by applicant]