Granted Patent B2

US 12,469,170 · App. 17/967,534 · Granted Nov 11, 2025

Inventor: Hyun Kyu Lim (Seoul, KR)

Assignees: HYUNDAI MOTOR COMPANY; KIA CORPORATION

G06T7/75G06T7/10G06T2207/20084

Abstract

Claims (55)

1 . An apparatus for detecting a three-dimensional (3D) object, the apparatus comprising:

a camera;

storage; and

a processor configured to

obtain a two-dimensional (2D) image using the camera,

segment the 2D image to obtain feature information associated with a free space,

obtain feature information associated with an object included in the 2D image from the 2D image,

determine an attention score based on the feature information associated with the free space and the feature information associated with the object, using an attention mechanism, wherein the processor is configured to

compare the feature information associated with the free space with previously stored segmentation reference information to determine a segmentation loss for the feature information associated with the free space, and

extract a key from the feature information associated with the free space with regard to the segmentation loss, and

detect 3D location information of the object based on the attention score.

2 . The apparatus of claim 1 , wherein the processor is configured to:

segment the 2D image into a plurality of regions;

perform prediction of the free space with respect to the plurality of regions; and

obtain the feature information associated with the free space based on a result of performing the prediction.

3 . The apparatus of claim 1 , wherein the processor is configured to:

extract a query used in the attention mechanism from the feature information associated with the object; and

determine the attention score by multiplying the query by the key.

4 . The apparatus of claim 3 , wherein the processor is configured to:

determine a value used in the attention mechanism to be the same as the query;

apply the value as a weight to the attention score to determine an attention value; and

decode the attention value to detect the 3D location information.

5 . The apparatus of claim 4 , wherein the 3D location information includes 3D coordinate information of the object.

6 . The apparatus of claim 4 , wherein the processor is configured to compare the 3D location information with previously stored 3D location reference information to determine a loss of the 3D location information.

7 . The apparatus of claim 1 , wherein the processor includes a backbone network configured to extract the feature information associated with the object from the 2D image based on a hierarchical structure of a convolutional neural network (CNN).

8 . The apparatus of claim 1 , wherein the processor is configured to detect 2D and 3D dimensions of the object and a 3D orientation of the object from the 2D image, based on the feature information associated with the object.

9 . The apparatus of claim 1 , wherein the camera is a monocular camera.

10 . A method for detecting a three-dimensional (3D) object, the method comprising:

obtaining a 2D image using a camera;

segmenting the 2D image to obtain feature information associated with a free space;

obtaining feature information associated with an object included in the 2D image from the 2D image;

determining an attention score based on the feature information associated with the free space and the feature information associated with the object, using an attention mechanism; and

detecting 3D location information of the object based on the attention score,

wherein determining the attention score includes extracting a key used in the attention mechanism from the feature information associated with the free space, and

wherein obtaining the key includes

comparing the feature information associated with the free space with previously stored segmentation reference information to determine a segmentation loss for the feature information associated with the free space, and

extracting the key from the feature information associated with the free space with regard to the segmentation loss.

11 . The method of claim 10 , wherein obtaining the feature information associated with the free space includes:

segmenting the 2D image into a plurality of regions,

performing prediction of the free space with respect to the plurality of regions, and

obtaining the feature information associated with the free space based on a result of performing the prediction.

12 . The method of claim 10 , wherein determining of the attention score includes:

extracting a query used in the attention mechanism from the feature information associated with the object; and

determining the attention score by multiplying the query by the key.

13 . The method of claim 12 , further comprising:

determining a value used in the attention mechanism to be the same as the query; and

applying the value as a weight to the attention score to determine an attention value, and

wherein detecting of the 3D location information includes decoding the attention value to detect the 3D location information.

14 . The method of claim 13 , wherein the 3D location information includes 3D coordinate information of the object.

15 . The method of claim 13 , further comprising comparing the 3D location information with previously stored 3D location reference information to determine a loss of the 3D location information.

16 . The method of claim 10 , wherein obtaining the feature information associated with the object includes extracting the feature information associated with the object from the 2D image using a backbone network based on a hierarchical structure of a convolutional neural network (CNN).

17 . The method of claim 10 , further comprising:

detecting 2D and 3D dimensions of the object from the 2D image, based on the feature information associated with the object; and

detecting an orientation of the object from the 2D image, based on the feature information associated with the object.

18 . The method of claim 10 , wherein the 2D image is a monocular image obtained using a monocular camera.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (21)