IP Library Granted Patent US 11,087,491
Granted Patent B2
US 11,087,491 · App. 16/560,088 · Granted Aug 10, 2021

Method for determining a coordinate of a feature point of an object in a 3D space

Inventor: Krzysztof Kaczor (Zakopane, PL)
Assignee: Aptiv Technologies Limited
G06T7/74B60K35/00G06F3/017G06K9/00355G06K9/00845B60K2370/146
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,087,491
App. No.
16/560,088
Filed
Sep 4, 2019
Granted
Aug 10, 2021
Kind
B2
Art Unit
2666
USPC
382/104
Abstract

A method for determining a coordinate of a feature point of an object in a 3D space comprises: arranging at least one dedicated pointer in the 3D space in a pre-determined relation to the feature point of the object in the 3D space, wherein each dedicated pointer has at least one visible feature in line of sight of the camera; capturing at least one image by using the camera; performing image feature detection on the at least one captured image to determining a coordinate of the respective at least one visible feature of each dedicated pointer; and determining the coordinate of the feature point of the object in the 3D space based on the determined coordinate of the respective at least one visible feature of each dedicated pointer.

Claims (71)

1. A method for determining a coordinate of a feature point of an object in a 3D space, comprising:

arranging, with a computer, at least one dedicated pointer in a 3D space in a pre-determined relation to the feature point, wherein each dedicated pointer has at least one visible feature in line of sight of a camera;

capturing at least one image by using the camera;

performing image feature detection on the at least one captured image, with the computer, to determine the coordinate of the respective at least one visible feature of each dedicated pointer; and

determining, with the computer, the coordinate of the feature point based on the determined coordinate of the respective at least one visible feature of each dedicated pointer, wherein:

the at least one dedicated pointer comprises a first dedicated pointer and a second dedicated pointer;

the at least one visible feature in line of sight of the camera of the first dedicated pointer comprises a first reference point and a first direction vector;

the at least one visible feature in line of sight of the camera of the second dedicated pointer comprises a second reference point and a second direction vector; and

determining the coordinate of the respective at least one visible feature of each dedicated pointer comprises determining a position of the first reference point, a direction of the first direction vector, the position of the second reference point, and the direction of the second direction vector.

2. The method of claim 1 ,

wherein the at least one dedicated pointer has at least two visible points in line of sight of the camera and one invisible point not in line of sight of the camera; wherein the at least one visible feature comprises the at least two visible points; and

wherein arranging the at least one dedicated pointer in the 3D space in the pre-determined relation to the feature point of the object in the 3D space comprises arranging an invisible point of the at least one dedicated pointer on or close to the feature point of the object in the 3D space.

3. The method of claim 1 ,

wherein arranging the at least one dedicated pointer in the 3D space in the pre-determined relation to the feature point comprises arranging the first dedicated pointer in the 3D space so that the direction from the first reference point to the feature point is represented by the first direction vector and arranging the second dedicated pointer in the 3D space so that the direction from the second reference point to the feature point is represented by the second direction vector.

4. The method of claim 1 ,

wherein the coordinate of the feature point is determined based on the coordinate of a point where a distance between a first line through the determined position of the first reference point and having the direction corresponding to the determined direction of the first direction vector and a second line through the determined position of the second reference point and having the direction corresponding to the determined direction of the second direction vector comprises a minimum.

5. The method claim 1 ,

wherein the at least one dedicated pointer comprises a dedicated pointer having a plurality of visible features in line of sight of the camera, wherein a plurality of visible features comprises a plurality of points arranged on a reference circle; and

wherein determining the coordinate of the respective at least one visible feature of each dedicated pointer further comprises determining the respective positions of the plurality of points arranged on the reference circle.

6. The method of claim 5 ,

wherein arranging the at least one dedicated pointer in the 3D space in the pre-determined relation to the feature point comprises arranging the dedicated pointer in the 3D space so that a center of the reference circle coincides with the feature point.

7. The method of claim 5 , further comprising:

determining the coordinate of the feature point based on an observed circle that is determined based on a plurality of determined points arranged on the reference circle.

8. The method of claim 1 ,

wherein said 3D space corresponds to an interior space of a vehicle.

9. The method of claim 1 ,

wherein the feature point is at least one of a feature point of a car window, a feature point of a car mirror, and a feature point of an exclusion box, exclusion surface or pointing zone used in a 3D gesture recognition system.

10. The method of claim 9 ,

wherein the feature point of the car window is a corner of the car window, and the feature point of the car mirror is a corner of an exterior rear-view mirror.

11. The method of claim 1 ,

wherein the arranging, the capturing, and performing the image feature detection are carried out iteratively.

12. The method of claim 1 , wherein the at least one designated pointer corresponds to at least one physical item arranged in the 3D space including, an elongated physical item having an end pointing to the feature point of the object in the 3D space, the elongated physical item comprising at least one of plastic or paper.

13. A system for determining a coordinate of a feature point of an object in a 3D space, comprising:

a camera configured to capture at least one image of a visible feature; and

a computer configured to arrange at least one dedicated pointer in a 3D space in a pre-determined relation to a feature point;

wherein each dedicated pointer has at least one visible feature in a line of sight of the camera; wherein

the computer is further configured to:

perform image feature detection on the at least one captured image to determine a coordinate of the respective at least one visible feature of each dedicated pointer; and

determine the coordinate of the feature point based on the determined coordinate of the respective at least one visible feature of each dedicated pointer, wherein:

the at least one dedicated pointer comprises a first dedicated pointer and a second dedicated pointer;

the at least one visible feature in the line of sight of the camera of the first dedicated pointer comprises a first reference point and a first direction vector;

the at least one visible feature in the line of sight of the camera of the second dedicated pointer comprises a second reference point and a second direction vector; and

the computer is configured to determine the coordinate of the respective at least one visible feature of each dedicated pointer by determining a position of the first reference point, a direction of the first direction vector, the position of the second reference point, and the direction of the second direction vector.

14. The system of claim 13 ,

wherein the at least one dedicated pointer has at least two visible points in line of sight of the camera and one invisible point not in line of sight of the camera;

wherein the at least one visible feature comprises the at least two visible points; and

wherein arranging the at least one dedicated pointer in the 3D space in the pre-determined relation to the feature point of the object in the 3D space comprises arranging an invisible point of the at least one dedicated pointer on or close to the feature point of the object in the 3D space.

15. The system of claim 13 , wherein:

the computer is configured to arrange the at least one dedicated pointer in the 3D space in the pre-determined relation to the feature point by arranging the first dedicated pointer in the 3D space so that the direction from the first reference point to the feature point is represented by the first direction vector and arranging the second dedicated pointer in the 3D space so that the direction from the second reference point to the feature point is represented by the second direction vector.

16. The system of claim 13 , wherein:

the coordinate of the feature point is determined based on the coordinate of a point where a distance between a first line through the determined position of the first reference point and having the direction corresponding to the determined direction of the first direction vector and a second line through the determined position of the second reference point and having the direction corresponding to the determined direction of the second direction vector comprises a minimum.

17. The system claim 13 ,

wherein the at least one dedicated pointer comprises a dedicated pointer having a plurality of visible features in line of sight of the camera, wherein a plurality of visible features comprises a plurality of points arranged on a reference circle; and wherein determining the coordinate of the respective at least one visible feature of each dedicated pointer further comprises determining the respective positions of the plurality of points arranged on the reference circle.

18. The system of claim 17 ,

wherein computer is configured to arrange the at least one dedicated pointer in the 3D space in the pre-determined relation to the feature point comprises arranging the dedicated pointer in the 3D space so that a center of the reference circle coincides with the feature point.

19. The system of claim 17 , further comprising:

determining the coordinate of the feature point based on an observed circle that is determined based on a plurality of determined points arranged on the reference circle.

20. The system of claim 13 ,

wherein said 3D space corresponds to an interior space of a vehicle.

21. The system of claim 13 ,

wherein the feature point is at least one of a feature point of a car window, a feature point of a car mirror, and a feature point of an exclusion box, exclusion surface or pointing zone used in a 3D gesture recognition system.

22. The system of claim 21 ,

wherein the feature point of the car window is a corner of the car window, and the feature point of the car mirror is a corner of an exterior rear-view mirror.

23. The system of claim 13 ,

wherein the computer is configured to arrange, capture, and perform the image feature detection iteratively.

24. The system of claim 13 , wherein the at least one designated pointer corresponds to at least one physical item arranged in the 3D space including an elongated physical item having an end pointing to the feature point of the object in the 3D space, the elongated physical item comprising at least one of plastic or paper.

25. The method claim 2 ,

wherein the at least one dedicated pointer comprises a dedicated pointer having a plurality of visible features in line of sight of the camera, wherein a plurality of visible features comprises a plurality of points arranged on a reference circle; and

wherein determining the coordinate of the respective at least one visible feature of each dedicated pointer further comprises determining the respective positions of the plurality of points arranged on the reference circle.

26. The system claim 14 ,

wherein the at least one dedicated pointer comprises a dedicated pointer having a plurality of visible features in line of sight of the camera, wherein a plurality of visible features comprises a plurality of points arranged on a reference circle; and wherein determining the coordinate of the respective at least one visible feature of each dedicated pointer comprises determining the respective positions of the plurality of points arranged on the reference circle.

Assignments (4)
MERGER Recorded Feb 11, 2024
From: APTIV TECHNOLOGIES (2) S.À R.L.
To: APTIV MANUFACTURING MANAGEMENT SERVICES S.À R.L.
Reel/Frame 066566/0173 →
ENTITY CONVERSION Recorded Feb 11, 2024
From: APTIV TECHNOLOGIES LIMITED
To: APTIV TECHNOLOGIES (2) S.À R.L.
Reel/Frame 066746/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 11, 2024
From: APTIV MANUFACTURING MANAGEMENT SERVICES S.À R.L.
To: APTIV TECHNOLOGIES AG
Reel/Frame 066551/0219 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 25, 2019
From: KACZOR, KRZYSZTOF
To: APTIV TECHNOLOGIES LIMITED
Reel/Frame 051102/0649 →
Priority Claims (2)
EP 18194116 · Sep 12, 2018 · regional
EP 19191839 · Aug 14, 2019 · regional
Continuity (1)
Related Publication 20200082564A1 · Mar 12, 2020