IP Library Granted Patent US 12,417,549
Granted Patent B2
US 12,417,549 · App. 17/819,584 · Granted Sep 16, 2025

Vanishing point determination, symmetry-based boundary refinement, and component detection for vehicle object detection or other applications

Inventors: Dae Jin Kim (Rancho Palos Verdes, CA); David R. Arft (Torrance, CA); Jongmoo Choi (Gardena, CA)
Assignee: CANOO TECHNOLOGIES INC.
G06T7/536B60W10/18B60W10/20B60W50/14B60W50/16G06T7/68G06T7/73G06V20/56G06V20/58G06V20/625B60W2050/0004B60W2050/143B60W2050/146B60W60/001B60W2420/403B60W2552/50B60W2554/20B60W2710/18B60W2710/20B60W2720/10B60W2720/106G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,417,549
App. No.
17/819,584
Granted
Sep 16, 2025
Kind
B2
Abstract

A method includes obtaining, using at least one processing device, a vanishing point and a boundary based on image data associated with a scene, where the boundary is associated with a detected object within the scene. The method also includes repeatedly, during multiple iterations and using the at least one processing device, (i) identifying multiple patches within the boundary and (ii) determining a similarity of the image data contained within the multiple patches. The method further includes identifying, using the at least one processing device, a modification to be applied to the boundary based on the identified patches and the determined similarities. In addition, the method includes generating, using the at least one processing device, a refined boundary based on the modification, where the refined boundary identifies a specified portion of the detected object.

Claims (89)

1. A method comprising:

obtaining, using at least one processing device, a vanishing point and a boundary based on image data associated with a scene, the boundary associated with a detected object within the scene;

repeatedly, during multiple iterations and using the at least one processing device, (i) identifying multiple patches within the boundary and (ii) determining a similarity of the image data contained within the multiple patches;

identifying, using the at least one processing device, a modification to be applied to the boundary based on the identified patches and the determined similarities; and

generating, using the at least one processing device, a refined boundary based on the modification, the refined boundary identifying a specified portion of the detected object,

wherein, during each iteration, the multiple patches within the boundary include a first patch and a mirrored second patch within the boundary.

2. The method of claim 1 , further comprising:

identifying virtual left and right ego lines using the image data, each ego line passing through the vanishing point;

defining a triangular region based on the virtual left and right ego lines; and

selecting the boundary as having a specified point outside the triangular region.

3. The method of claim 1 , further comprising:

identifying an initial boundary ratio; and

updating the initial boundary ratio to generate an updated boundary ratio based on the modification,

wherein the refined boundary is generated based on the updated boundary ratio.

4. The method of claim 1 , wherein:

each iteration generates estimated template coordinates based on the identified patches within the boundary; and

identifying the modification comprises generating a weighted combination of the estimated template coordinates while using the determined similarities as weights for the estimated template coordinates, the modification representing the weighted combination.

5. The method of claim 1 , further comprising:

determining a position of the vanishing point based on multiple collections of line segments identified based on the image data.

6. The method of claim 1 , wherein:

the detected object comprises a vehicle; and

the specified portion of the detected object comprises a rear portion of the vehicle.

7. The method of claim 1 , further comprising:

identifying one or more components of the detected object based on the refined boundary.

8. The method of claim 1 , further comprising:

identifying at least one action to be performed based on the refined boundary; and

performing the at least one action.

9. The method of claim 8 , wherein the at least one action comprises at least one of:

an adjustment to at least one of: a steering of a vehicle, a speed of the vehicle, an acceleration of the vehicle, and a braking of the vehicle; and

an activation of an audible, visible, or haptic warning.

10. An apparatus comprising:

at least one processing device configured to:

obtain a vanishing point and a boundary based on image data associated with a scene, the boundary associated with a detected object within the scene;

repeatedly, during multiple iterations, (i) identify multiple patches within the boundary and (ii) determine a similarity of the image data contained within the multiple patches;

identify a modification to be applied to the boundary based on the identified patches and the determined similarities; and

generate a refined boundary based on the modification, the refined boundary identifying a specified portion of the detected object,

wherein, during each iteration, the multiple patches within the boundary include a first patch and a mirrored second patch within the boundary.

11. The apparatus of claim 10 , wherein the at least one processing device is further configured to:

identify virtual left and right ego lines using the image data, each ego line passing through the vanishing point;

define a triangular region based on the virtual left and right ego lines; and

select the boundary as having a specified point outside the triangular region.

12. The apparatus of claim 10 , wherein:

the at least one processing device is further configured to:

identify an initial boundary ratio; and

update the initial boundary ratio to generate an updated boundary ratio based on the modification; and

the at least one processing device is configured to generate the refined boundary based on the updated boundary ratio.

13. The apparatus of claim 10 , wherein:

the at least one processing device is configured, during each iteration, to generate estimated template coordinates based on the identified patches within the boundary; and

to identify the modification, the at least one processing device is configured to generate a weighted combination of the estimated template coordinates while using the determined similarities as weights for the estimated template coordinates, the modification representing the weighted combination.

14. The apparatus of claim 10 , wherein the at least one processing device is further configured to determine a position of the vanishing point based on multiple collections of line segments identified based on the image data.

15. The apparatus of claim 10 , wherein:

the detected object comprises a vehicle; and

the specified portion of the detected object comprises a rear portion of the vehicle.

16. The apparatus of claim 10 , wherein the at least one processing device is further configured to identify one or more components of the detected object based on the refined boundary.

17. The apparatus of claim 10 , wherein the at least one processing device is further configured to:

identify at least one action to be performed based on the refined boundary; and

perform the at least one action.

18. The apparatus of claim 17 , wherein the at least one action comprises at least one of:

an adjustment to at least one of: a steering of a vehicle, a speed of the vehicle, an acceleration of the vehicle, and a braking of the vehicle; and

an activation of an audible, visible, or haptic warning.

19. A non-transitory machine-readable medium containing instructions that when executed cause at least one processor to:

obtain a vanishing point and a boundary based on image data associated with a scene, the boundary associated with a detected object within the scene;

repeatedly, during multiple iterations, (i) identify multiple patches within the boundary and (ii) determine a similarity of the image data contained within the multiple patches;

identify a modification to be applied to the boundary based on the identified patches and the determined similarities; and

generate a refined boundary based on the modification, the refined boundary identifying a specified portion of the detected object,

wherein, during each iteration, the multiple patches within the boundary include a first patch and a mirrored second patch within the boundary.

20. The non-transitory machine-readable medium of claim 19 , further containing instructions that when executed cause the at least one processor to:

identify virtual left and right ego lines using the image data, each ego line passing through the vanishing point;

define a triangular region based on the virtual left and right ego lines; and

select the boundary as having a specified point outside the triangular region.

21. The non-transitory machine-readable medium of claim 19 , wherein:

the medium further contains instructions that when executed cause the at least one processor to:

identify an initial boundary ratio; and

update the initial boundary ratio to generate an updated boundary ratio based on the modification; and

the instructions that when executed cause the at least one processor to generate the refined boundary comprise instructions that when executed cause the at least one processor to generate the refined boundary based on the updated boundary ratio.

22. The non-transitory machine-readable medium of claim 19 , wherein:

the instructions when executed cause the at least one processor, during each iteration, to generate estimated template coordinates based on the identified patches within the boundary; and

the instructions that when executed cause the at least one processor to identify the modification comprise instructions that when executed cause the at least one processor to generate a weighted combination of the estimated template coordinates while using the determined similarities as weights for the estimated template coordinates, the modification representing the weighted combination.

23. The non-transitory machine-readable medium of claim 19 , further containing instructions that when executed cause the at least one processor to determine a position of the vanishing point based on multiple collections of line segments identifying based on the image data.

24. The non-transitory machine-readable medium of claim 19 , wherein:

the detected object comprises a vehicle; and

the specified portion of the detected object comprises a rear portion of the vehicle.

25. The non-transitory machine-readable medium of claim 19 , further containing instructions that when executed cause the at least one processor to identify one or more components of the detected object based on the refined boundary.

26. The non-transitory machine-readable medium of claim 19 , further containing instructions that when executed cause the at least one processor to:

identify at least one action to be performed based on the refined boundary; and

perform the at least one action.

27. The non-transitory machine-readable medium of claim 26 , wherein the at least one action comprises at least one of:

an adjustment to at least one of: a steering of a vehicle, a speed of the vehicle, an acceleration of the vehicle, and a braking of the vehicle; and

an activation of an audible, visible, or haptic warning.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 30, 2026
From: CANOO TECHNOLOGIES INC.
To: WHS ENERGY SOLUTIONS, LLC
Reel/Frame 075311/0490 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 12, 2022
From: KIM, DAE JIN; ARFT, DAVID R.; CHOI, JONGMOO
To: CANOO TECHNOLOGIES INC.
Reel/Frame 060799/0946 →