IP Library Granted Patent US 12,646,300
Granted Patent B2
US 12,646,300 · App. 17/815,702 · Granted Jun 2, 2026

Metamorphic labeling using aligned sensor data

Inventors: Jongmoo Choi (Gardena, CA); Mayukh Sattiraju (Redondo Beach, CA); Kilsoo Kim (Hermosa Beach, CA); David R. Arft (Torrance, CA)
Assignee: WHS ENERGY SOLUTIONS, LLC
G06V10/776G06V10/751G06V10/774
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,646,300
App. No.
17/815,702
Filed
Jul 28, 2022
Granted
Jun 2, 2026
Kind
B2
Art Unit
2666
USPC
382/113
Abstract

A method includes obtaining first and second data captured using different types of sensors. The method also includes obtaining first object detection results based on the first data and generated using a machine learning model, where the first object detection results identify one or more objects detected using the first data. The method further includes obtaining second object detection results based on the second data, where the second object detection results identify one or more objects detected using the second data. The method also includes identifying one or more inconsistencies between the first and second object detection results and generating labeled training data based on the one or more identified inconsistencies. In addition, the method includes retraining the machine learning model or training an additional machine learning model using the labeled training data.

Claims (61)

1 . A method comprising:

obtaining first data comprising camera image data and second data comprising light detection and ranging (LIDAR) range data;

obtaining first object detection results based on the first data using a first dynamically adjustable detection threshold and generated using a machine learning model, the first object detection results identifying one or more objects detected using the first data;

obtaining second object detection results based on the second data using a second static detection threshold, the second object detection results identifying one or more objects detected using the second data;

dynamically adjusting the first dynamically adjustable detection threshold using a biased detector;

identifying one or more inconsistencies between the first and second object detection results;

generating labeled training data based on the one or more identified inconsistencies, the labeled training data including one or more corrected labels, and the labeled training data being generated using the biased detector; and

retraining the machine learning model or training an additional machine learning model using the labeled training data.

2 . The method of claim 1 , wherein:

identifying the one or more inconsistencies comprises identifying one or more false positive results and one or more false negative results in the first object detection results compared to the second object detection results; and

generating the labeled training data comprises generating the labeled training data based on the one or more false positive results and the one or more false negative results.

3 . The method of claim 2 , wherein the labeled training data comprises:

the image data associated with the one or more false positive results without the first object detection results associated with the one or more false positive results being used as labels; and

the image data associated with the one or more false negative results with the one or more corrected labels replacing the first object detection results associated with the one or more false negative results.

4 . The method of claim 1 , further comprising:

identifying at least one action to be performed using the machine learning model or the additional machine learning model; and

performing the at least one action.

5 . The method of claim 4 , wherein the at least one action comprises at least one of:

an adjustment to at least one of: a steering of a vehicle, a speed of the vehicle, an acceleration of the vehicle, and a braking of the vehicle; and

an activation of an audible, visible, or haptic warning.

6 . An apparatus comprising:

at least one processing device configured to:

obtain first data comprising camera image data and second data comprising light detection and ranging (LIDAR) range data;

obtain first object detection results based on the first data using a first dynamically adjustable detection threshold and generated using a machine learning model, the first object detection results identifying one or more objects detected using the first data;

obtain second object detection results based on the second data using a second static detection threshold, the second object detection results identifying one or more objects detected using the second data;

dynamically adjusting the first dynamically adjustable detection threshold using a biased detector;

identify one or more inconsistencies between the first and second object detection results;

generate labeled training data based on the one or more identified inconsistencies, the labeled training data including one or more corrected labels, and the corrected labels being generated using the biased detector; and

retrain the machine learning model or train an additional machine learning model using the labeled training data.

7 . The apparatus of claim 6 , wherein:

to identify the one or more inconsistencies, the at least one processing device is configured to identify one or more false positive results and one or more false negative results in the first object detection results compared to the second object detection results; and

to generate the labeled training data, the at least one processing device is configured to generate the labeled training data based on the one or more false positive results and the one or more false negative results.

8 . The apparatus of claim 7 , wherein the labeled training data comprises:

the image data associated with the one or more false positive results without the first object detection results associated with the one or more false positive results being used as labels; and

the image data associated with the one or more false negative results with the one or more corrected labels replacing the first object detection results associated with the one or more false negative results.

9 . The apparatus of claim 6 , wherein the at least one processing device is further configured to:

identify at least one action to be performed using the machine learning model or the additional machine learning model; and

perform the at least one action.

10 . The apparatus of claim 9 , wherein the at least one action comprises at least one of:

an adjustment to at least one of: a steering of a vehicle, a speed of the vehicle, an acceleration of the vehicle, and a braking of the vehicle; and

an activation of an audible, visible, or haptic warning.

11 . A non-transitory machine-readable medium containing instructions that when executed cause at least one processor to:

obtain first data comprising camera image data and second data comprising light detection and ranging (LIDAR) range data;

obtain first object detection results based on the first data and generated using a machine learning model, the first object detection results identifying one or more objects detected using the first data;

obtain second object detection results based on the second data using a first dynamically adjustable detection threshold, the second object detection results identifying one or more objects detected using the second data;

dynamically adjust the first dynamically adjustable detection threshold using a biased detector;

identify one or more inconsistencies between the first and second object detection results;

generate labeled training data based on the one or more identified inconsistencies, the labeled training data including one or more corrected labels, and the labeled training data being generated using the biased detector; and

retrain the machine learning model or train an additional machine learning model using the labeled training data.

12 . The non-transitory machine-readable medium of claim 11 , wherein:

the instructions that when executed cause the at least one processor to identify the one or more inconsistencies comprise instructions that when executed cause the at least one processor to identify one or more false positive results and one or more false negative results in the first object detection results compared to the second object detection results; and

the instructions that when executed cause the at least one processor to generate the labeled training data comprise instructions that when executed cause the at least one processor to generate the labeled training data based on the one or more false positive results and the one or more false negative results.

13 . The non-transitory machine-readable medium of claim 12 , wherein the labeled training data comprises:

the image data associated with the one or more false positive results without the first object detection results associated with the one or more false positive results being used as labels; and

the image data associated with the one or more false negative results with the one or more corrected labels replacing the first object detection results associated with the one or more false negative results.

14 . The non-transitory machine-readable medium of claim 11 , further containing instructions that when executed cause the at least one processor to:

identify at least one action to be performed using the machine learning model or the additional machine learning model; and

perform the at least one action.

15 . The non-transitory machine-readable medium of claim 14 , wherein the at least one action comprises at least one of:

an adjustment to at least one of: a steering of a vehicle, a speed of the vehicle, an acceleration of the vehicle, and a braking of the vehicle; and

an activation of an audible, visible, or haptic warning.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 28, 2022
From: CHOI, JONGMOO; SATTIRAJU, MAYUKH; KIM, KILSOO; ARFT, DAVID R.
To: CANOO TECHNOLOGIES INC.
Reel/Frame 060658/0241 →