IP Library Granted Patent US 12,651,443
Granted Patent B2
US 12,651,443 · App. 18/369,403 · Granted Jun 9, 2026

Method and device for validating annotations of objects

Inventors: Urs Zimmermann (Krefeld, DE); Dennis Müller (Moers, DE)
Assignee: Aptiv Technologies AG
G06V10/776G01S7/4802G01S17/42G01S17/58G01S17/931G06T7/246G06T7/73G06V10/764G06V10/98G06V20/70G06T2207/10028
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,651,443
App. No.
18/369,403
Granted
Jun 9, 2026
Kind
B2
Abstract

A method is provided for validating annotations of objects. Spatial datapoints acquired by a sensor and annotation data are received. The annotation data is associated with the acquired spatial data points and includes an identification of each respective object. Via a processing unit, the annotations of the objects are validated by performing the steps of: determining a target range for at least one property of the objects, determining, from the acquired spatial datapoints and/or from the annotation data, a respective value of the at least one property for each respective object, and for each object, identifying the object as an erroneous object if the respective value of the at least one property is outside the target range for the at least one property. The erroneous object is selected for review regarding erroneous annotation.

Claims (65)

1 . A computer implemented method for validating annotations of objects, the method comprising:

receiving a plurality of spatial datapoints acquired by a sensor, wherein the spatial datapoints are related to an external environment of the sensor,

receiving annotation data of objects associated with the acquired spatial datapoints, the annotation data including an identification of each respective object, and

validating, via a processing unit, the annotations of the objects by performing the steps of:

determining a target range for at least one property of the objects,

determining, from the acquired spatial datapoints and/or from the annotation data, a respective value of the at least one property for each respective object, and

for each object, identifying the object as an erroneous object if the respective value of the at least one property is outside the target range for the at least one property, the erroneous object being selected for review regarding erroneous annotation;

wherein:

the identification of each respective object includes a classification and a predefined geometrical shape, the predefined geometrical shape being associated with a subset of the acquired spatial datapoints for each respective object;

the target range is determined by performing the steps of:

selecting a portion of the spatial datapoints which include a respective subset of the spatial datapoints for each of a plurality of sample objects,

determining a respective value of the at least one property for each sample object based on the predefined geometrical shape and/or the respective subset of the spatial datapoints,

estimating at least one probability distribution for the property based on a statistical distribution for the values of the property for the sample objects,

deriving the target range for the at least one property of the respective objects from the at least one probability distribution;

a probability value is determined for the value of the at least one property based on at least one probability distribution,

the respective value of the at least one property is outside the target range if the probability value is smaller than a predetermined threshold;

a percentage share of a total number of objects is predefined for the review regarding erroneous annotation; and

one respective object having a lowest probability value is iteratively selected for the review until the number of selected objects is equal to the predefined percentage share of the total number of objects.

2 . The method according to claim 1 , wherein

the predefined geometrical shape is a cuboid.

3 . The method according to claim 1 , wherein

the classification of the respective object comprises that the respective object is associated with one of a plurality of object classes, and

estimating the at least one probability distribution for the at least one property comprises estimating a separate probability distribution for each of the plurality of object classes.

4 . The method according to claim 3 , wherein

each probability distribution is based on a Gaussian mixture model.

5 . The method according to claim 4 , wherein

the Gaussian mixture model includes a plurality of Gaussians distributions, and

for each Gaussian distribution:

a center is determined based on a median of the determined values of the at least one property for the respective object class, and

a standard deviation is determined based on a median of an absolute deviation for the determined values of the at least one property for the respective object class.

6 . The method according to claim 4 , wherein:

the Gaussian mixture model includes a plurality of Gaussians distributions; and

for each Gaussian distribution:

a center is determined based on a median of the determined values of the at least one property for the respective object class, and

a standard deviation is determined based on a median of an absolute deviation for the determined values of the at least one property for the respective object class.

7 . The method according to claim 1 , wherein

the at least one property is derived from a spatial distribution of the datapoints of the respective subset with respect to the predefined geometrical shape.

8 . The method according to claim 1 , wherein

the at least one property includes at least one statistical property of the datapoints of the respective subset.

9 . The method according to claim 1 , wherein

the at least one property of the objects includes parameters of a spatial location of the objects.

10 . The method according to claim 1 , wherein

the plurality of spatial datapoints is based on a sequence of Lidar scans for a predetermined time period, and

the at least one property includes a respective velocity of the objects with respect to the sensor, wherein the respective velocity is determined based on the sequence of Lidar scans.

11 . The method according to claim 1 , wherein

for each object being selected for the review regarding erroneous annotation, a potential annotation error is indicated.

12 . A computer system, configured to carry out the computer implemented method of claim 1 .

13 . A non-transitory computer readable medium comprising instructions for carrying out the computer implemented method of claim 1 .

14 . A computer implemented method for validating annotations of objects, the method comprising:

receiving a plurality of spatial datapoints acquired by a sensor, wherein the spatial datapoints are related to an external environment of the sensor,

receiving annotation data of objects associated with the acquired spatial datapoints, the annotation data including an identification of each respective object, and

validating, via a processing unit, the annotations of the objects by performing the steps of:

determining a target range for at least one property of the objects,

determining, from the acquired spatial datapoints and/or from the annotation data, a respective value of the at least one property for each respective object, and

for each object, identifying the object as an erroneous object if the respective value of the at least one property is outside the target range for the at least one property, the erroneous object being selected for review regarding erroneous annotation;

wherein:

the identification of each respective object includes a classification and a predefined geometrical shape, the predefined geometrical shape being associated with a subset of the acquired spatial datapoints for each respective object;

the target range is determined by performing the steps of:

selecting a portion of the spatial datapoints which include a respective subset of the spatial datapoints for each of a plurality of sample objects,

determining a respective value of the at least one property for each sample object based on the predefined geometrical shape and/or the respective subset of the spatial datapoints,

estimating at least one probability distribution for the property based on a statistical distribution for the values of the property for the sample objects, and

deriving the target range for the at least one property of the respective objects from the at least one probability distribution;

the classification of the respective object comprises that the respective object is associated with one of a plurality of object classes; and

estimating the at least one probability distribution for the at least one property comprises estimating a separate probability distribution for each of the plurality of object classes.

15 . The method according to claim 14 , wherein each probability distribution is based on a Gaussian mixture model.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 11, 2024
From: APTIV MANUFACTURING MANAGEMENT SERVICES S.À R.L.
To: APTIV TECHNOLOGIES AG
Reel/Frame 066551/0219 →
MERGER Recorded Feb 11, 2024
From: APTIV TECHNOLOGIES (2) S.À R.L.
To: APTIV MANUFACTURING MANAGEMENT SERVICES S.À R.L.
Reel/Frame 066566/0173 →
ENTITY CONVERSION Recorded Feb 11, 2024
From: APTIV TECHNOLOGIES LIMITED
To: APTIV TECHNOLOGIES (2) S.À R.L.
Reel/Frame 066746/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 18, 2023
From: ZIMMERMANN, URS; MÜLLER, DENNIS
To: APTIV TECHNOLOGIES LIMITED
Reel/Frame 064935/0924 →
Priority Claims (1)
EP 21163747 · Mar 19, 2021 · regional
Continuity (2)
Continuation PCTEP2022057210 · Mar 18, 2022
Related Publication 20240078794A1 · Mar 7, 2024
References Cited (10)
US 20180136332A1 · Barfield, Jr. et al. · 2018 [cited by applicant]
US 20220188554A1 · Huang · 2022 [cited by examiner]
EP 3772704A1 · 2021 [cited by applicant]
Iterative Bounding Box, Adhikari et al 2020 (Year: 2020). [cited by examiner]
European Search Report regarding European Patent Application No. 21163747.5-1207, dated Sep. 22, 2021. [cited by applicant]
International Search Report and Written Opinion of the International Searching Authority regarding International Patent Application No. PCT/EP2022/057210, dated Jul. 12, 2022. [cited by applicant]
“An Efficient Approach for Outlier Detection with Imperfect Data Labels,” B. Liu, et al., IEEE Transactions on Knowledge and Data Engineering, vol. 26, No. 7, Jul. 2014. [cited by applicant]
“Crowdsourced Interactive Computer Vision,” M.E. Schwarzer-Becker, Dissertation submitted to the Combined Faculties for the Natural Sciences and for Mathematics of the Ruperto-Carola University of Heidelberg, Germany fo… [cited by applicant]
“Clickstream analysis for crowd-based object segmentation with confidence,” E. Heim et al., IEEE Transactions on Pattern Analysis and Machine Intelligence, DOI: 10.1109/TPAMI.2017.2777967, dated Nov. 25, 2016. [cited by applicant]
International Preliminary Report on Patentability regarding International Patent Application No. PCT/EP2022/057210 dated Sep. 28, 2023. [cited by applicant]