IP Library › Granted Patent US 12,596,939
Granted Patent B2
US 12,596,939 · App. 18/356,702 · Granted Apr 7, 2026

Automatic labeling of data by an entity gathering the data

Inventors: Harsh Bhupendra Bhate (Dearborn, MI); Vyacheslav Zavadsky (Ottawa, CA); Ziwei Zeng (Redmond, WA); Senthil Kumar Natarajan (Chennai, IN); Uttara Thakre (Dearborn, MI); Panduranga Chary Kondoju (Canton, MI); Aishwarya Vaibhav Kadam (Canton, MI)
G06N5/04G06N3/08G06N5/046G06N20/00H04L67/12G05B13/027G05B13/04G06N3/098G06N20/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,596,939
App. No.
18/356,702
Granted
Apr 7, 2026
Kind
B2
Abstract

A system receives identification of data to be gathered, via a request configured based on a configuration file associated with a machine learning model stored by a vehicle. The system receives identification of how the data is to be labeled, defined by the configuration file and create one or more topics for publication of the data, onboard the vehicle, the publication including both the gathered data and any meta-data usable to label the data in accordance with the definitions in the configuration file. Also, the system subscribes to the topics to receive the published data and the metadata and appends labels to the data, using the metadata, to label the data in accordance with the definitions for labeling in the configuration file. The system saves the labeled data in vehicle memory as data associated with the model.

Claims (45)

1 . A system comprising:

one or more processors of a vehicle configured to:

receive first identification of data to be gathered, via a request configured based on a configuration file associated with a machine learning model stored by the vehicle;

receive second identification of how the data is to be labeled, based on at least metadata, defined by the configuration file;

gather the data identified by the first identification;

create one or more topics for publication of the gathered data, onboard the vehicle;

publish, to the one or more topics, both the gathered data and the metadata usable to label the gathered data in accordance with second identification defined by the configuration file, as published data;

subscribe to the topics to receive the published data and the metadata;

append labels to the data, using the metadata, to label the data in accordance with the second identification of how the data is to be labeled defined by the configuration file; and

save the labeled data in vehicle memory as data associated with the model.

2 . The system of claim 1 , wherein the metadata includes vehicle sensor data published to the topics in conjunction with the data gathered pursuant to the first identification and designated, by the configuration file, to be labeled based on one or more sensor values present at the time the data was gathered.

3 . The system of claim 1 , wherein the request includes timing for gathering of the data, the timing defined by the configuration file.

4 . The system of claim 1 , wherein at least one of the one or more processors is further configured to remove data from the gathered data that does not correspond to a region of interest in the gathered data, the region of interest defined based on the configuration file.

5 . The system of claim 1 , wherein the metadata includes identification of a positive or negative response by a human occupant, responsive to a vehicle state change action generated based on the model and the positive or negative response indicating whether the vehicle state change was accepted or modified, respectively.

6 . The system of claim 1 , wherein the metadata includes a response to a query asked directly, via a vehicle display, of a vehicle occupant, the query defined by the configuration file.

7 . The system claim 1 , wherein the configuration file defines an algorithm usable to modify the gathered data to produce a modified version of the gathered data and to determine a characteristic of the gathered data, used for labeling the gathered data, based on the modified version of the gathered data.

8 . A method comprising:

receiving first identification of data to be gathered, via a request configured based on a configuration file associated with a machine learning model stored by a vehicle;

receiving second identification of how the data is to be labeled, based on at least metadata defined by the configuration file;

gathering the data identified by the first identification;

creating one or more topics for publication of the gathered data, onboard the vehicle;

publishing, to the one or more topics, both the gathered data and the metadata usable to label the gathered data in accordance with the second identification defined by the configuration file, as published data;

subscribing to the topics to receive the published data and the metadata;

appending labels to the data, using the metadata, to label the data in accordance with the second identification of how the data is to be labeled defined by the configuration file; and

saving the labeled data in vehicle memory as data associated with the model.

9 . The method of claim 8 , wherein the metadata includes vehicle sensor data published to the topics in conjunction with the data gathered pursuant to the first identification and designated, by the configuration file, to be labeled based on one or more sensor values present at the time the data was gathered.

10 . The method of claim 8 , wherein the request includes timing for gathering of the data, the timing defined by the configuration file.

11 . The method of claim 8 , further comprising removing data from the gathered data that does not correspond to a region of interest in the gathered data, the region of interest defined based on the configuration file.

12 . The method of claim 8 , wherein the metadata includes identification of a positive or negative response by a human occupant, responsive to a vehicle state change action generated based on the model and the positive or negative response indicating whether the vehicle state change was accepted or modified, respectively.

13 . The method of claim 8 , wherein the metadata includes a response to a query asked directly, via a vehicle display, of a vehicle occupant, the query defined by the configuration file.

14 . The method of claim 8 , wherein the configuration file defines an algorithm usable to modify the gathered data to produce a modified version of the gathered data and to determine a characteristic of the gathered data, used for labeling the gathered data, based on the modified version of the gathered data.

15 . A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform a method comprising:

receiving first identification of data to be gathered, via a request configured based on a configuration file associated with a machine learning model stored by a vehicle;

receiving second identification of how the data is to be labeled, based on at least metadata defined by the configuration file;

gathering the data identified by the first identification;

creating one or more topics for publication of the gathered data, onboard the vehicle;

publishing, to the one or more topics, both the gathered data and the metadata usable to label the gathered data in accordance with the second identification defined by the configuration file, as published data;

subscribing to the topics to receive the published data and the metadata;

appending labels to the data, using the metadata, to label the data in accordance with the second identification of how the data is to be labeled defined by the configuration file; and

saving the labeled data in vehicle memory as data associated with the model.

16 . The storage medium of claim 15 , wherein the metadata includes vehicle sensor data published to the topics in conjunction with the data gathered pursuant to the first identification and designated, by the configuration file, to be labeled based on one or more sensor values present at the time the data was gathered.

17 . The storage medium of claim 15 , wherein the request includes timing for gathering of the data, the timing defined by the configuration file.

18 . The storage medium of claim 15 , further comprising removing data from the gathered data that does not correspond to a region of interest in the gathered data, the region of interest defined based on the configuration file.

19 . The storage medium of claim 15 , wherein the metadata includes at least one of identification of a positive or negative response by a human occupant, responsive to a vehicle state change action generated based on the model and the positive or negative response indicating whether the vehicle state change was accepted or modified, respectively or a response to a query asked directly, via a vehicle display, of a vehicle occupant, the query defined by the configuration file.

20 . The storage medium of claim 15 , wherein the configuration file defines an algorithm usable to modify the gathered data to produce a modified version of the gathered data and to determine a characteristic of the gathered data, used for labeling the gathered data, based on the modified version of the gathered data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 21, 2025
From: BHATE, HARSH BHUPENDRA; ZAVADSKY, VYACHESLAV; ZENG, ZIWEI; NATARAJAN, SENTHIL KUMAR; THAKRE, UTTARA; JONDOJU, PANDURANGA CHARY; KADAM, AISHWARYA
To: FORD GLOBAL TECHNOLOGIES, LLC
Reel/Frame 073000/0607 →
Continuity (2)
Continuation 17746746 · May 17, 2022
Related Publication 20230376805A1 · Nov 23, 2023
References Cited (8)
US 11568081B2 · Mukhopadhyay · 2023 [cited by examiner]
US 12243420B1 · Rothschild · 2025 [cited by examiner]
US 20200201727A1 · Nie et al. · 2020 [cited by applicant]
US 20210133632A1 · Elprin et al. · 2021 [cited by applicant]
US 20230368043A1 · Macklin · 2023 [cited by examiner]
EP 3422262A1 · 2019 [cited by applicant]
Singh, Mithilesh Kumar, AADGen: Automatic Annotated Data Generation for Training Text Detection and Recognition Models, IEEE International Conference for Innovation in Technology (INOCON), Nov. 2020, 5 pages, [retrieved… [cited by examiner]
Azure Monitor Overview, https://learn.microsoft.com/en-us/azure/azure-monitor/overview, Nov. 1, 2022, 14 pages. [cited by applicant]