IP Library › Granted Patent US 11,625,557
Granted Patent B2
US 11,625,557 · App. 17/080,673 · Granted Apr 11, 2023

Process to learn new image classes without labels

Inventors: Heiko Hoffmann (Simi Valley, CA); Soheil Kolouri (Agoura Hills, CA)
Assignee: HRL LABORATORIES, LLC
G06K9/6259B60W50/06B60W60/00272G06K9/627G06K9/6215G06K9/6255G06N3/02G06N20/00B60W2420/42
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,625,557
App. No.
17/080,673
Granted
Apr 11, 2023
Kind
B2
Abstract

Described is a system for learning object labels for control of an autonomous platform. Pseudo-task optimization is performed to identify an optimal pseudo-task for each source model of one or more source models. An initial target network is trained using the optimal pseudo-task. Source image components are extracted from source models, and an attribute dictionary of attributes is generated from the source image components. Using zero-shot attribution distillation, the unlabeled target data is aligned with the source models similar to the unlabeled target data. The unlabeled target data are mapped onto attributes in the attribute dictionary. A new target network is generated from the mapping, and the new target network is used to assign an object label to an object in the unlabeled target data. The autonomous platform is controlled based on the object label.

Claims (63)

1. A system for learning object labels for control of an autonomous platform, the system comprising:

one or more processors and a non-transitory computer-readable medium having executable instructions encoded thereon such that when executed, the one or more processors

based on a similarity measure computed between a set of prior trained source models, creating a source similarity graph of similar source models;

performing pseudo-task optimization to identify a pseudo-task for each source model in the source similarity graph, wherein performing pseudo-task optimization comprises:

identifying a pseudo-task for a target data set X T ϵ as a function of {f i : → } i and {α i } i ,

where {α i } i denotes similarities of the target data set to the set of prior trained source models {X i } i such that Σ i α i =1, and

where {f i : → } i denotes corresponding pseudo-tasks for the set of prior trained source models;

combining the pseudo-tasks identified;

using the combined pseudo-tasks and a set of unlabeled target data, training a target network;

using the trained target network, extracting salient components from input image data, resulting in a plurality of source image components having corresponding labels for each source model;

mapping the plurality of source image components and corresponding labels onto sets of abstract attributes, resulting in semantically meaningful clusters of abstract attributes;

aligning, in feature space, new image components from a previously unseen input image with feature space representations of the plurality of source image components;

mapping the new image components onto corresponding abstract attributes;

generating a new target network from the mapping;

using the new target network, assigning an object label to an object in the previously unseen input image; and

controlling the autonomous platform based on the assigned object label.

2. The system as set forth in claim 1 , wherein controlling the autonomous platform further comprises:

generating an executable control script appropriate for the object label; and

causing the autonomous platform to execute the control script and perform an action corresponding to the control script.

3. The system as set forth in claim 1 , wherein the salient components are extracted using unsupervised data decomposition.

4. The system as set forth in claim 1 , wherein the autonomous platform is a vehicle, and wherein the one or more processors further perform an operation of causing the vehicle to perform a driving operation in accordance with the assigned object label.

5. A computer implemented method for learning object labels for control of an autonomous platform, the method comprising an act of:

causing one or more processors to execute instructions encoded on a non-transitory computer-readable medium, such that upon execution, the one or more processors perform operations of:

based on a similarity measure computed between a set of prior trained source models, creating a source similarity graph of similar source models;

performing pseudo-task optimization to identify a pseudo-task for each source model in the source similarity graph, wherein performing pseudo-task optimization comprises:

identifying a pseudo-task for a target data set X T ϵ as a function of {f i : → } i and {α i } i ,

where {α i } i denotes similarities of the target data set to the set of prior trained source models {X i } i such that Σ i α i =1, and

where {f i : → } i denotes corresponding pseudo-tasks for the set of prior trained source models;

combining the pseudo-tasks identified;

using the combined pseudo-tasks and a set of unlabeled target data, training a target network;

using the trained target network, extracting salient components from input image data, resulting in a plurality of source image components having corresponding labels for each source model;

mapping the plurality of source image components and corresponding labels onto sets of abstract attributes, resulting in semantically meaningful clusters of abstract attributes;

aligning, in feature space, new image components from a previously unseen input image with feature space representations of the plurality of source image components;

mapping the new image components onto corresponding abstract attributes;

generating a new target network from the mapping;

using the new target network, assigning an object label to an object in the previously unseen input image; and

controlling the autonomous platform based on the assigned object label.

6. The method as set forth in claim 5 , wherein controlling the autonomous platform further comprises:

generating an executable control script appropriate for the object label; and

causing the autonomous platform to execute the control script and perform an action corresponding to the control script.

7. The method as set forth in claim 5 , wherein the salient components are extracted using unsupervised data decomposition.

8. The method as set forth in claim 5 , wherein the autonomous platform is a vehicle, and wherein the one or more processors further perform an operation of causing the vehicle to perform a driving operation in accordance with the assigned object label.

9. A computer program product for learning object labels for control of an autonomous platform, the computer program product comprising:

computer-readable instructions stored on a non-transitory computer-readable medium that are executable by a computer having one or more processors for causing the processor to perform operations of:

based on a similarity measure computed between a set of prior trained source models, creating a source similarity graph of similar source models;

performing pseudo-task optimization to identify a pseudo-task for each source model in the source similarity graph, wherein performing pseudo-task optimization comprises:

identifying a pseudo-task for a target data set X T ϵ as a function of {f i : → } i and {α i } i ,

where {α i } i denotes similarities of the target data set to the set of prior trained source models {X i } i such that Σ i α i =1, and

where {f i : → } i denotes corresponding pseudo-tasks for the set of prior trained source models;

combining the pseudo-tasks identified;

using the combined pseudo-tasks and a set of unlabeled target data, training a target network;

using the trained target network, extracting salient components from input image data, resulting in a plurality of source image components having corresponding labels for each source model;

mapping the plurality of source image components and corresponding labels onto sets of abstract attributes, resulting in semantically meaningful clusters of abstract attributes;

aligning, in feature space, new image components from a previously unseen input image with feature space representations of the plurality of source image components;

mapping the new image components onto corresponding abstract attributes;

generating a new target network from the mapping;

using the new target network, assigning an object label to an object in the previously unseen input image; and

controlling the autonomous platform based on the assigned object label.

10. The computer program product as set forth in claim 9 , wherein controlling the autonomous platform further comprises:

generating an executable control script appropriate for the object label; and

causing the autonomous platform to execute the control script and perform an action corresponding to the control script.

11. The computer program product as set forth in claim 9 , wherein the salient components are extracted using unsupervised data decomposition.

12. The computer program product as set forth in claim 9 , wherein the autonomous platform is a vehicle, and wherein controlling the autonomous platform further comprises causing the vehicle to perform a driving operation in accordance with the assigned object label.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 26, 2020
From: HOFFMANN, HEIKO; KOLOURI, SOHEIL
To: HRL LABORATORIES, LLC
Reel/Frame 054170/0284 →
Continuity (4)
Continuation In Part 16532321 · Aug 5, 2019
Provisional Application 62946277 · Dec 10, 2019
Provisional Application 62752166 · Oct 29, 2018
Related Publication 20210182618A1 · Jun 17, 2021
Cited By (1)
US 12,566,244