IP Library › Granted Patent US 11,875,270
Granted Patent B2
US 11,875,270 · App. 17/114,957 · Granted Jan 16, 2024

Adversarial semi-supervised one-shot learning

Inventors: Takayuki Katsuki (Tokyo, JP); Takayuki Osogami (Yamato, JP)
Assignee: International Business Machines Corporation
G06N3/088G06N3/045G06N3/047
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,875,270
App. No.
17/114,957
Granted
Jan 16, 2024
Kind
B2
Abstract

A method, a computer program product, and a system of adversarial semi-supervised one-shot training using a data stream. The method includes receiving a data stream based on an observation, wherein the data stream includes unlabeled data and labeled data. The method also includes training a prediction model with the labeled data using stochastic gradient descent based on a classification loss and an adversarial term and training a representation model with the labeled data and the unlabeled data based on a reconstruction loss and the adversarial term. The adversarial term is a cross-entropy between the middle layer output data from the models. The classification loss is a cross-entropy between the labeled data and an output from the prediction model. The method further includes updating a discriminator with middle layer output data from the prediction model and the representation model and based on a discrimination loss, and discarding the data stream.

Claims (37)

1. A computer-implemented method of adversarial semi-supervised one-shot training using a data stream, the computer-implemented method comprising:

receiving a data stream based on an observation, wherein the data stream includes unlabeled data and labeled data;

training a prediction model with the labeled data using stochastic gradient descent based on a classification loss and an adversarial term;

training a representation model with the labeled data and the unlabeled data based on a reconstruction loss and the adversarial term;

updating a discriminator with middle layer output data from the prediction model and the representation model and based on a discrimination loss; and

discarding the data stream.

2. The computer-implemented method of claim 1 , wherein the classification loss is a cross-entropy between the labeled data and a prediction output from the prediction model.

3. The computer-implemented method of claim 1 , wherein the adversarial term is a cross-entropy between the middle layer output data from the prediction model and the representation model.

4. The computer-implemented method of claim 1 , wherein the reconstruction loss is a squared loss between the data stream and a reconstruction output by the representation model.

5. The computer-implemented method of claim 1 , wherein the discrimination loss is a cross-entropy between the middle layer output data of the prediction model and the representation model.

6. The computer-implemented method of claim 1 , wherein the representation model is an autoencoder based on a neural network.

7. The computer-implemented method of claim 1 , wherein the prediction model is a neural network with a bottom layer as an encoder.

8. The computer-implemented method of claim 1 , wherein the discriminator is a neural network configured to discriminate between middle layer outputs of the prediction model and middle layer outputs of the representation model.

9. A system of adversarial semi-supervised one-shot training using a data stream, the system comprising:

a memory;

a processor;

local data storage having stored thereon computer executable code;

a prediction model configured to predict a vehicle behavior, wherein the prediction model is trained with labeled data from a data stream using stochastic gradient descent based on a classification loss and an adversarial term;

a representation model configured to predict the vehicle behavior, wherein the representation model is trained with unlabeled data and the labeled data from the data stream based on a reconstruction loss and the adversarial term; and

a discriminator configured to predict predictions made by the prediction model and the representation model, wherein the discriminator is trained with middle layer output data from the prediction model and the representation model and based on a discrimination loss.

10. The system of claim 9 , wherein the classification loss is a cross-entropy between the labeled data and a prediction output from the prediction model.

11. The system of claim 9 , wherein the adversarial term is a cross-entropy between the middle layer output data from the prediction model and the representation model.

12. The system of claim 9 , wherein the reconstruction loss is a squared loss between the data stream and a reconstruction output by the representation model.

13. The system of claim 9 , wherein the discrimination loss is a cross-entropy between the middle layer output data of the prediction model and the representation model.

14. The system of claim 9 , wherein the representation model is an autoencoder based on a neural network.

15. The system of claim 9 , wherein the prediction model is a neural network with a bottom layer as an encoder.

16. The system of claim 9 , wherein the discriminator is a neural network configured to discriminate between outputs of middle layers of the prediction model and middle layers of the representation model.

17. A computer program product for adversarial semi-supervised one-shot training using a data stream, the computer program product comprising:

one or more computer readable storage medium, and program instructions stored on the one or more computer readable storage media, the program instructions comprising:

program instructions to receive a data stream based on an observation, wherein the data stream includes unlabeled data and labeled data;

program instructions to train a prediction model with the labeled data using stochastic gradient descent based on a classification loss and an adversarial term;

program instructions to train a representation model with the labeled data and the unlabeled data based on a reconstruction loss and the adversarial term;

program instructions to update a discriminator with middle layer output data from the prediction model and the representation model and based on a discrimination loss; and

program instructions to discard the data stream.

18. The computer program product of claim 17 , wherein the classification loss is a cross-entropy between the labeled data and a prediction output from the prediction model.

19. The computer program product of claim 17 , wherein the adversarial term is a cross-entropy between the middle layer output data from the prediction model and the representation model.

20. The computer program product of claim 17 , wherein the reconstruction loss is a squared loss between the data stream and a reconstruction output by the representation model.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 8, 2020
From: KATSUKI, TAKAYUKI; OSOGAMI, TAKAYUKI
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 054576/0879 →
Continuity (1)
Related Publication 20220180204A1 · Jun 9, 2022