IP Library › Granted Patent US 11,429,856
Granted Patent B2
US 11,429,856 · App. 16/128,614 · Granted Aug 30, 2022

Neural networks adaptive boosting using semi-supervised learning

Inventors: Jamal Hammoud (Paris, FR); Marc Joel Herve Legroux (de Strasbourg Angers, FR)
Assignee: International Business Machines Corporation
G06N3/08G06N3/04
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,429,856
App. No.
16/128,614
Granted
Aug 30, 2022
Kind
B2
Abstract

An approach for generating a trained neural network is provided. In an embodiment, a neural network, which can have an input layer, an output layer, and a hidden layer, is created. An initial training of the neural network is performed using a set of labeled data. The boosted neural network resulting from the initial training is applied to unlabeled data to determine whether any of the unlabeled data qualifies as additional labeled data. If it is determined that any of the unlabeled data qualifies as additional labeled data, the boosted neural network is retrained using the additional labeled data. Otherwise, if it is determined that none of the unlabeled data qualifies as additional labeled data, the neural network is updated to change a number of predictor nodes in the neural network.

Claims (55)

1. A method for generating a trained neural network, comprising:

creating a neural network;

performing an initial training of the neural network using a set of labeled data;

performing a plurality of iterations in which a boosted neural network is applied to unlabeled data to determine whether any of the unlabeled data qualifies as additional labeled data;

retraining, in response to performing of an iteration in which any of the unlabeled data qualifies as additional labeled data, the boosted neural network using the additional labeled data; and

updating, in response to the performing of an iteration in which none of the unlabeled data qualifies as additional labeled data, the neural network to change a number of predictor nodes in the neural network, the updating further comprising removing at least one predictor node from a hidden layer of the neural network in response to a determination that the number of predictor nodes is greater than a predetermined number.

2. The method of claim 1 , wherein the neural network includes an input layer, an output layer, and a hidden layer.

3. The method of claim 2 , wherein the at least one predictor node includes a predictor node with a lowest filling rate.

4. The method of claim 2 , the updating further comprising: re-adding at least one node to the hidden layer of the neural network in response to a determination that the number of predictor nodes has reached a predetermined number.

5. The method of claim 4 , further comprising executing a final boosting on the neural network to finalize a structure of the neural network in response to the re-adding.

6. The method of claim 2 , wherein the initial training includes, for each labeled instance of the set of labeled data:

introducing the labeled instance at the input layer of the neural network;

evaluating the labeled instance by at least one node of the hidden layer;

outputting a solution at the output layer based on the evaluation;

comparing the solution with a labeled solution associated with the labeled instance; and

weighting the at least one node based on a result of the comparing to get the boosted neural network.

7. The method of claim 1 , further comprising:

determining, for each unlabeled instance of the unlabeled data, whether an outputted solution by the neural network is accurate;

labeling, in response to a determination that the solution is accurate, the unlabeled instance with the outputted solution to yield a labeled instance; and

adding the labeled instance to the set of labeled data.

8. A computer program product for generating a trained neural network, the computer program product comprising a computer readable storage media, and program instructions stored on the computer readable storage media, that cause at least one computer device to:

create a neural network;

perform an initial training of the neural network using a set of labeled data;

apply a boosted neural network to unlabeled data to determine whether any of the unlabeled data qualifies as additional labeled data;

retrain, in response to performing of an iteration in which any of the unlabeled data qualifies as additional labeled data, the boosted neural network using the additional labeled data; and

update, in response to the performing of an iteration in which none of the unlabeled data qualifies as additional labeled data, the neural network to change a number of predictor nodes in the neural network, the update further causing the at least one computer device to remove at least one predictor node from a hidden layer of the neural network in response to a determination that the number of predictor nodes is greater than a predetermined number.

9. The computer program product of claim 8 , wherein the neural network includes an input layer, an output layer, and a hidden layer.

10. The computer program product of claim 9 , wherein the at least one predictor node includes a predictor node with a lowest filling rate.

11. The computer program product of claim 9 , the instructions that cause the at least one computer device to update further causing the at least one computer device to re-add at least one node to the hidden layer of the neural network in response to a determination that the number of predictor nodes has reached a predetermined number.

12. The computer program product of claim 11 , the instructions further causing the at least one computer device to execute a final boosting on the neural network to finalize a structure of the neural network in response to the re-adding.

13. The computer program product of claim 9 , wherein the initial training includes, for each labeled instance of the set of labeled data:

introducing the labeled instance at the input layer of the neural network;

evaluating the labeled instance by at least one node of the hidden layer;

outputting a solution at the output layer based on the evaluation;

comparing the solution with a labeled solution associated with the labeled instance; and

weighting the at least one node based on a result of the comparing to get the boosted neural network.

14. The computer program product of claim 8 , the instructions further causing the at least one computer device to parse, prior to forming a machine language model, annotated documents to remove from a document unannotated portions of the document.

15. A system for generating a trained neural network, comprising:

a neural network having an input layer, an output layer, and a hidden layer;

a memory medium comprising instructions;

a bus coupled to the memory medium; and

a processor coupled to the bus that when executing the instructions causes the system to:

perform an initial training of the neural network using a set of labeled data;

apply a boosted neural network to unlabeled data to determine whether any of the unlabeled data qualifies as additional labeled data;

retrain, in response to performing of an iteration in which any of the unlabeled data qualifies as additional labeled data, the boosted neural network using the additional labeled data; and

update, in response to the performing of an iteration in which none of the unlabeled data qualifies as additional labeled data, the neural network to change a number of predictor nodes in the neural network, the update further causing the system to remove at least one predictor node from a hidden layer of the neural network in response to a determination that the number of predictor nodes is greater than a predetermined number.

16. The system of claim 15 , the instructions that cause the system to update further causing the system to re-add at least one node to the hidden layer of the neural network in response to a determination that the number of predictor nodes has reached a predetermined number, the at least one predictor node includes a predictor node with a lowest filling rate.

17. The system of claim 16 , the instructions further causing the at least one computer device to executing a final boosting on the neural network to finalize a structure of the neural network in response to the re-adding.

18. The system of claim 15 , wherein the initial training includes, for each labeled instance of the set of labeled data:

introducing the labeled instance at the input layer of the neural network;

evaluating the labeled instance by at least one node of the hidden layer;

outputting a solution at the output layer based on the evaluation;

comparing the solution with a labeled solution associated with the labeled instance; and

weighting the at least one node based on a result of the comparing to get the boosted neural network.

19. The system of claim 15 , the instructions further causing the system to train the artificial intelligence using a machine learning model.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 12, 2018
From: HAMMOUD, JAMAL; LEGROUX, MARC JOEL HERVE
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 046847/0586 →
Continuity (1)
Related Publication 20200082260A1 · Mar 12, 2020