IP Library Granted Patent US 12,165,028
Granted Patent B2
US 12,165,028 · App. 16/940,925 · Granted Dec 10, 2024

Extracting sequences from d-dimensional input data for sequential processing with neural networks

Inventors: Lorenz K. Muller (Dubendorf, CH); Pascal Stark (Thalwil, CH); Stefan Abel (Zurich, CH)
Assignee: International Business Machines Corporation
G06N20/20G06F17/18G06F18/24G06N3/04G06N3/065G06N3/08G06N5/04G06V10/44G06N3/044G06N3/045G06N3/049G06V10/82G06V30/10
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,165,028
App. No.
16/940,925
Granted
Dec 10, 2024
Kind
B2
Abstract

A method and computer program product for obtaining values are run using a neural network according to a machine learning algorithm. One embodiment may comprise accessing one or more datafiles of input data, where the input data is representable in a d-dimensional space, with d>1. The method may explore N distinct paths of the input data in the d-dimensional space, where N≥1, and collects data along the N distinct paths explored to respectively form N sequences of M objects each, with M≥2. For one or more sequences of the N sequences formed, values obtained from the M objects of each sequence may be coupled into one or more input nodes of a neural network, which is then run according to the machine learning algorithm to obtain L output values from, L≥1.

Claims (53)

1. A computer-implemented method of obtaining values by running a neural network according to a machine learning algorithm designed to operate on sequences of data, the method comprising:

accessing one or more datafiles of input data representable in a d-dimensional space, wherein d>1;

by a processor, forming N sequences of M objects each by exploring N distinct paths of said input data in the d-dimensional space and collecting data along the N distinct paths explored, wherein N≥1 and M≥2;

for one or more sequences of the N sequences formed, by the processor, coupling values obtained from the M objects of each sequence into one or more input nodes of a neural network and running the neural network according to said machine learning algorithm to obtain L output values for each sequence of the N sequences, wherein L≥1, and wherein N sets of L output values are obtained for the N sequences;

by the processor, training parameters of a cognitive model using the N sets of L output values; and

by the processor, performing N inferences for the N sequences using the cognitive model after training.

2. The computer-implemented method according to claim 1 , further comprising:

prior to coupling said values, setting, for each of the N sequences formed, the network according to a given set of parameters of a cognitive model.

3. The computer-implemented method according to claim 2 , further comprising:

transmitting the prediction or the classification based on the N inferences performed.

4. The computer-implemented method according to claim 1 , wherein:

N>1 and the method further comprises averaging out outcomes from said N inferences.

5. The computer-implemented method according to claim 1 , wherein:

said machine learning algorithm is a cognitive algorithm and the network is run according to said cognitive algorithm to train a cognitive model in a supervised manner with respect to a same set of L target output values for each of the N sequences formed, permitting the cognitive algorithm to learn N sets of parameters of the cognitive model based on the same set of L target output values.

6. The computer-implemented method according to claim 5 , wherein:

N>1 and the method further comprises obtaining an ensemble model based on the N sets of parameters learned.

7. The computer-implemented method according to claim 6 , wherein the method further comprises, after having obtained the ensemble model, performing inferences by:

accessing one or more further datafiles of further input data representable in the d-dimensional space;

exploring N′ distinct paths of said further input data in the d-dimensional space, wherein the N′ distinct paths are selected from the N distinct paths previously explored, 1≤N′≤N, and, based on the further input data, collecting data along the N′ distinct paths explored to respectively form N′ sequences of M objects each; and

for each sequence of the N′ sequences formed, coupling values obtained from the M objects of each of the N′ sequences formed into the one or more input nodes of the neural network and running the latter to obtain L output values, to eventually obtain N′ sets of L output values.

8. The computer-implemented method according to claim 7 , wherein:

the cognitive model obtained includes a set of N base learners, each corresponding to a respective one of the N distinct paths; and

the method further comprises selecting N′ base learners corresponding to respective ones of the N′ distinct paths selected and assigning each of the N′ base learners selected to a respective one of the N′ sequences formed.

9. The computer-implemented method according to claim 5 , wherein the method further comprises, while the network is being run to train the cognitive model, and for each sequence of one or more of the N sequences formed:

pausing the cognitive algorithm that is being trained; and

coupling values obtained from the M objects of distinct one of the N sequences formed into the one or more input nodes of the network and resuming the training of the cognitive algorithm.

10. The computer-implemented method according to claim 1 , further comprising:

devising each path of said N distinct data paths according to a random walk.

11. The computer-implemented method according to claim 10 , wherein:

at devising said each path, step sizes of said random walk are chosen at random, in addition to said directions.

12. The computer-implemented method according to claim 10 , wherein:

devising said random walk comprises, at each step j thereof, j≥2, generating a j th direction at random in said d-dimensional space, merging the j th direction generated with the j−1 th direction as generated at step j−1 of the walk to obtain a merged direction, and advancing the walk by one step in the merged direction.

13. The computer-implemented method according to claim 1 , further comprising:

devising each path of said N distinct path according to a random shot.

14. The computer-implemented method according to claim 1 , wherein:

the network is run according to a recurrent neural network algorithm.

15. The computer-implemented method according to claim 1 , wherein:

the network is run according to an echo state network algorithm.

16. The computer-implemented method according to claim 1 , wherein:

the network is run according to a feedforward neural network algorithm.

17. The computer-implemented method according to claim 1 , wherein:

each of the M objects of each of the N sequences is a k-dimensional object, k≥1, comprising N k values, N k ≥2.

18. The computer-implemented method according to claim 17 , wherein:

the network comprises N k input nodes; and

the method further comprises, for each of the N sequences, sequentially coupling M sets of values into said N k input nodes, in order to sequentially run the network for each of M objects of each of the N sequences, according to a sequential processing algorithm.

19. The computer-implemented method according to claim 17 , wherein:

k≥2.

20. A computer program product for obtaining values by running a neural network according to a machine learning algorithm designed to operate on sequences of data, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor, so as to cause the processor to:

access one or more datafiles of input data representable in a d-dimensional space, d>1;

explore N distinct paths of said input data in the d-dimensional space, N≥1, and collect data along the N distinct paths explored to respectively form N sequences of M objects each, M≥2;

for each sequence of the N sequences formed, couple values obtained from the M objects of said each sequence into one or more input nodes of a neural network and run the neural network according to said machine learning algorithm to obtain L output values for each sequence of the N sequences, L≥1, and wherein N sets of L output values are obtained for the N sequences;

by the processor, training parameters of a cognitive model using the N sets of L output values; and

by the processor, performing N inferences for the N sequences using the cognitive model after training.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 28, 2020
From: MULLER, LORENZ K.; STARK, PASCAL; ABEL, STEFAN
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 053330/0807 →
Continuity (1)
Related Publication 20220036245A1 · Feb 3, 2022