Granted Patent B2

US 12,505,315 · App. 17/618,372 · Granted Dec 23, 2025

Inventors: Makoto Morishita (Tokyo, JP); Jun Suzuki (Tokyo, JP); Masaaki Nagata (Tokyo, JP)

Assignee: NTT, Inc.

G06F40/58G06N3/08

Abstract

Claims (34)

1 . An information learning apparatus for learning a model of a neural network that generates, from a first series, a second series and reconstructs the first sequence, the information learning apparatus comprising:

a memory and a processor, wherein

the memory stores parameters of the neural network,

the neural network comprises a first model and a second model,

the first model comprises a first self-attention layer, a first attention layer, a first feed-forward layer, and a first output layer,

the neural network performs:

encoding, based on the first self-attention layer and the first feed-forward layer, first data in training data, by further associating the first data of the first series and second data in the second series with each other, and the second data represents correct data of the first data;

decoding, based on the second model, data generated in the encoding to generate third data related to the second series;

reconstructing, based at least on the first self-attention layer, the first feed-forward layer, and the first output layer, the first series for data generated in the decoding as fourth data; and

learning, based on an error between the second data and the third data and an error between the first data and the fourth data, the parameters of the neural network used by the encoding, the decoding, and the reconstructing, and the learning further comprises updating the parameters of the neural network stored in the memory, wherein

the encoding and the reconstructing operations share parameters of the first self-attention layer and the first feed-forward layer of the neural network, as stored in the memory.

2 . The information learning apparatus according to claim 1 , wherein the neural network further performs:

encoding fifth data in the training data in which the fifth data related to the second series and sixth data which is correct data for the fifth data in the first series are associated with each other, wherein

the reconstructing further comprises decoding data generated in the encoding by the encoding of the fifth data to generate seventh data related to the first series,

the decoding further comprises generating eighth data related to the second series for data generated in the decoding by the generating,

the learning further comprises learning, based on an error between the sixth data and the seventh data and an error between the fifth data and the eighth data, parameters used by the generating, the encoding of the fifth data, and the decoding, and

the decoding and the encoding of the fifth data share at least a part of the parameters of the neural network.

3 . An information processing apparatus that generates an output sentence for an input sentence based on the parameters learned by the learning of the information learning apparatus according to claim 1 , the

the neural network further performs:

encoding the input sentence based on the parameters of the neural network; and

decoding data generated in the encoding based on the parameters to generate the output sentence.

4 . An information learning method performed by a computer including a memory and a processor, the method comprising:

encoding, based on a first self-attention layer of a first model of a neural network and a first feed-forward layer of the first model of the neural network, first data in training data, by further associating the first data of a first series and second data in a second series with each other, and the second data represents correct data of the first data;

decoding, based on a second model of the neural network, data generated in the encoding to generate third data related to the second series;

reconstructing, based at least on the first self-attention layer, the first feed-forward layer, and the first output layer, the first series for data generated in the decoding as fourth data; and

learning, based on an error between the second data and the third data and an error between the first data and the fourth data, parameters of the neural network used by the encoding, the decoding, and the reconstructing, wherein the learning further comprises updating the parameters of the neural network stored in the memory, and

the encoding and the reconstructing operations share parameters of the first self-attention layer and the first feed-forward layer of the neural network, as stored in the memory.

5 . An information processing method of generating an output sentence for an input sentence based on the parameters of the neural network learned by the learning of the information learning apparatus according to claim 1 , the information processing method performed by a computer including a memory and a processor, the method comprising:

encoding the input sentence based on the parameters of the neural network; and

decoding data generated in the encoding based on the parameters of the neural network to generate the output sentence.

6 . A non-transitory computer-readable recording medium having computer-readable instructions stored thereon, which when executed, cause a computer to function as the information learning apparatus according to claim 1 .

7 . A non-transitory computer-readable recording medium having computer-readable instructions stored thereon, which when executed, cause a computer to function as the information processing apparatus according to claim 3 .

8 . The information learning apparatus according to claim 1 , wherein the second series comprises text.

9 . The information learning apparatus according to claim 1 , wherein the neural network performs text-to-speech conversion, document summarization, document parsing, and/or response sentence generation to a given query.

Assignments (2)

Continuity (1)

References Cited (15)